[==========] Running 5 tests from 1 test suite.
[----------] Global test environment set-up.
[----------] 5 tests from TabletCopyClientSessionITest
[ RUN ] TabletCopyClientSessionITest.TestStartTabletCopyWhileSourceBootstrapping
2024-05-03T02:14:02Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2024-05-03T02:14:02Z Disabled control of system clock
WARNING: Logging before InitGoogleLogging() is written to STDERR
I20240503 02:14:02.585111 9822 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknrxqHk/build/asan/bin/kudu
/tmp/dist-test-tasknrxqHk/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknrxqHk/test-tmp/tablet_copy_client_session-itest.0.TabletCopyClientSessionITest.TestStartTabletCopyWhileSourceBootstrapping.1714702442467385-9822-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasknrxqHk/test-tmp/tablet_copy_client_session-itest.0.TabletCopyClientSessionITest.TestStartTabletCopyWhileSourceBootstrapping.1714702442467385-9822-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknrxqHk/test-tmp/tablet_copy_client_session-itest.0.TabletCopyClientSessionITest.TestStartTabletCopyWhileSourceBootstrapping.1714702442467385-9822-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasknrxqHk/test-tmp/tablet_copy_client_session-itest.0.TabletCopyClientSessionITest.TestStartTabletCopyWhileSourceBootstrapping.1714702442467385-9822-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.151.190:36085
--webserver_interface=127.9.151.190
--webserver_port=0
--builtin_ntp_servers=127.9.151.148:38865
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.151.190:36085
--master_tombstone_evicted_tablet_replicas=false with env {}
W20240503 02:14:02.953114 9830 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240503 02:14:02.953475 9830 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240503 02:14:02.953615 9830 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240503 02:14:02.961632 9830 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20240503 02:14:02.961803 9830 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240503 02:14:02.961885 9830 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20240503 02:14:02.961967 9830 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20240503 02:14:02.973287 9830 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.151.148:38865
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknrxqHk/test-tmp/tablet_copy_client_session-itest.0.TabletCopyClientSessionITest.TestStartTabletCopyWhileSourceBootstrapping.1714702442467385-9822-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasknrxqHk/test-tmp/tablet_copy_client_session-itest.0.TabletCopyClientSessionITest.TestStartTabletCopyWhileSourceBootstrapping.1714702442467385-9822-0/minicluster-data/master-0/wal
--master_tombstone_evicted_tablet_replicas=false
--ipki_ca_key_size=768
--master_addresses=127.9.151.190:36085
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.151.190:36085
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknrxqHk/test-tmp/tablet_copy_client_session-itest.0.TabletCopyClientSessionITest.TestStartTabletCopyWhileSourceBootstrapping.1714702442467385-9822-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.151.190
--webserver_port=0
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasknrxqHk/test-tmp/tablet_copy_client_session-itest.0.TabletCopyClientSessionITest.TestStartTabletCopyWhileSourceBootstrapping.1714702442467385-9822-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c39e72a98e124e5a12080bac42105b996594d806
build type FASTDEBUG
built by None at 03 May 2024 01:43:22 UTC on efd1337a2aa3
build id 99
ASAN enabled
I20240503 02:14:02.975320 9830 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240503 02:14:02.977452 9830 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240503 02:14:02.990229 9835 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240503 02:14:02.990859 9838 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240503 02:14:02.991448 9836 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240503 02:14:02.992105 9830 server_base.cc:1034] running on GCE node
I20240503 02:14:02.993026 9830 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240503 02:14:02.994760 9830 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240503 02:14:02.996008 9830 hybrid_clock.cc:648] HybridClock initialized: now 1714702442995930 us; error 63 us; skew 500 ppm
I20240503 02:14:02.996378 9830 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20240503 02:14:03.003448 9830 webserver.cc:460] Webserver started at http://127.9.151.190:43689/ using document root <none> and password file <none>
I20240503 02:14:03.004381 9830 fs_manager.cc:359] Metadata directory not provided
I20240503 02:14:03.004532 9830 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240503 02:14:03.004904 9830 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240503 02:14:03.008121 9830 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknrxqHk/test-tmp/tablet_copy_client_session-itest.0.TabletCopyClientSessionITest.TestStartTabletCopyWhileSourceBootstrapping.1714702442467385-9822-0/minicluster-data/master-0/data/instance:
uuid: "9515dc4386d5467290be9ed534098d9b"
format_stamp: "Formatted at 2024-05-03 02:14:02 on dist-test-slave-twlh"
I20240503 02:14:03.009537 9830 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknrxqHk/test-tmp/tablet_copy_client_session-itest.0.TabletCopyClientSessionITest.TestStartTabletCopyWhileSourceBootstrapping.1714702442467385-9822-0/minicluster-data/master-0/wal/instance:
uuid: "9515dc4386d5467290be9ed534098d9b"
format_stamp: "Formatted at 2024-05-03 02:14:02 on dist-test-slave-twlh"
I20240503 02:14:03.015867 9830 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.008s sys 0.000s
I20240503 02:14:03.020184 9844 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240503 02:14:03.021994 9830 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.001s
I20240503 02:14:03.022199 9830 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknrxqHk/test-tmp/tablet_copy_client_session-itest.0.TabletCopyClientSessionITest.TestStartTabletCopyWhileSourceBootstrapping.1714702442467385-9822-0/minicluster-data/master-0/data,/tmp/dist-test-tasknrxqHk/test-tmp/tablet_copy_client_session-itest.0.TabletCopyClientSessionITest.TestStartTabletCopyWhileSourceBootstrapping.1714702442467385-9822-0/minicluster-data/master-0/wal
uuid: "9515dc4386d5467290be9ed534098d9b"
format_stamp: "Formatted at 2024-05-03 02:14:02 on dist-test-slave-twlh"
I20240503 02:14:03.022459 9830 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknrxqHk/test-tmp/tablet_copy_client_session-itest.0.TabletCopyClientSessionITest.TestStartTabletCopyWhileSourceBootstrapping.1714702442467385-9822-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasknrxqHk/test-tmp/tablet_copy_client_session-itest.0.TabletCopyClientSessionITest.TestStartTabletCopyWhileSourceBootstrapping.1714702442467385-9822-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasknrxqHk/test-tmp/tablet_copy_client_session-itest.0.TabletCopyClientSessionITest.TestStartTabletCopyWhileSourceBootstrapping.1714702442467385-9822-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240503 02:14:03.049718 9830 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240503 02:14:03.050732 9830 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240503 02:14:03.051061 9830 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240503 02:14:03.072727 9830 rpc_server.cc:307] RPC server started. Bound to: 127.9.151.190:36085
I20240503 02:14:03.072772 9895 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.151.190:36085 every 8 connection(s)
I20240503 02:14:03.074322 9830 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknrxqHk/test-tmp/tablet_copy_client_session-itest.0.TabletCopyClientSessionITest.TestStartTabletCopyWhileSourceBootstrapping.1714702442467385-9822-0/minicluster-data/master-0/data/info.pb
I20240503 02:14:03.078869 9896 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:14:03.078933 9822 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknrxqHk/build/asan/bin/kudu as pid 9830
I20240503 02:14:03.079218 9822 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknrxqHk/test-tmp/tablet_copy_client_session-itest.0.TabletCopyClientSessionITest.TestStartTabletCopyWhileSourceBootstrapping.1714702442467385-9822-0/minicluster-data/master-0/wal/instance
I20240503 02:14:03.090744 9896 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9515dc4386d5467290be9ed534098d9b: Bootstrap starting.
I20240503 02:14:03.094167 9896 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9515dc4386d5467290be9ed534098d9b: Neither blocks nor log segments found. Creating new log.
I20240503 02:14:03.095669 9896 log.cc:824] T 00000000000000000000000000000000 P 9515dc4386d5467290be9ed534098d9b: Log is configured to *not* fsync() on all Append() calls
I20240503 02:14:03.099957 9896 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9515dc4386d5467290be9ed534098d9b: No bootstrap required, opened a new log
I20240503 02:14:03.107491 9896 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9515dc4386d5467290be9ed534098d9b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9515dc4386d5467290be9ed534098d9b" member_type: VOTER last_known_addr { host: "127.9.151.190" port: 36085 } }
I20240503 02:14:03.108057 9896 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9515dc4386d5467290be9ed534098d9b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:14:03.108175 9896 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9515dc4386d5467290be9ed534098d9b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9515dc4386d5467290be9ed534098d9b, State: Initialized, Role: FOLLOWER
I20240503 02:14:03.108876 9896 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9515dc4386d5467290be9ed534098d9b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9515dc4386d5467290be9ed534098d9b" member_type: VOTER last_known_addr { host: "127.9.151.190" port: 36085 } }
I20240503 02:14:03.109143 9896 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9515dc4386d5467290be9ed534098d9b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20240503 02:14:03.109274 9896 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9515dc4386d5467290be9ed534098d9b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20240503 02:14:03.109436 9896 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9515dc4386d5467290be9ed534098d9b [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:14:03.112382 9896 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9515dc4386d5467290be9ed534098d9b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9515dc4386d5467290be9ed534098d9b" member_type: VOTER last_known_addr { host: "127.9.151.190" port: 36085 } }
I20240503 02:14:03.112953 9896 leader_election.cc:302] T 00000000000000000000000000000000 P 9515dc4386d5467290be9ed534098d9b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9515dc4386d5467290be9ed534098d9b; no voters:
I20240503 02:14:03.113664 9896 leader_election.cc:288] T 00000000000000000000000000000000 P 9515dc4386d5467290be9ed534098d9b [CANDIDATE]: Term 1 election: Requested vote from peers
I20240503 02:14:03.113826 9901 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9515dc4386d5467290be9ed534098d9b [term 1 FOLLOWER]: Leader election won for term 1
I20240503 02:14:03.114511 9901 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9515dc4386d5467290be9ed534098d9b [term 1 LEADER]: Becoming Leader. State: Replica: 9515dc4386d5467290be9ed534098d9b, State: Running, Role: LEADER
I20240503 02:14:03.115160 9901 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9515dc4386d5467290be9ed534098d9b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9515dc4386d5467290be9ed534098d9b" member_type: VOTER last_known_addr { host: "127.9.151.190" port: 36085 } }
I20240503 02:14:03.115926 9896 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9515dc4386d5467290be9ed534098d9b [sys.catalog]: configured and running, proceeding with master startup.
I20240503 02:14:03.118283 9903 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9515dc4386d5467290be9ed534098d9b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9515dc4386d5467290be9ed534098d9b. Latest consensus state: current_term: 1 leader_uuid: "9515dc4386d5467290be9ed534098d9b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9515dc4386d5467290be9ed534098d9b" member_type: VOTER last_known_addr { host: "127.9.151.190" port: 36085 } } }
I20240503 02:14:03.118727 9903 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9515dc4386d5467290be9ed534098d9b [sys.catalog]: This master's current role is: LEADER
I20240503 02:14:03.118731 9902 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9515dc4386d5467290be9ed534098d9b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9515dc4386d5467290be9ed534098d9b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9515dc4386d5467290be9ed534098d9b" member_type: VOTER last_known_addr { host: "127.9.151.190" port: 36085 } } }
I20240503 02:14:03.119421 9902 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9515dc4386d5467290be9ed534098d9b [sys.catalog]: This master's current role is: LEADER
I20240503 02:14:03.122836 9906 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20240503 02:14:03.129393 9906 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20240503 02:14:03.141072 9906 catalog_manager.cc:1347] Generated new cluster ID: 5b1f9b52b5d1438295da79db46b7114b
I20240503 02:14:03.141240 9906 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20240503 02:14:03.159575 9906 catalog_manager.cc:1370] Generated new certificate authority record
I20240503 02:14:03.160970 9906 catalog_manager.cc:1504] Loading token signing keys...
I20240503 02:14:03.172068 9906 catalog_manager.cc:5857] T 00000000000000000000000000000000 P 9515dc4386d5467290be9ed534098d9b: Generated new TSK 0
I20240503 02:14:03.173339 9906 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20240503 02:14:03.180984 9822 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknrxqHk/build/asan/bin/kudu
/tmp/dist-test-tasknrxqHk/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknrxqHk/test-tmp/tablet_copy_client_session-itest.0.TabletCopyClientSessionITest.TestStartTabletCopyWhileSourceBootstrapping.1714702442467385-9822-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasknrxqHk/test-tmp/tablet_copy_client_session-itest.0.TabletCopyClientSessionITest.TestStartTabletCopyWhileSourceBootstrapping.1714702442467385-9822-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknrxqHk/test-tmp/tablet_copy_client_session-itest.0.TabletCopyClientSessionITest.TestStartTabletCopyWhileSourceBootstrapping.1714702442467385-9822-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasknrxqHk/test-tmp/tablet_copy_client_session-itest.0.TabletCopyClientSessionITest.TestStartTabletCopyWhileSourceBootstrapping.1714702442467385-9822-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.151.129:0
--local_ip_for_outbound_sockets=127.9.151.129
--webserver_interface=127.9.151.129
--webserver_port=0
--tserver_master_addrs=127.9.151.190:36085
--builtin_ntp_servers=127.9.151.148:38865
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin with env {}
W20240503 02:14:03.552588 9920 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240503 02:14:03.552975 9920 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240503 02:14:03.553114 9920 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240503 02:14:03.561678 9920 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240503 02:14:03.562002 9920 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.151.129
I20240503 02:14:03.572959 9920 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.151.148:38865
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknrxqHk/test-tmp/tablet_copy_client_session-itest.0.TabletCopyClientSessionITest.TestStartTabletCopyWhileSourceBootstrapping.1714702442467385-9822-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasknrxqHk/test-tmp/tablet_copy_client_session-itest.0.TabletCopyClientSessionITest.TestStartTabletCopyWhileSourceBootstrapping.1714702442467385-9822-0/minicluster-data/ts-0/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.151.129:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknrxqHk/test-tmp/tablet_copy_client_session-itest.0.TabletCopyClientSessionITest.TestStartTabletCopyWhileSourceBootstrapping.1714702442467385-9822-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.151.129
--webserver_port=0
--tserver_master_addrs=127.9.151.190:36085
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.151.129
--log_dir=/tmp/dist-test-tasknrxqHk/test-tmp/tablet_copy_client_session-itest.0.TabletCopyClientSessionITest.TestStartTabletCopyWhileSourceBootstrapping.1714702442467385-9822-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c39e72a98e124e5a12080bac42105b996594d806
build type FASTDEBUG
built by None at 03 May 2024 01:43:22 UTC on efd1337a2aa3
build id 99
ASAN enabled
I20240503 02:14:03.574541 9920 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240503 02:14:03.577188 9920 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240503 02:14:03.593173 9925 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240503 02:14:03.593744 9926 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240503 02:14:03.594364 9928 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240503 02:14:03.595211 9920 server_base.cc:1034] running on GCE node
I20240503 02:14:03.595947 9920 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240503 02:14:03.597092 9920 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240503 02:14:03.598484 9920 hybrid_clock.cc:648] HybridClock initialized: now 1714702443598363 us; error 47 us; skew 500 ppm
I20240503 02:14:03.599027 9920 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20240503 02:14:03.602084 9920 webserver.cc:460] Webserver started at http://127.9.151.129:43885/ using document root <none> and password file <none>
I20240503 02:14:03.603058 9920 fs_manager.cc:359] Metadata directory not provided
I20240503 02:14:03.603240 9920 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240503 02:14:03.603571 9920 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240503 02:14:03.606600 9920 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknrxqHk/test-tmp/tablet_copy_client_session-itest.0.TabletCopyClientSessionITest.TestStartTabletCopyWhileSourceBootstrapping.1714702442467385-9822-0/minicluster-data/ts-0/data/instance:
uuid: "7cd9a8da65604af78132e9045dac5180"
format_stamp: "Formatted at 2024-05-03 02:14:03 on dist-test-slave-twlh"
I20240503 02:14:03.607578 9920 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknrxqHk/test-tmp/tablet_copy_client_session-itest.0.TabletCopyClientSessionITest.TestStartTabletCopyWhileSourceBootstrapping.1714702442467385-9822-0/minicluster-data/ts-0/wal/instance:
uuid: "7cd9a8da65604af78132e9045dac5180"
format_stamp: "Formatted at 2024-05-03 02:14:03 on dist-test-slave-twlh"
I20240503 02:14:03.614112 9920 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.000s
I20240503 02:14:03.619020 9934 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240503 02:14:03.621320 9920 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.002s sys 0.002s
I20240503 02:14:03.621534 9920 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknrxqHk/test-tmp/tablet_copy_client_session-itest.0.TabletCopyClientSessionITest.TestStartTabletCopyWhileSourceBootstrapping.1714702442467385-9822-0/minicluster-data/ts-0/data,/tmp/dist-test-tasknrxqHk/test-tmp/tablet_copy_client_session-itest.0.TabletCopyClientSessionITest.TestStartTabletCopyWhileSourceBootstrapping.1714702442467385-9822-0/minicluster-data/ts-0/wal
uuid: "7cd9a8da65604af78132e9045dac5180"
format_stamp: "Formatted at 2024-05-03 02:14:03 on dist-test-slave-twlh"
I20240503 02:14:03.621799 9920 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknrxqHk/test-tmp/tablet_copy_client_session-itest.0.TabletCopyClientSessionITest.TestStartTabletCopyWhileSourceBootstrapping.1714702442467385-9822-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasknrxqHk/test-tmp/tablet_copy_client_session-itest.0.TabletCopyClientSessionITest.TestStartTabletCopyWhileSourceBootstrapping.1714702442467385-9822-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasknrxqHk/test-tmp/tablet_copy_client_session-itest.0.TabletCopyClientSessionITest.TestStartTabletCopyWhileSourceBootstrapping.1714702442467385-9822-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240503 02:14:03.641352 9920 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240503 02:14:03.642343 9920 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240503 02:14:03.642717 9920 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240503 02:14:03.643827 9920 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240503 02:14:03.645825 9920 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240503 02:14:03.645928 9920 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240503 02:14:03.646066 9920 ts_tablet_manager.cc:590] Registered 0 tablets
I20240503 02:14:03.646123 9920 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240503 02:14:03.690409 9920 rpc_server.cc:307] RPC server started. Bound to: 127.9.151.129:33077
I20240503 02:14:03.690472 10046 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.151.129:33077 every 8 connection(s)
I20240503 02:14:03.691854 9920 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknrxqHk/test-tmp/tablet_copy_client_session-itest.0.TabletCopyClientSessionITest.TestStartTabletCopyWhileSourceBootstrapping.1714702442467385-9822-0/minicluster-data/ts-0/data/info.pb
I20240503 02:14:03.692612 9822 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknrxqHk/build/asan/bin/kudu as pid 9920
I20240503 02:14:03.692864 9822 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknrxqHk/test-tmp/tablet_copy_client_session-itest.0.TabletCopyClientSessionITest.TestStartTabletCopyWhileSourceBootstrapping.1714702442467385-9822-0/minicluster-data/ts-0/wal/instance
I20240503 02:14:03.697875 9822 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknrxqHk/build/asan/bin/kudu
/tmp/dist-test-tasknrxqHk/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknrxqHk/test-tmp/tablet_copy_client_session-itest.0.TabletCopyClientSessionITest.TestStartTabletCopyWhileSourceBootstrapping.1714702442467385-9822-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasknrxqHk/test-tmp/tablet_copy_client_session-itest.0.TabletCopyClientSessionITest.TestStartTabletCopyWhileSourceBootstrapping.1714702442467385-9822-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknrxqHk/test-tmp/tablet_copy_client_session-itest.0.TabletCopyClientSessionITest.TestStartTabletCopyWhileSourceBootstrapping.1714702442467385-9822-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasknrxqHk/test-tmp/tablet_copy_client_session-itest.0.TabletCopyClientSessionITest.TestStartTabletCopyWhileSourceBootstrapping.1714702442467385-9822-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.151.130:0
--local_ip_for_outbound_sockets=127.9.151.130
--webserver_interface=127.9.151.130
--webserver_port=0
--tserver_master_addrs=127.9.151.190:36085
--builtin_ntp_servers=127.9.151.148:38865
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin with env {}
I20240503 02:14:03.712718 10047 heartbeater.cc:347] Connected to a master server at 127.9.151.190:36085
I20240503 02:14:03.713230 10047 heartbeater.cc:464] Registering TS with master...
I20240503 02:14:03.714491 10047 heartbeater.cc:511] Master 127.9.151.190:36085 requested a full tablet report, sending...
I20240503 02:14:03.717533 9861 ts_manager.cc:194] Registered new tserver with Master: 7cd9a8da65604af78132e9045dac5180 (127.9.151.129:33077)
I20240503 02:14:03.719417 9861 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.9.151.129:36207
W20240503 02:14:04.081255 10051 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240503 02:14:04.081678 10051 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240503 02:14:04.081815 10051 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240503 02:14:04.091151 10051 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240503 02:14:04.091706 10051 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.151.130
I20240503 02:14:04.102803 10051 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.151.148:38865
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknrxqHk/test-tmp/tablet_copy_client_session-itest.0.TabletCopyClientSessionITest.TestStartTabletCopyWhileSourceBootstrapping.1714702442467385-9822-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasknrxqHk/test-tmp/tablet_copy_client_session-itest.0.TabletCopyClientSessionITest.TestStartTabletCopyWhileSourceBootstrapping.1714702442467385-9822-0/minicluster-data/ts-1/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.151.130:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknrxqHk/test-tmp/tablet_copy_client_session-itest.0.TabletCopyClientSessionITest.TestStartTabletCopyWhileSourceBootstrapping.1714702442467385-9822-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.151.130
--webserver_port=0
--tserver_master_addrs=127.9.151.190:36085
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.151.130
--log_dir=/tmp/dist-test-tasknrxqHk/test-tmp/tablet_copy_client_session-itest.0.TabletCopyClientSessionITest.TestStartTabletCopyWhileSourceBootstrapping.1714702442467385-9822-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c39e72a98e124e5a12080bac42105b996594d806
build type FASTDEBUG
built by None at 03 May 2024 01:43:22 UTC on efd1337a2aa3
build id 99
ASAN enabled
I20240503 02:14:04.104504 10051 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240503 02:14:04.106668 10051 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240503 02:14:04.122737 10057 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240503 02:14:04.123417 10059 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240503 02:14:04.123783 10056 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240503 02:14:04.124389 10051 server_base.cc:1034] running on GCE node
I20240503 02:14:04.125159 10051 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240503 02:14:04.126317 10051 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240503 02:14:04.127573 10051 hybrid_clock.cc:648] HybridClock initialized: now 1714702444127493 us; error 57 us; skew 500 ppm
I20240503 02:14:04.127938 10051 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20240503 02:14:04.131122 10051 webserver.cc:460] Webserver started at http://127.9.151.130:42537/ using document root <none> and password file <none>
I20240503 02:14:04.131990 10051 fs_manager.cc:359] Metadata directory not provided
I20240503 02:14:04.132131 10051 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240503 02:14:04.132488 10051 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240503 02:14:04.135670 10051 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknrxqHk/test-tmp/tablet_copy_client_session-itest.0.TabletCopyClientSessionITest.TestStartTabletCopyWhileSourceBootstrapping.1714702442467385-9822-0/minicluster-data/ts-1/data/instance:
uuid: "4c8a3af9b5184a51a3c0fe3221c01c75"
format_stamp: "Formatted at 2024-05-03 02:14:04 on dist-test-slave-twlh"
I20240503 02:14:04.136679 10051 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknrxqHk/test-tmp/tablet_copy_client_session-itest.0.TabletCopyClientSessionITest.TestStartTabletCopyWhileSourceBootstrapping.1714702442467385-9822-0/minicluster-data/ts-1/wal/instance:
uuid: "4c8a3af9b5184a51a3c0fe3221c01c75"
format_stamp: "Formatted at 2024-05-03 02:14:04 on dist-test-slave-twlh"
I20240503 02:14:04.143914 10051 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.007s sys 0.000s
I20240503 02:14:04.148772 10065 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240503 02:14:04.150456 10051 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20240503 02:14:04.150717 10051 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknrxqHk/test-tmp/tablet_copy_client_session-itest.0.TabletCopyClientSessionITest.TestStartTabletCopyWhileSourceBootstrapping.1714702442467385-9822-0/minicluster-data/ts-1/data,/tmp/dist-test-tasknrxqHk/test-tmp/tablet_copy_client_session-itest.0.TabletCopyClientSessionITest.TestStartTabletCopyWhileSourceBootstrapping.1714702442467385-9822-0/minicluster-data/ts-1/wal
uuid: "4c8a3af9b5184a51a3c0fe3221c01c75"
format_stamp: "Formatted at 2024-05-03 02:14:04 on dist-test-slave-twlh"
I20240503 02:14:04.150946 10051 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknrxqHk/test-tmp/tablet_copy_client_session-itest.0.TabletCopyClientSessionITest.TestStartTabletCopyWhileSourceBootstrapping.1714702442467385-9822-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasknrxqHk/test-tmp/tablet_copy_client_session-itest.0.TabletCopyClientSessionITest.TestStartTabletCopyWhileSourceBootstrapping.1714702442467385-9822-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasknrxqHk/test-tmp/tablet_copy_client_session-itest.0.TabletCopyClientSessionITest.TestStartTabletCopyWhileSourceBootstrapping.1714702442467385-9822-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240503 02:14:04.167623 10051 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240503 02:14:04.168610 10051 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240503 02:14:04.168941 10051 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240503 02:14:04.170176 10051 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240503 02:14:04.172053 10051 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240503 02:14:04.172151 10051 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240503 02:14:04.172262 10051 ts_tablet_manager.cc:590] Registered 0 tablets
I20240503 02:14:04.172338 10051 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240503 02:14:04.209883 10051 rpc_server.cc:307] RPC server started. Bound to: 127.9.151.130:35753
I20240503 02:14:04.209913 10177 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.151.130:35753 every 8 connection(s)
I20240503 02:14:04.211436 10051 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknrxqHk/test-tmp/tablet_copy_client_session-itest.0.TabletCopyClientSessionITest.TestStartTabletCopyWhileSourceBootstrapping.1714702442467385-9822-0/minicluster-data/ts-1/data/info.pb
I20240503 02:14:04.214779 9822 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknrxqHk/build/asan/bin/kudu as pid 10051
I20240503 02:14:04.215057 9822 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknrxqHk/test-tmp/tablet_copy_client_session-itest.0.TabletCopyClientSessionITest.TestStartTabletCopyWhileSourceBootstrapping.1714702442467385-9822-0/minicluster-data/ts-1/wal/instance
I20240503 02:14:04.229126 10178 heartbeater.cc:347] Connected to a master server at 127.9.151.190:36085
I20240503 02:14:04.229496 10178 heartbeater.cc:464] Registering TS with master...
I20240503 02:14:04.230502 10178 heartbeater.cc:511] Master 127.9.151.190:36085 requested a full tablet report, sending...
I20240503 02:14:04.232208 9861 ts_manager.cc:194] Registered new tserver with Master: 4c8a3af9b5184a51a3c0fe3221c01c75 (127.9.151.130:35753)
I20240503 02:14:04.233479 9861 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.9.151.130:52459
I20240503 02:14:04.233572 9822 external_mini_cluster.cc:930] 2 TS(s) registered with all masters
I20240503 02:14:04.257962 9822 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasknrxqHk/build/asan/bin/kudu pid 10051
I20240503 02:14:04.269369 10173 generic_service.cc:196] Checking for leaks (request via RPC)
I20240503 02:14:04.357263 9822 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknrxqHk/build/asan/bin/kudu with pid 10051
I20240503 02:14:04.375080 9822 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasknrxqHk/build/asan/bin/kudu pid 9830
I20240503 02:14:04.376461 9891 generic_service.cc:196] Checking for leaks (request via RPC)
I20240503 02:14:04.481832 9822 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknrxqHk/build/asan/bin/kudu with pid 9830
I20240503 02:14:04.504115 9822 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknrxqHk/build/asan/bin/kudu
/tmp/dist-test-tasknrxqHk/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknrxqHk/test-tmp/tablet_copy_client_session-itest.0.TabletCopyClientSessionITest.TestStartTabletCopyWhileSourceBootstrapping.1714702442467385-9822-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasknrxqHk/test-tmp/tablet_copy_client_session-itest.0.TabletCopyClientSessionITest.TestStartTabletCopyWhileSourceBootstrapping.1714702442467385-9822-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknrxqHk/test-tmp/tablet_copy_client_session-itest.0.TabletCopyClientSessionITest.TestStartTabletCopyWhileSourceBootstrapping.1714702442467385-9822-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasknrxqHk/test-tmp/tablet_copy_client_session-itest.0.TabletCopyClientSessionITest.TestStartTabletCopyWhileSourceBootstrapping.1714702442467385-9822-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.151.190:36085
--webserver_interface=127.9.151.190
--webserver_port=43689
--builtin_ntp_servers=127.9.151.148:38865
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.151.190:36085
--master_tombstone_evicted_tablet_replicas=false with env {}
I20240503 02:14:04.723790 10047 heartbeater.cc:503] Master 127.9.151.190:36085 was elected leader, sending a full tablet report...
W20240503 02:14:04.725814 10047 heartbeater.cc:644] Failed to heartbeat to 127.9.151.190:36085 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.9.151.190:36085: connect: Connection refused (error 111)
W20240503 02:14:04.885663 10191 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240503 02:14:04.886024 10191 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240503 02:14:04.886139 10191 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240503 02:14:04.894680 10191 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20240503 02:14:04.895004 10191 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240503 02:14:04.895215 10191 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20240503 02:14:04.895325 10191 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20240503 02:14:04.905752 10191 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.151.148:38865
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknrxqHk/test-tmp/tablet_copy_client_session-itest.0.TabletCopyClientSessionITest.TestStartTabletCopyWhileSourceBootstrapping.1714702442467385-9822-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasknrxqHk/test-tmp/tablet_copy_client_session-itest.0.TabletCopyClientSessionITest.TestStartTabletCopyWhileSourceBootstrapping.1714702442467385-9822-0/minicluster-data/master-0/wal
--master_tombstone_evicted_tablet_replicas=false
--ipki_ca_key_size=768
--master_addresses=127.9.151.190:36085
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.151.190:36085
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknrxqHk/test-tmp/tablet_copy_client_session-itest.0.TabletCopyClientSessionITest.TestStartTabletCopyWhileSourceBootstrapping.1714702442467385-9822-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.151.190
--webserver_port=43689
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasknrxqHk/test-tmp/tablet_copy_client_session-itest.0.TabletCopyClientSessionITest.TestStartTabletCopyWhileSourceBootstrapping.1714702442467385-9822-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c39e72a98e124e5a12080bac42105b996594d806
build type FASTDEBUG
built by None at 03 May 2024 01:43:22 UTC on efd1337a2aa3
build id 99
ASAN enabled
I20240503 02:14:04.907545 10191 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240503 02:14:04.909567 10191 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240503 02:14:04.922963 10198 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240503 02:14:04.923791 10191 server_base.cc:1034] running on GCE node
W20240503 02:14:04.924263 10197 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240503 02:14:04.925027 10200 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240503 02:14:04.926074 10191 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240503 02:14:04.928663 10191 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240503 02:14:04.930016 10191 hybrid_clock.cc:648] HybridClock initialized: now 1714702444929928 us; error 52 us; skew 500 ppm
I20240503 02:14:04.930481 10191 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20240503 02:14:04.933512 10191 webserver.cc:460] Webserver started at http://127.9.151.190:43689/ using document root <none> and password file <none>
I20240503 02:14:04.934415 10191 fs_manager.cc:359] Metadata directory not provided
I20240503 02:14:04.934600 10191 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240503 02:14:04.941057 10191 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.004s sys 0.000s
I20240503 02:14:04.944769 10206 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240503 02:14:04.946640 10191 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.000s
I20240503 02:14:04.946952 10191 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknrxqHk/test-tmp/tablet_copy_client_session-itest.0.TabletCopyClientSessionITest.TestStartTabletCopyWhileSourceBootstrapping.1714702442467385-9822-0/minicluster-data/master-0/data,/tmp/dist-test-tasknrxqHk/test-tmp/tablet_copy_client_session-itest.0.TabletCopyClientSessionITest.TestStartTabletCopyWhileSourceBootstrapping.1714702442467385-9822-0/minicluster-data/master-0/wal
uuid: "9515dc4386d5467290be9ed534098d9b"
format_stamp: "Formatted at 2024-05-03 02:14:02 on dist-test-slave-twlh"
I20240503 02:14:04.947933 10191 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknrxqHk/test-tmp/tablet_copy_client_session-itest.0.TabletCopyClientSessionITest.TestStartTabletCopyWhileSourceBootstrapping.1714702442467385-9822-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasknrxqHk/test-tmp/tablet_copy_client_session-itest.0.TabletCopyClientSessionITest.TestStartTabletCopyWhileSourceBootstrapping.1714702442467385-9822-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasknrxqHk/test-tmp/tablet_copy_client_session-itest.0.TabletCopyClientSessionITest.TestStartTabletCopyWhileSourceBootstrapping.1714702442467385-9822-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240503 02:14:04.966835 10191 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240503 02:14:04.967814 10191 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240503 02:14:04.968125 10191 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240503 02:14:04.989524 10191 rpc_server.cc:307] RPC server started. Bound to: 127.9.151.190:36085
I20240503 02:14:04.989543 10257 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.151.190:36085 every 8 connection(s)
I20240503 02:14:04.991282 10191 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknrxqHk/test-tmp/tablet_copy_client_session-itest.0.TabletCopyClientSessionITest.TestStartTabletCopyWhileSourceBootstrapping.1714702442467385-9822-0/minicluster-data/master-0/data/info.pb
I20240503 02:14:04.996709 9822 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknrxqHk/build/asan/bin/kudu as pid 10191
I20240503 02:14:04.999619 10258 sys_catalog.cc:263] Verifying existing consensus state
I20240503 02:14:05.003552 10258 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9515dc4386d5467290be9ed534098d9b: Bootstrap starting.
I20240503 02:14:05.022130 10258 log.cc:824] T 00000000000000000000000000000000 P 9515dc4386d5467290be9ed534098d9b: Log is configured to *not* fsync() on all Append() calls
I20240503 02:14:05.032862 10258 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9515dc4386d5467290be9ed534098d9b: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240503 02:14:05.033490 10258 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9515dc4386d5467290be9ed534098d9b: Bootstrap complete.
I20240503 02:14:05.042090 10258 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9515dc4386d5467290be9ed534098d9b [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9515dc4386d5467290be9ed534098d9b" member_type: VOTER last_known_addr { host: "127.9.151.190" port: 36085 } }
I20240503 02:14:05.043040 10258 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9515dc4386d5467290be9ed534098d9b [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9515dc4386d5467290be9ed534098d9b, State: Initialized, Role: FOLLOWER
I20240503 02:14:05.043768 10258 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9515dc4386d5467290be9ed534098d9b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9515dc4386d5467290be9ed534098d9b" member_type: VOTER last_known_addr { host: "127.9.151.190" port: 36085 } }
I20240503 02:14:05.044036 10258 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9515dc4386d5467290be9ed534098d9b [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20240503 02:14:05.044157 10258 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9515dc4386d5467290be9ed534098d9b [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20240503 02:14:05.044337 10258 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9515dc4386d5467290be9ed534098d9b [term 1 FOLLOWER]: Advancing to term 2
I20240503 02:14:05.048485 10258 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9515dc4386d5467290be9ed534098d9b [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9515dc4386d5467290be9ed534098d9b" member_type: VOTER last_known_addr { host: "127.9.151.190" port: 36085 } }
I20240503 02:14:05.049129 10258 leader_election.cc:302] T 00000000000000000000000000000000 P 9515dc4386d5467290be9ed534098d9b [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9515dc4386d5467290be9ed534098d9b; no voters:
I20240503 02:14:05.049881 10258 leader_election.cc:288] T 00000000000000000000000000000000 P 9515dc4386d5467290be9ed534098d9b [CANDIDATE]: Term 2 election: Requested vote from peers
I20240503 02:14:05.050200 10263 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9515dc4386d5467290be9ed534098d9b [term 2 FOLLOWER]: Leader election won for term 2
I20240503 02:14:05.051674 10263 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9515dc4386d5467290be9ed534098d9b [term 2 LEADER]: Becoming Leader. State: Replica: 9515dc4386d5467290be9ed534098d9b, State: Running, Role: LEADER
I20240503 02:14:05.052596 10258 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9515dc4386d5467290be9ed534098d9b [sys.catalog]: configured and running, proceeding with master startup.
I20240503 02:14:05.052556 10263 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9515dc4386d5467290be9ed534098d9b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9515dc4386d5467290be9ed534098d9b" member_type: VOTER last_known_addr { host: "127.9.151.190" port: 36085 } }
I20240503 02:14:05.055864 10264 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9515dc4386d5467290be9ed534098d9b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "9515dc4386d5467290be9ed534098d9b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9515dc4386d5467290be9ed534098d9b" member_type: VOTER last_known_addr { host: "127.9.151.190" port: 36085 } } }
I20240503 02:14:05.056224 10264 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9515dc4386d5467290be9ed534098d9b [sys.catalog]: This master's current role is: LEADER
I20240503 02:14:05.055766 10265 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9515dc4386d5467290be9ed534098d9b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9515dc4386d5467290be9ed534098d9b. Latest consensus state: current_term: 2 leader_uuid: "9515dc4386d5467290be9ed534098d9b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9515dc4386d5467290be9ed534098d9b" member_type: VOTER last_known_addr { host: "127.9.151.190" port: 36085 } } }
I20240503 02:14:05.056743 10265 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9515dc4386d5467290be9ed534098d9b [sys.catalog]: This master's current role is: LEADER
I20240503 02:14:05.058866 10268 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20240503 02:14:05.067266 10268 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20240503 02:14:05.069041 10268 catalog_manager.cc:1259] Loaded cluster ID: 5b1f9b52b5d1438295da79db46b7114b
I20240503 02:14:05.069180 10268 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20240503 02:14:05.072523 10268 catalog_manager.cc:1504] Loading token signing keys...
I20240503 02:14:05.073983 10268 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9515dc4386d5467290be9ed534098d9b: Loaded TSK: 0
I20240503 02:14:05.075644 10268 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20240503 02:14:05.733808 10047 heartbeater.cc:347] Connected to a master server at 127.9.151.190:36085
I20240503 02:14:05.734076 10047 heartbeater.cc:503] Master 127.9.151.190:36085 was elected leader, sending a full tablet report...
I20240503 02:14:05.737164 10223 master_service.cc:432] Got heartbeat from unknown tserver (permanent_uuid: "7cd9a8da65604af78132e9045dac5180" instance_seqno: 1714702443681340) as {username='slave'} at 127.9.151.129:35475; Asking this server to re-register.
I20240503 02:14:05.738214 10047 heartbeater.cc:464] Registering TS with master...
I20240503 02:14:05.738504 10047 heartbeater.cc:511] Master 127.9.151.190:36085 requested a full tablet report, sending...
I20240503 02:14:05.740262 10223 ts_manager.cc:194] Registered new tserver with Master: 7cd9a8da65604af78132e9045dac5180 (127.9.151.129:33077)
I20240503 02:14:05.751987 9822 external_mini_cluster.cc:930] 1 TS(s) registered with all masters
I20240503 02:14:05.752403 9822 test_util.cc:268] Using random seed: 1177333803
I20240503 02:14:05.782850 10223 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:55284:
name: "test-workload"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20240503 02:14:05.813840 9982 tablet_service.cc:1467] Processing CreateTablet for tablet 375647a877014ca48ca32ff220098cd1 (DEFAULT_TABLE table=test-workload [id=b4f9f08c6ffe4c88b0fd627cfb56d319]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:14:05.815865 9982 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 375647a877014ca48ca32ff220098cd1. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:14:05.828393 10293 tablet_bootstrap.cc:492] T 375647a877014ca48ca32ff220098cd1 P 7cd9a8da65604af78132e9045dac5180: Bootstrap starting.
I20240503 02:14:05.831948 10293 tablet_bootstrap.cc:654] T 375647a877014ca48ca32ff220098cd1 P 7cd9a8da65604af78132e9045dac5180: Neither blocks nor log segments found. Creating new log.
I20240503 02:14:05.833181 10293 log.cc:824] T 375647a877014ca48ca32ff220098cd1 P 7cd9a8da65604af78132e9045dac5180: Log is configured to *not* fsync() on all Append() calls
I20240503 02:14:05.836455 10293 tablet_bootstrap.cc:492] T 375647a877014ca48ca32ff220098cd1 P 7cd9a8da65604af78132e9045dac5180: No bootstrap required, opened a new log
I20240503 02:14:05.836807 10293 ts_tablet_manager.cc:1367] T 375647a877014ca48ca32ff220098cd1 P 7cd9a8da65604af78132e9045dac5180: Time spent bootstrapping tablet: real 0.009s user 0.006s sys 0.000s
I20240503 02:14:05.843344 10293 raft_consensus.cc:348] T 375647a877014ca48ca32ff220098cd1 P 7cd9a8da65604af78132e9045dac5180 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7cd9a8da65604af78132e9045dac5180" member_type: VOTER last_known_addr { host: "127.9.151.129" port: 33077 } }
I20240503 02:14:05.843762 10293 raft_consensus.cc:374] T 375647a877014ca48ca32ff220098cd1 P 7cd9a8da65604af78132e9045dac5180 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:14:05.843880 10293 raft_consensus.cc:729] T 375647a877014ca48ca32ff220098cd1 P 7cd9a8da65604af78132e9045dac5180 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7cd9a8da65604af78132e9045dac5180, State: Initialized, Role: FOLLOWER
I20240503 02:14:05.844568 10293 consensus_queue.cc:260] T 375647a877014ca48ca32ff220098cd1 P 7cd9a8da65604af78132e9045dac5180 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7cd9a8da65604af78132e9045dac5180" member_type: VOTER last_known_addr { host: "127.9.151.129" port: 33077 } }
I20240503 02:14:05.844805 10293 raft_consensus.cc:388] T 375647a877014ca48ca32ff220098cd1 P 7cd9a8da65604af78132e9045dac5180 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20240503 02:14:05.844954 10293 raft_consensus.cc:482] T 375647a877014ca48ca32ff220098cd1 P 7cd9a8da65604af78132e9045dac5180 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20240503 02:14:05.845110 10293 raft_consensus.cc:3037] T 375647a877014ca48ca32ff220098cd1 P 7cd9a8da65604af78132e9045dac5180 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:14:05.847878 10293 raft_consensus.cc:504] T 375647a877014ca48ca32ff220098cd1 P 7cd9a8da65604af78132e9045dac5180 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7cd9a8da65604af78132e9045dac5180" member_type: VOTER last_known_addr { host: "127.9.151.129" port: 33077 } }
I20240503 02:14:05.848625 10293 leader_election.cc:302] T 375647a877014ca48ca32ff220098cd1 P 7cd9a8da65604af78132e9045dac5180 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7cd9a8da65604af78132e9045dac5180; no voters:
I20240503 02:14:05.849398 10293 leader_election.cc:288] T 375647a877014ca48ca32ff220098cd1 P 7cd9a8da65604af78132e9045dac5180 [CANDIDATE]: Term 1 election: Requested vote from peers
I20240503 02:14:05.849570 10295 raft_consensus.cc:2781] T 375647a877014ca48ca32ff220098cd1 P 7cd9a8da65604af78132e9045dac5180 [term 1 FOLLOWER]: Leader election won for term 1
I20240503 02:14:05.850322 10295 raft_consensus.cc:686] T 375647a877014ca48ca32ff220098cd1 P 7cd9a8da65604af78132e9045dac5180 [term 1 LEADER]: Becoming Leader. State: Replica: 7cd9a8da65604af78132e9045dac5180, State: Running, Role: LEADER
I20240503 02:14:05.851095 10295 consensus_queue.cc:237] T 375647a877014ca48ca32ff220098cd1 P 7cd9a8da65604af78132e9045dac5180 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7cd9a8da65604af78132e9045dac5180" member_type: VOTER last_known_addr { host: "127.9.151.129" port: 33077 } }
I20240503 02:14:05.851697 10293 ts_tablet_manager.cc:1398] T 375647a877014ca48ca32ff220098cd1 P 7cd9a8da65604af78132e9045dac5180: Time spent starting tablet: real 0.015s user 0.012s sys 0.004s
I20240503 02:14:05.857972 10222 catalog_manager.cc:5484] T 375647a877014ca48ca32ff220098cd1 P 7cd9a8da65604af78132e9045dac5180 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7cd9a8da65604af78132e9045dac5180 (127.9.151.129). New cstate: current_term: 1 leader_uuid: "7cd9a8da65604af78132e9045dac5180" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7cd9a8da65604af78132e9045dac5180" member_type: VOTER last_known_addr { host: "127.9.151.129" port: 33077 } health_report { overall_health: HEALTHY } } }
I20240503 02:14:06.583051 9822 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknrxqHk/build/asan/bin/kudu
/tmp/dist-test-tasknrxqHk/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknrxqHk/test-tmp/tablet_copy_client_session-itest.0.TabletCopyClientSessionITest.TestStartTabletCopyWhileSourceBootstrapping.1714702442467385-9822-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasknrxqHk/test-tmp/tablet_copy_client_session-itest.0.TabletCopyClientSessionITest.TestStartTabletCopyWhileSourceBootstrapping.1714702442467385-9822-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknrxqHk/test-tmp/tablet_copy_client_session-itest.0.TabletCopyClientSessionITest.TestStartTabletCopyWhileSourceBootstrapping.1714702442467385-9822-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasknrxqHk/test-tmp/tablet_copy_client_session-itest.0.TabletCopyClientSessionITest.TestStartTabletCopyWhileSourceBootstrapping.1714702442467385-9822-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.151.130:35753
--local_ip_for_outbound_sockets=127.9.151.130
--tserver_master_addrs=127.9.151.190:36085
--webserver_port=42537
--webserver_interface=127.9.151.130
--builtin_ntp_servers=127.9.151.148:38865
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin with env {}
W20240503 02:14:06.945655 10310 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240503 02:14:06.945999 10310 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240503 02:14:06.946098 10310 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240503 02:14:06.954309 10310 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240503 02:14:06.954545 10310 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.151.130
I20240503 02:14:06.964159 10310 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.151.148:38865
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknrxqHk/test-tmp/tablet_copy_client_session-itest.0.TabletCopyClientSessionITest.TestStartTabletCopyWhileSourceBootstrapping.1714702442467385-9822-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasknrxqHk/test-tmp/tablet_copy_client_session-itest.0.TabletCopyClientSessionITest.TestStartTabletCopyWhileSourceBootstrapping.1714702442467385-9822-0/minicluster-data/ts-1/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.151.130:35753
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknrxqHk/test-tmp/tablet_copy_client_session-itest.0.TabletCopyClientSessionITest.TestStartTabletCopyWhileSourceBootstrapping.1714702442467385-9822-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.151.130
--webserver_port=42537
--tserver_master_addrs=127.9.151.190:36085
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.151.130
--log_dir=/tmp/dist-test-tasknrxqHk/test-tmp/tablet_copy_client_session-itest.0.TabletCopyClientSessionITest.TestStartTabletCopyWhileSourceBootstrapping.1714702442467385-9822-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c39e72a98e124e5a12080bac42105b996594d806
build type FASTDEBUG
built by None at 03 May 2024 01:43:22 UTC on efd1337a2aa3
build id 99
ASAN enabled
I20240503 02:14:06.965559 10310 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240503 02:14:06.967430 10310 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240503 02:14:06.981289 10315 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240503 02:14:06.981863 10316 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240503 02:14:06.982636 10318 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240503 02:14:06.983302 10310 server_base.cc:1034] running on GCE node
I20240503 02:14:06.984050 10310 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240503 02:14:06.985183 10310 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240503 02:14:06.986459 10310 hybrid_clock.cc:648] HybridClock initialized: now 1714702446986392 us; error 37 us; skew 500 ppm
I20240503 02:14:06.986856 10310 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20240503 02:14:06.989610 10310 webserver.cc:460] Webserver started at http://127.9.151.130:42537/ using document root <none> and password file <none>
I20240503 02:14:06.990630 10310 fs_manager.cc:359] Metadata directory not provided
I20240503 02:14:06.990846 10310 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240503 02:14:06.998380 10310 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.006s sys 0.000s
I20240503 02:14:07.001678 10324 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240503 02:14:07.003310 10310 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20240503 02:14:07.003505 10310 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknrxqHk/test-tmp/tablet_copy_client_session-itest.0.TabletCopyClientSessionITest.TestStartTabletCopyWhileSourceBootstrapping.1714702442467385-9822-0/minicluster-data/ts-1/data,/tmp/dist-test-tasknrxqHk/test-tmp/tablet_copy_client_session-itest.0.TabletCopyClientSessionITest.TestStartTabletCopyWhileSourceBootstrapping.1714702442467385-9822-0/minicluster-data/ts-1/wal
uuid: "4c8a3af9b5184a51a3c0fe3221c01c75"
format_stamp: "Formatted at 2024-05-03 02:14:04 on dist-test-slave-twlh"
I20240503 02:14:07.004426 10310 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknrxqHk/test-tmp/tablet_copy_client_session-itest.0.TabletCopyClientSessionITest.TestStartTabletCopyWhileSourceBootstrapping.1714702442467385-9822-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasknrxqHk/test-tmp/tablet_copy_client_session-itest.0.TabletCopyClientSessionITest.TestStartTabletCopyWhileSourceBootstrapping.1714702442467385-9822-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasknrxqHk/test-tmp/tablet_copy_client_session-itest.0.TabletCopyClientSessionITest.TestStartTabletCopyWhileSourceBootstrapping.1714702442467385-9822-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240503 02:14:07.038410 10310 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240503 02:14:07.039458 10310 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240503 02:14:07.039791 10310 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240503 02:14:07.040835 10310 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240503 02:14:07.042486 10310 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240503 02:14:07.042588 10310 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240503 02:14:07.042703 10310 ts_tablet_manager.cc:590] Registered 0 tablets
I20240503 02:14:07.042785 10310 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.001s sys 0.000s
I20240503 02:14:07.077320 10310 rpc_server.cc:307] RPC server started. Bound to: 127.9.151.130:35753
I20240503 02:14:07.077378 10436 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.151.130:35753 every 8 connection(s)
I20240503 02:14:07.078985 10310 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknrxqHk/test-tmp/tablet_copy_client_session-itest.0.TabletCopyClientSessionITest.TestStartTabletCopyWhileSourceBootstrapping.1714702442467385-9822-0/minicluster-data/ts-1/data/info.pb
I20240503 02:14:07.085894 9822 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknrxqHk/build/asan/bin/kudu as pid 10310
I20240503 02:14:07.090467 9822 tablet_copy_client_session-itest.cc:139] Starting loop 0
I20240503 02:14:07.090796 9822 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasknrxqHk/build/asan/bin/kudu pid 9920
I20240503 02:14:07.092175 10042 generic_service.cc:196] Checking for leaks (request via RPC)
I20240503 02:14:07.093992 10437 heartbeater.cc:347] Connected to a master server at 127.9.151.190:36085
I20240503 02:14:07.094373 10437 heartbeater.cc:464] Registering TS with master...
I20240503 02:14:07.095707 10437 heartbeater.cc:511] Master 127.9.151.190:36085 requested a full tablet report, sending...
I20240503 02:14:07.097654 10221 ts_manager.cc:194] Registered new tserver with Master: 4c8a3af9b5184a51a3c0fe3221c01c75 (127.9.151.130:35753)
I20240503 02:14:07.098919 10221 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.9.151.130:35427
I20240503 02:14:07.191888 9822 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknrxqHk/build/asan/bin/kudu with pid 9920
I20240503 02:14:07.216192 9822 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknrxqHk/build/asan/bin/kudu
/tmp/dist-test-tasknrxqHk/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknrxqHk/test-tmp/tablet_copy_client_session-itest.0.TabletCopyClientSessionITest.TestStartTabletCopyWhileSourceBootstrapping.1714702442467385-9822-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasknrxqHk/test-tmp/tablet_copy_client_session-itest.0.TabletCopyClientSessionITest.TestStartTabletCopyWhileSourceBootstrapping.1714702442467385-9822-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknrxqHk/test-tmp/tablet_copy_client_session-itest.0.TabletCopyClientSessionITest.TestStartTabletCopyWhileSourceBootstrapping.1714702442467385-9822-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasknrxqHk/test-tmp/tablet_copy_client_session-itest.0.TabletCopyClientSessionITest.TestStartTabletCopyWhileSourceBootstrapping.1714702442467385-9822-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.151.129:33077
--local_ip_for_outbound_sockets=127.9.151.129
--tserver_master_addrs=127.9.151.190:36085
--webserver_port=43885
--webserver_interface=127.9.151.129
--builtin_ntp_servers=127.9.151.148:38865
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin with env {}
I20240503 02:14:07.232537 10449 ts_tablet_manager.cc:902] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: Initiating tablet copy from peer 7cd9a8da65604af78132e9045dac5180 (127.9.151.129:33077)
I20240503 02:14:07.240010 10449 tablet_copy_client.cc:308] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: tablet copy: Beginning tablet copy session from remote peer at address 127.9.151.129:33077
I20240503 02:14:07.247469 10449 ts_tablet_manager.cc:902] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: Initiating tablet copy from peer 7cd9a8da65604af78132e9045dac5180 (127.9.151.129:33077)
I20240503 02:14:07.248965 10449 tablet_copy_client.cc:308] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: tablet copy: Beginning tablet copy session from remote peer at address 127.9.151.129:33077
I20240503 02:14:07.266048 10449 ts_tablet_manager.cc:902] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: Initiating tablet copy from peer 7cd9a8da65604af78132e9045dac5180 (127.9.151.129:33077)
I20240503 02:14:07.267382 10449 tablet_copy_client.cc:308] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: tablet copy: Beginning tablet copy session from remote peer at address 127.9.151.129:33077
I20240503 02:14:07.277551 10449 ts_tablet_manager.cc:902] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: Initiating tablet copy from peer 7cd9a8da65604af78132e9045dac5180 (127.9.151.129:33077)
I20240503 02:14:07.278986 10449 tablet_copy_client.cc:308] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: tablet copy: Beginning tablet copy session from remote peer at address 127.9.151.129:33077
I20240503 02:14:07.286054 10449 ts_tablet_manager.cc:902] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: Initiating tablet copy from peer 7cd9a8da65604af78132e9045dac5180 (127.9.151.129:33077)
I20240503 02:14:07.287328 10449 tablet_copy_client.cc:308] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: tablet copy: Beginning tablet copy session from remote peer at address 127.9.151.129:33077
I20240503 02:14:07.290922 10449 ts_tablet_manager.cc:902] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: Initiating tablet copy from peer 7cd9a8da65604af78132e9045dac5180 (127.9.151.129:33077)
I20240503 02:14:07.292116 10449 tablet_copy_client.cc:308] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: tablet copy: Beginning tablet copy session from remote peer at address 127.9.151.129:33077
I20240503 02:14:07.301635 10449 ts_tablet_manager.cc:902] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: Initiating tablet copy from peer 7cd9a8da65604af78132e9045dac5180 (127.9.151.129:33077)
I20240503 02:14:07.302937 10449 tablet_copy_client.cc:308] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: tablet copy: Beginning tablet copy session from remote peer at address 127.9.151.129:33077
I20240503 02:14:07.312911 10449 ts_tablet_manager.cc:902] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: Initiating tablet copy from peer 7cd9a8da65604af78132e9045dac5180 (127.9.151.129:33077)
I20240503 02:14:07.314533 10449 tablet_copy_client.cc:308] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: tablet copy: Beginning tablet copy session from remote peer at address 127.9.151.129:33077
I20240503 02:14:07.318265 10449 ts_tablet_manager.cc:902] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: Initiating tablet copy from peer 7cd9a8da65604af78132e9045dac5180 (127.9.151.129:33077)
I20240503 02:14:07.319734 10449 tablet_copy_client.cc:308] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: tablet copy: Beginning tablet copy session from remote peer at address 127.9.151.129:33077
I20240503 02:14:07.326361 10449 ts_tablet_manager.cc:902] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: Initiating tablet copy from peer 7cd9a8da65604af78132e9045dac5180 (127.9.151.129:33077)
I20240503 02:14:07.331051 10449 tablet_copy_client.cc:308] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: tablet copy: Beginning tablet copy session from remote peer at address 127.9.151.129:33077
I20240503 02:14:07.333707 10449 ts_tablet_manager.cc:902] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: Initiating tablet copy from peer 7cd9a8da65604af78132e9045dac5180 (127.9.151.129:33077)
I20240503 02:14:07.335063 10449 tablet_copy_client.cc:308] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: tablet copy: Beginning tablet copy session from remote peer at address 127.9.151.129:33077
I20240503 02:14:07.351050 10449 ts_tablet_manager.cc:902] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: Initiating tablet copy from peer 7cd9a8da65604af78132e9045dac5180 (127.9.151.129:33077)
I20240503 02:14:07.352547 10449 tablet_copy_client.cc:308] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: tablet copy: Beginning tablet copy session from remote peer at address 127.9.151.129:33077
I20240503 02:14:07.357426 10449 ts_tablet_manager.cc:902] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: Initiating tablet copy from peer 7cd9a8da65604af78132e9045dac5180 (127.9.151.129:33077)
I20240503 02:14:07.359069 10449 tablet_copy_client.cc:308] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: tablet copy: Beginning tablet copy session from remote peer at address 127.9.151.129:33077
I20240503 02:14:07.369119 10449 ts_tablet_manager.cc:902] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: Initiating tablet copy from peer 7cd9a8da65604af78132e9045dac5180 (127.9.151.129:33077)
I20240503 02:14:07.370501 10449 tablet_copy_client.cc:308] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: tablet copy: Beginning tablet copy session from remote peer at address 127.9.151.129:33077
I20240503 02:14:07.380348 10449 ts_tablet_manager.cc:902] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: Initiating tablet copy from peer 7cd9a8da65604af78132e9045dac5180 (127.9.151.129:33077)
I20240503 02:14:07.381731 10449 tablet_copy_client.cc:308] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: tablet copy: Beginning tablet copy session from remote peer at address 127.9.151.129:33077
I20240503 02:14:07.390831 10449 ts_tablet_manager.cc:902] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: Initiating tablet copy from peer 7cd9a8da65604af78132e9045dac5180 (127.9.151.129:33077)
I20240503 02:14:07.392100 10449 tablet_copy_client.cc:308] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: tablet copy: Beginning tablet copy session from remote peer at address 127.9.151.129:33077
I20240503 02:14:07.408987 10449 ts_tablet_manager.cc:902] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: Initiating tablet copy from peer 7cd9a8da65604af78132e9045dac5180 (127.9.151.129:33077)
I20240503 02:14:07.410562 10449 tablet_copy_client.cc:308] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: tablet copy: Beginning tablet copy session from remote peer at address 127.9.151.129:33077
I20240503 02:14:07.413213 10452 ts_tablet_manager.cc:902] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: Initiating tablet copy from peer 7cd9a8da65604af78132e9045dac5180 (127.9.151.129:33077)
I20240503 02:14:07.415059 10452 tablet_copy_client.cc:308] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: tablet copy: Beginning tablet copy session from remote peer at address 127.9.151.129:33077
I20240503 02:14:07.423657 10452 ts_tablet_manager.cc:902] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: Initiating tablet copy from peer 7cd9a8da65604af78132e9045dac5180 (127.9.151.129:33077)
I20240503 02:14:07.424830 10452 tablet_copy_client.cc:308] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: tablet copy: Beginning tablet copy session from remote peer at address 127.9.151.129:33077
I20240503 02:14:07.427346 10452 ts_tablet_manager.cc:902] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: Initiating tablet copy from peer 7cd9a8da65604af78132e9045dac5180 (127.9.151.129:33077)
I20240503 02:14:07.428604 10452 tablet_copy_client.cc:308] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: tablet copy: Beginning tablet copy session from remote peer at address 127.9.151.129:33077
I20240503 02:14:07.437288 10452 ts_tablet_manager.cc:902] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: Initiating tablet copy from peer 7cd9a8da65604af78132e9045dac5180 (127.9.151.129:33077)
I20240503 02:14:07.438594 10452 tablet_copy_client.cc:308] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: tablet copy: Beginning tablet copy session from remote peer at address 127.9.151.129:33077
I20240503 02:14:07.441731 10452 ts_tablet_manager.cc:902] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: Initiating tablet copy from peer 7cd9a8da65604af78132e9045dac5180 (127.9.151.129:33077)
I20240503 02:14:07.442991 10452 tablet_copy_client.cc:308] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: tablet copy: Beginning tablet copy session from remote peer at address 127.9.151.129:33077
I20240503 02:14:07.445618 10452 ts_tablet_manager.cc:902] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: Initiating tablet copy from peer 7cd9a8da65604af78132e9045dac5180 (127.9.151.129:33077)
I20240503 02:14:07.446939 10452 tablet_copy_client.cc:308] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: tablet copy: Beginning tablet copy session from remote peer at address 127.9.151.129:33077
I20240503 02:14:07.450057 10452 ts_tablet_manager.cc:902] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: Initiating tablet copy from peer 7cd9a8da65604af78132e9045dac5180 (127.9.151.129:33077)
I20240503 02:14:07.451256 10452 tablet_copy_client.cc:308] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: tablet copy: Beginning tablet copy session from remote peer at address 127.9.151.129:33077
I20240503 02:14:07.461598 10452 ts_tablet_manager.cc:902] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: Initiating tablet copy from peer 7cd9a8da65604af78132e9045dac5180 (127.9.151.129:33077)
I20240503 02:14:07.462881 10452 tablet_copy_client.cc:308] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: tablet copy: Beginning tablet copy session from remote peer at address 127.9.151.129:33077
I20240503 02:14:07.466166 10452 ts_tablet_manager.cc:902] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: Initiating tablet copy from peer 7cd9a8da65604af78132e9045dac5180 (127.9.151.129:33077)
I20240503 02:14:07.471535 10452 tablet_copy_client.cc:308] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: tablet copy: Beginning tablet copy session from remote peer at address 127.9.151.129:33077
I20240503 02:14:07.477895 10452 ts_tablet_manager.cc:902] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: Initiating tablet copy from peer 7cd9a8da65604af78132e9045dac5180 (127.9.151.129:33077)
I20240503 02:14:07.479297 10452 tablet_copy_client.cc:308] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: tablet copy: Beginning tablet copy session from remote peer at address 127.9.151.129:33077
I20240503 02:14:07.496558 10452 ts_tablet_manager.cc:902] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: Initiating tablet copy from peer 7cd9a8da65604af78132e9045dac5180 (127.9.151.129:33077)
I20240503 02:14:07.499295 10452 tablet_copy_client.cc:308] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: tablet copy: Beginning tablet copy session from remote peer at address 127.9.151.129:33077
I20240503 02:14:07.507302 10452 ts_tablet_manager.cc:902] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: Initiating tablet copy from peer 7cd9a8da65604af78132e9045dac5180 (127.9.151.129:33077)
I20240503 02:14:07.509045 10452 tablet_copy_client.cc:308] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: tablet copy: Beginning tablet copy session from remote peer at address 127.9.151.129:33077
I20240503 02:14:07.524883 10452 ts_tablet_manager.cc:902] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: Initiating tablet copy from peer 7cd9a8da65604af78132e9045dac5180 (127.9.151.129:33077)
I20240503 02:14:07.527482 10452 tablet_copy_client.cc:308] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: tablet copy: Beginning tablet copy session from remote peer at address 127.9.151.129:33077
I20240503 02:14:07.535342 10452 ts_tablet_manager.cc:902] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: Initiating tablet copy from peer 7cd9a8da65604af78132e9045dac5180 (127.9.151.129:33077)
I20240503 02:14:07.536779 10452 tablet_copy_client.cc:308] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: tablet copy: Beginning tablet copy session from remote peer at address 127.9.151.129:33077
I20240503 02:14:07.543084 10452 ts_tablet_manager.cc:902] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: Initiating tablet copy from peer 7cd9a8da65604af78132e9045dac5180 (127.9.151.129:33077)
I20240503 02:14:07.550522 10452 tablet_copy_client.cc:308] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: tablet copy: Beginning tablet copy session from remote peer at address 127.9.151.129:33077
I20240503 02:14:07.571413 10452 ts_tablet_manager.cc:902] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: Initiating tablet copy from peer 7cd9a8da65604af78132e9045dac5180 (127.9.151.129:33077)
I20240503 02:14:07.573060 10452 tablet_copy_client.cc:308] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: tablet copy: Beginning tablet copy session from remote peer at address 127.9.151.129:33077
I20240503 02:14:07.575660 10452 ts_tablet_manager.cc:902] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: Initiating tablet copy from peer 7cd9a8da65604af78132e9045dac5180 (127.9.151.129:33077)
I20240503 02:14:07.579115 10452 tablet_copy_client.cc:308] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: tablet copy: Beginning tablet copy session from remote peer at address 127.9.151.129:33077
I20240503 02:14:07.589955 10452 ts_tablet_manager.cc:902] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: Initiating tablet copy from peer 7cd9a8da65604af78132e9045dac5180 (127.9.151.129:33077)
I20240503 02:14:07.592221 10452 tablet_copy_client.cc:308] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: tablet copy: Beginning tablet copy session from remote peer at address 127.9.151.129:33077
I20240503 02:14:07.605520 10452 ts_tablet_manager.cc:902] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: Initiating tablet copy from peer 7cd9a8da65604af78132e9045dac5180 (127.9.151.129:33077)
I20240503 02:14:07.606829 10452 tablet_copy_client.cc:308] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: tablet copy: Beginning tablet copy session from remote peer at address 127.9.151.129:33077
I20240503 02:14:07.622874 10452 ts_tablet_manager.cc:902] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: Initiating tablet copy from peer 7cd9a8da65604af78132e9045dac5180 (127.9.151.129:33077)
I20240503 02:14:07.624183 10452 tablet_copy_client.cc:308] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: tablet copy: Beginning tablet copy session from remote peer at address 127.9.151.129:33077
W20240503 02:14:07.627619 10448 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240503 02:14:07.627936 10448 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240503 02:14:07.628026 10448 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20240503 02:14:07.629670 10452 ts_tablet_manager.cc:902] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: Initiating tablet copy from peer 7cd9a8da65604af78132e9045dac5180 (127.9.151.129:33077)
I20240503 02:14:07.630878 10452 tablet_copy_client.cc:308] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: tablet copy: Beginning tablet copy session from remote peer at address 127.9.151.129:33077
W20240503 02:14:07.637589 10448 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240503 02:14:07.637805 10448 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.151.129
I20240503 02:14:07.640193 10452 ts_tablet_manager.cc:902] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: Initiating tablet copy from peer 7cd9a8da65604af78132e9045dac5180 (127.9.151.129:33077)
I20240503 02:14:07.641533 10452 tablet_copy_client.cc:308] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: tablet copy: Beginning tablet copy session from remote peer at address 127.9.151.129:33077
I20240503 02:14:07.648937 10448 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.151.148:38865
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknrxqHk/test-tmp/tablet_copy_client_session-itest.0.TabletCopyClientSessionITest.TestStartTabletCopyWhileSourceBootstrapping.1714702442467385-9822-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasknrxqHk/test-tmp/tablet_copy_client_session-itest.0.TabletCopyClientSessionITest.TestStartTabletCopyWhileSourceBootstrapping.1714702442467385-9822-0/minicluster-data/ts-0/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.151.129:33077
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknrxqHk/test-tmp/tablet_copy_client_session-itest.0.TabletCopyClientSessionITest.TestStartTabletCopyWhileSourceBootstrapping.1714702442467385-9822-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.151.129
--webserver_port=43885
--tserver_master_addrs=127.9.151.190:36085
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.151.129
--log_dir=/tmp/dist-test-tasknrxqHk/test-tmp/tablet_copy_client_session-itest.0.TabletCopyClientSessionITest.TestStartTabletCopyWhileSourceBootstrapping.1714702442467385-9822-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c39e72a98e124e5a12080bac42105b996594d806
build type FASTDEBUG
built by None at 03 May 2024 01:43:22 UTC on efd1337a2aa3
build id 99
ASAN enabled
I20240503 02:14:07.650400 10448 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240503 02:14:07.652300 10448 file_cache.cc:492] Constructed file cache file cache with capacity 419430
I20240503 02:14:07.654458 10452 ts_tablet_manager.cc:902] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: Initiating tablet copy from peer 7cd9a8da65604af78132e9045dac5180 (127.9.151.129:33077)
I20240503 02:14:07.655771 10452 tablet_copy_client.cc:308] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: tablet copy: Beginning tablet copy session from remote peer at address 127.9.151.129:33077
W20240503 02:14:07.667658 10617 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240503 02:14:07.668187 10618 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240503 02:14:07.668655 10620 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240503 02:14:07.668974 10452 ts_tablet_manager.cc:902] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: Initiating tablet copy from peer 7cd9a8da65604af78132e9045dac5180 (127.9.151.129:33077)
I20240503 02:14:07.670429 10452 tablet_copy_client.cc:308] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: tablet copy: Beginning tablet copy session from remote peer at address 127.9.151.129:33077
I20240503 02:14:07.670802 10448 server_base.cc:1034] running on GCE node
I20240503 02:14:07.671625 10448 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240503 02:14:07.672967 10448 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240503 02:14:07.674283 10448 hybrid_clock.cc:648] HybridClock initialized: now 1714702447674196 us; error 57 us; skew 500 ppm
I20240503 02:14:07.674650 10448 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20240503 02:14:07.677876 10448 webserver.cc:460] Webserver started at http://127.9.151.129:43885/ using document root <none> and password file <none>
I20240503 02:14:07.678819 10448 fs_manager.cc:359] Metadata directory not provided
I20240503 02:14:07.679021 10448 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240503 02:14:07.679522 10452 ts_tablet_manager.cc:902] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: Initiating tablet copy from peer 7cd9a8da65604af78132e9045dac5180 (127.9.151.129:33077)
I20240503 02:14:07.687520 10448 fs_manager.cc:709] Time spent opening directory manager: real 0.006s user 0.007s sys 0.000s
I20240503 02:14:07.689086 10452 tablet_copy_client.cc:308] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: tablet copy: Beginning tablet copy session from remote peer at address 127.9.151.129:33077
I20240503 02:14:07.694047 10634 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240503 02:14:07.696341 10448 fs_manager.cc:725] Time spent opening block manager: real 0.007s user 0.006s sys 0.001s
I20240503 02:14:07.696653 10448 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknrxqHk/test-tmp/tablet_copy_client_session-itest.0.TabletCopyClientSessionITest.TestStartTabletCopyWhileSourceBootstrapping.1714702442467385-9822-0/minicluster-data/ts-0/data,/tmp/dist-test-tasknrxqHk/test-tmp/tablet_copy_client_session-itest.0.TabletCopyClientSessionITest.TestStartTabletCopyWhileSourceBootstrapping.1714702442467385-9822-0/minicluster-data/ts-0/wal
uuid: "7cd9a8da65604af78132e9045dac5180"
format_stamp: "Formatted at 2024-05-03 02:14:03 on dist-test-slave-twlh"
I20240503 02:14:07.697788 10448 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknrxqHk/test-tmp/tablet_copy_client_session-itest.0.TabletCopyClientSessionITest.TestStartTabletCopyWhileSourceBootstrapping.1714702442467385-9822-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasknrxqHk/test-tmp/tablet_copy_client_session-itest.0.TabletCopyClientSessionITest.TestStartTabletCopyWhileSourceBootstrapping.1714702442467385-9822-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasknrxqHk/test-tmp/tablet_copy_client_session-itest.0.TabletCopyClientSessionITest.TestStartTabletCopyWhileSourceBootstrapping.1714702442467385-9822-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240503 02:14:07.702040 10452 ts_tablet_manager.cc:902] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: Initiating tablet copy from peer 7cd9a8da65604af78132e9045dac5180 (127.9.151.129:33077)
I20240503 02:14:07.703481 10452 tablet_copy_client.cc:308] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: tablet copy: Beginning tablet copy session from remote peer at address 127.9.151.129:33077
I20240503 02:14:07.711275 10452 ts_tablet_manager.cc:902] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: Initiating tablet copy from peer 7cd9a8da65604af78132e9045dac5180 (127.9.151.129:33077)
I20240503 02:14:07.713580 10452 tablet_copy_client.cc:308] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: tablet copy: Beginning tablet copy session from remote peer at address 127.9.151.129:33077
I20240503 02:14:07.719230 10448 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240503 02:14:07.720160 10448 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240503 02:14:07.720450 10448 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240503 02:14:07.721493 10448 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240503 02:14:07.723975 10649 ts_tablet_manager.cc:522] Loading tablet metadata (0/1 complete)
I20240503 02:14:07.730074 10448 ts_tablet_manager.cc:559] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20240503 02:14:07.730315 10448 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.007s user 0.000s sys 0.001s
I20240503 02:14:07.730453 10448 ts_tablet_manager.cc:574] Registering tablets (0/1 complete)
I20240503 02:14:07.730706 10452 ts_tablet_manager.cc:902] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: Initiating tablet copy from peer 7cd9a8da65604af78132e9045dac5180 (127.9.151.129:33077)
I20240503 02:14:07.733673 10452 tablet_copy_client.cc:308] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: tablet copy: Beginning tablet copy session from remote peer at address 127.9.151.129:33077
I20240503 02:14:07.734360 10448 ts_tablet_manager.cc:590] Registered 1 tablets
I20240503 02:14:07.734479 10448 ts_tablet_manager.cc:569] Time spent register tablets: real 0.004s user 0.001s sys 0.000s
I20240503 02:14:07.735029 10649 tablet_bootstrap.cc:492] T 375647a877014ca48ca32ff220098cd1 P 7cd9a8da65604af78132e9045dac5180: Bootstrap starting.
I20240503 02:14:07.746201 10452 ts_tablet_manager.cc:902] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: Initiating tablet copy from peer 7cd9a8da65604af78132e9045dac5180 (127.9.151.129:33077)
I20240503 02:14:07.787295 10452 tablet_copy_client.cc:308] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: tablet copy: Beginning tablet copy session from remote peer at address 127.9.151.129:33077
I20240503 02:14:07.802418 10452 ts_tablet_manager.cc:902] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: Initiating tablet copy from peer 7cd9a8da65604af78132e9045dac5180 (127.9.151.129:33077)
I20240503 02:14:07.806174 10452 tablet_copy_client.cc:308] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: tablet copy: Beginning tablet copy session from remote peer at address 127.9.151.129:33077
I20240503 02:14:07.811188 10452 ts_tablet_manager.cc:902] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: Initiating tablet copy from peer 7cd9a8da65604af78132e9045dac5180 (127.9.151.129:33077)
I20240503 02:14:07.817057 10452 tablet_copy_client.cc:308] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: tablet copy: Beginning tablet copy session from remote peer at address 127.9.151.129:33077
I20240503 02:14:07.831174 10452 ts_tablet_manager.cc:902] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: Initiating tablet copy from peer 7cd9a8da65604af78132e9045dac5180 (127.9.151.129:33077)
I20240503 02:14:07.841405 10452 tablet_copy_client.cc:308] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: tablet copy: Beginning tablet copy session from remote peer at address 127.9.151.129:33077
I20240503 02:14:07.848189 10452 ts_tablet_manager.cc:902] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: Initiating tablet copy from peer 7cd9a8da65604af78132e9045dac5180 (127.9.151.129:33077)
I20240503 02:14:07.853794 10452 tablet_copy_client.cc:308] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: tablet copy: Beginning tablet copy session from remote peer at address 127.9.151.129:33077
I20240503 02:14:07.856997 10452 ts_tablet_manager.cc:902] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: Initiating tablet copy from peer 7cd9a8da65604af78132e9045dac5180 (127.9.151.129:33077)
I20240503 02:14:07.863799 10452 tablet_copy_client.cc:308] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: tablet copy: Beginning tablet copy session from remote peer at address 127.9.151.129:33077
I20240503 02:14:07.866771 10452 ts_tablet_manager.cc:902] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: Initiating tablet copy from peer 7cd9a8da65604af78132e9045dac5180 (127.9.151.129:33077)
I20240503 02:14:07.885887 10452 tablet_copy_client.cc:308] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: tablet copy: Beginning tablet copy session from remote peer at address 127.9.151.129:33077
I20240503 02:14:07.889554 10452 ts_tablet_manager.cc:902] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: Initiating tablet copy from peer 7cd9a8da65604af78132e9045dac5180 (127.9.151.129:33077)
I20240503 02:14:07.891528 10452 tablet_copy_client.cc:308] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: tablet copy: Beginning tablet copy session from remote peer at address 127.9.151.129:33077
I20240503 02:14:07.895992 10448 rpc_server.cc:307] RPC server started. Bound to: 127.9.151.129:33077
I20240503 02:14:07.896042 10792 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.151.129:33077 every 8 connection(s)
I20240503 02:14:07.897737 10448 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknrxqHk/test-tmp/tablet_copy_client_session-itest.0.TabletCopyClientSessionITest.TestStartTabletCopyWhileSourceBootstrapping.1714702442467385-9822-0/minicluster-data/ts-0/data/info.pb
I20240503 02:14:07.899528 9822 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknrxqHk/build/asan/bin/kudu as pid 10448
I20240503 02:14:07.905745 10452 ts_tablet_manager.cc:902] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: Initiating tablet copy from peer 7cd9a8da65604af78132e9045dac5180 (127.9.151.129:33077)
I20240503 02:14:07.910090 10452 tablet_copy_client.cc:308] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: tablet copy: Beginning tablet copy session from remote peer at address 127.9.151.129:33077
I20240503 02:14:07.911250 10649 log.cc:824] T 375647a877014ca48ca32ff220098cd1 P 7cd9a8da65604af78132e9045dac5180: Log is configured to *not* fsync() on all Append() calls
I20240503 02:14:07.920797 10793 heartbeater.cc:347] Connected to a master server at 127.9.151.190:36085
I20240503 02:14:07.921221 10793 heartbeater.cc:464] Registering TS with master...
I20240503 02:14:07.922497 10793 heartbeater.cc:511] Master 127.9.151.190:36085 requested a full tablet report, sending...
I20240503 02:14:07.925671 10221 ts_manager.cc:194] Re-registered known tserver with Master: 7cd9a8da65604af78132e9045dac5180 (127.9.151.129:33077)
I20240503 02:14:07.927280 10221 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.9.151.129:50155
I20240503 02:14:07.929040 10752 tablet_copy_service.cc:140] P 7cd9a8da65604af78132e9045dac5180: Received BeginTabletCopySession request for tablet 375647a877014ca48ca32ff220098cd1 from peer 4c8a3af9b5184a51a3c0fe3221c01c75 ({username='slave'} at 127.9.151.130:57223)
I20240503 02:14:07.929415 10752 tablet_copy_service.cc:161] P 7cd9a8da65604af78132e9045dac5180: Beginning new tablet copy session on tablet 375647a877014ca48ca32ff220098cd1 from peer 4c8a3af9b5184a51a3c0fe3221c01c75 at {username='slave'} at 127.9.151.130:57223: session id = 4c8a3af9b5184a51a3c0fe3221c01c75-375647a877014ca48ca32ff220098cd1
W20240503 02:14:07.931926 10752 tablet_copy_service.cc:478] P 7cd9a8da65604af78132e9045dac5180: Error handling TabletCopyService RPC request from {username='slave'} at 127.9.151.130:57223: Error beginning tablet copy session for tablet 375647a877014ca48ca32ff220098cd1: Illegal state: The tablet is not in a running state: BOOTSTRAPPING
I20240503 02:14:07.948392 10452 ts_tablet_manager.cc:902] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: Initiating tablet copy from peer 7cd9a8da65604af78132e9045dac5180 (127.9.151.129:33077)
I20240503 02:14:07.949586 10452 tablet_copy_client.cc:308] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: tablet copy: Beginning tablet copy session from remote peer at address 127.9.151.129:33077
I20240503 02:14:07.950405 10752 tablet_copy_service.cc:140] P 7cd9a8da65604af78132e9045dac5180: Received BeginTabletCopySession request for tablet 375647a877014ca48ca32ff220098cd1 from peer 4c8a3af9b5184a51a3c0fe3221c01c75 ({username='slave'} at 127.9.151.130:57223)
I20240503 02:14:07.950611 10752 tablet_copy_service.cc:161] P 7cd9a8da65604af78132e9045dac5180: Beginning new tablet copy session on tablet 375647a877014ca48ca32ff220098cd1 from peer 4c8a3af9b5184a51a3c0fe3221c01c75 at {username='slave'} at 127.9.151.130:57223: session id = 4c8a3af9b5184a51a3c0fe3221c01c75-375647a877014ca48ca32ff220098cd1
W20240503 02:14:07.950820 10752 tablet_copy_service.cc:478] P 7cd9a8da65604af78132e9045dac5180: Error handling TabletCopyService RPC request from {username='slave'} at 127.9.151.130:57223: Error beginning tablet copy session for tablet 375647a877014ca48ca32ff220098cd1: Illegal state: The tablet is not in a running state: BOOTSTRAPPING
I20240503 02:14:07.953086 10452 ts_tablet_manager.cc:902] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: Initiating tablet copy from peer 7cd9a8da65604af78132e9045dac5180 (127.9.151.129:33077)
I20240503 02:14:07.963624 10452 tablet_copy_client.cc:308] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: tablet copy: Beginning tablet copy session from remote peer at address 127.9.151.129:33077
I20240503 02:14:07.964708 10752 tablet_copy_service.cc:140] P 7cd9a8da65604af78132e9045dac5180: Received BeginTabletCopySession request for tablet 375647a877014ca48ca32ff220098cd1 from peer 4c8a3af9b5184a51a3c0fe3221c01c75 ({username='slave'} at 127.9.151.130:57223)
I20240503 02:14:07.964876 10752 tablet_copy_service.cc:161] P 7cd9a8da65604af78132e9045dac5180: Beginning new tablet copy session on tablet 375647a877014ca48ca32ff220098cd1 from peer 4c8a3af9b5184a51a3c0fe3221c01c75 at {username='slave'} at 127.9.151.130:57223: session id = 4c8a3af9b5184a51a3c0fe3221c01c75-375647a877014ca48ca32ff220098cd1
W20240503 02:14:07.965085 10752 tablet_copy_service.cc:478] P 7cd9a8da65604af78132e9045dac5180: Error handling TabletCopyService RPC request from {username='slave'} at 127.9.151.130:57223: Error beginning tablet copy session for tablet 375647a877014ca48ca32ff220098cd1: Illegal state: The tablet is not in a running state: BOOTSTRAPPING
I20240503 02:14:07.977331 10452 ts_tablet_manager.cc:902] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: Initiating tablet copy from peer 7cd9a8da65604af78132e9045dac5180 (127.9.151.129:33077)
I20240503 02:14:07.978735 10452 tablet_copy_client.cc:308] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: tablet copy: Beginning tablet copy session from remote peer at address 127.9.151.129:33077
I20240503 02:14:07.979746 10752 tablet_copy_service.cc:140] P 7cd9a8da65604af78132e9045dac5180: Received BeginTabletCopySession request for tablet 375647a877014ca48ca32ff220098cd1 from peer 4c8a3af9b5184a51a3c0fe3221c01c75 ({username='slave'} at 127.9.151.130:57223)
I20240503 02:14:07.979996 10752 tablet_copy_service.cc:161] P 7cd9a8da65604af78132e9045dac5180: Beginning new tablet copy session on tablet 375647a877014ca48ca32ff220098cd1 from peer 4c8a3af9b5184a51a3c0fe3221c01c75 at {username='slave'} at 127.9.151.130:57223: session id = 4c8a3af9b5184a51a3c0fe3221c01c75-375647a877014ca48ca32ff220098cd1
W20240503 02:14:07.980221 10752 tablet_copy_service.cc:478] P 7cd9a8da65604af78132e9045dac5180: Error handling TabletCopyService RPC request from {username='slave'} at 127.9.151.130:57223: Error beginning tablet copy session for tablet 375647a877014ca48ca32ff220098cd1: Illegal state: The tablet is not in a running state: BOOTSTRAPPING
I20240503 02:14:07.984535 10452 ts_tablet_manager.cc:902] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: Initiating tablet copy from peer 7cd9a8da65604af78132e9045dac5180 (127.9.151.129:33077)
I20240503 02:14:07.986035 10452 tablet_copy_client.cc:308] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: tablet copy: Beginning tablet copy session from remote peer at address 127.9.151.129:33077
I20240503 02:14:07.987071 10752 tablet_copy_service.cc:140] P 7cd9a8da65604af78132e9045dac5180: Received BeginTabletCopySession request for tablet 375647a877014ca48ca32ff220098cd1 from peer 4c8a3af9b5184a51a3c0fe3221c01c75 ({username='slave'} at 127.9.151.130:57223)
I20240503 02:14:07.987321 10752 tablet_copy_service.cc:161] P 7cd9a8da65604af78132e9045dac5180: Beginning new tablet copy session on tablet 375647a877014ca48ca32ff220098cd1 from peer 4c8a3af9b5184a51a3c0fe3221c01c75 at {username='slave'} at 127.9.151.130:57223: session id = 4c8a3af9b5184a51a3c0fe3221c01c75-375647a877014ca48ca32ff220098cd1
W20240503 02:14:07.987562 10752 tablet_copy_service.cc:478] P 7cd9a8da65604af78132e9045dac5180: Error handling TabletCopyService RPC request from {username='slave'} at 127.9.151.130:57223: Error beginning tablet copy session for tablet 375647a877014ca48ca32ff220098cd1: Illegal state: The tablet is not in a running state: BOOTSTRAPPING
I20240503 02:14:08.001324 10452 ts_tablet_manager.cc:902] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: Initiating tablet copy from peer 7cd9a8da65604af78132e9045dac5180 (127.9.151.129:33077)
I20240503 02:14:08.003507 10452 tablet_copy_client.cc:308] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: tablet copy: Beginning tablet copy session from remote peer at address 127.9.151.129:33077
I20240503 02:14:08.004405 10752 tablet_copy_service.cc:140] P 7cd9a8da65604af78132e9045dac5180: Received BeginTabletCopySession request for tablet 375647a877014ca48ca32ff220098cd1 from peer 4c8a3af9b5184a51a3c0fe3221c01c75 ({username='slave'} at 127.9.151.130:57223)
I20240503 02:14:08.004642 10752 tablet_copy_service.cc:161] P 7cd9a8da65604af78132e9045dac5180: Beginning new tablet copy session on tablet 375647a877014ca48ca32ff220098cd1 from peer 4c8a3af9b5184a51a3c0fe3221c01c75 at {username='slave'} at 127.9.151.130:57223: session id = 4c8a3af9b5184a51a3c0fe3221c01c75-375647a877014ca48ca32ff220098cd1
W20240503 02:14:08.004868 10752 tablet_copy_service.cc:478] P 7cd9a8da65604af78132e9045dac5180: Error handling TabletCopyService RPC request from {username='slave'} at 127.9.151.130:57223: Error beginning tablet copy session for tablet 375647a877014ca48ca32ff220098cd1: Illegal state: The tablet is not in a running state: BOOTSTRAPPING
I20240503 02:14:08.007432 10452 ts_tablet_manager.cc:902] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: Initiating tablet copy from peer 7cd9a8da65604af78132e9045dac5180 (127.9.151.129:33077)
I20240503 02:14:08.008599 10452 tablet_copy_client.cc:308] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: tablet copy: Beginning tablet copy session from remote peer at address 127.9.151.129:33077
I20240503 02:14:08.009410 10752 tablet_copy_service.cc:140] P 7cd9a8da65604af78132e9045dac5180: Received BeginTabletCopySession request for tablet 375647a877014ca48ca32ff220098cd1 from peer 4c8a3af9b5184a51a3c0fe3221c01c75 ({username='slave'} at 127.9.151.130:57223)
I20240503 02:14:08.009601 10752 tablet_copy_service.cc:161] P 7cd9a8da65604af78132e9045dac5180: Beginning new tablet copy session on tablet 375647a877014ca48ca32ff220098cd1 from peer 4c8a3af9b5184a51a3c0fe3221c01c75 at {username='slave'} at 127.9.151.130:57223: session id = 4c8a3af9b5184a51a3c0fe3221c01c75-375647a877014ca48ca32ff220098cd1
W20240503 02:14:08.009807 10752 tablet_copy_service.cc:478] P 7cd9a8da65604af78132e9045dac5180: Error handling TabletCopyService RPC request from {username='slave'} at 127.9.151.130:57223: Error beginning tablet copy session for tablet 375647a877014ca48ca32ff220098cd1: Illegal state: The tablet is not in a running state: BOOTSTRAPPING
I20240503 02:14:08.012249 10452 ts_tablet_manager.cc:902] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: Initiating tablet copy from peer 7cd9a8da65604af78132e9045dac5180 (127.9.151.129:33077)
I20240503 02:14:08.013345 10452 tablet_copy_client.cc:308] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: tablet copy: Beginning tablet copy session from remote peer at address 127.9.151.129:33077
I20240503 02:14:08.014154 10752 tablet_copy_service.cc:140] P 7cd9a8da65604af78132e9045dac5180: Received BeginTabletCopySession request for tablet 375647a877014ca48ca32ff220098cd1 from peer 4c8a3af9b5184a51a3c0fe3221c01c75 ({username='slave'} at 127.9.151.130:57223)
I20240503 02:14:08.014344 10752 tablet_copy_service.cc:161] P 7cd9a8da65604af78132e9045dac5180: Beginning new tablet copy session on tablet 375647a877014ca48ca32ff220098cd1 from peer 4c8a3af9b5184a51a3c0fe3221c01c75 at {username='slave'} at 127.9.151.130:57223: session id = 4c8a3af9b5184a51a3c0fe3221c01c75-375647a877014ca48ca32ff220098cd1
W20240503 02:14:08.014510 10752 tablet_copy_service.cc:478] P 7cd9a8da65604af78132e9045dac5180: Error handling TabletCopyService RPC request from {username='slave'} at 127.9.151.130:57223: Error beginning tablet copy session for tablet 375647a877014ca48ca32ff220098cd1: Illegal state: The tablet is not in a running state: BOOTSTRAPPING
I20240503 02:14:08.038978 10452 ts_tablet_manager.cc:902] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: Initiating tablet copy from peer 7cd9a8da65604af78132e9045dac5180 (127.9.151.129:33077)
I20240503 02:14:08.040757 10452 tablet_copy_client.cc:308] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: tablet copy: Beginning tablet copy session from remote peer at address 127.9.151.129:33077
I20240503 02:14:08.041808 10752 tablet_copy_service.cc:140] P 7cd9a8da65604af78132e9045dac5180: Received BeginTabletCopySession request for tablet 375647a877014ca48ca32ff220098cd1 from peer 4c8a3af9b5184a51a3c0fe3221c01c75 ({username='slave'} at 127.9.151.130:57223)
I20240503 02:14:08.042016 10752 tablet_copy_service.cc:161] P 7cd9a8da65604af78132e9045dac5180: Beginning new tablet copy session on tablet 375647a877014ca48ca32ff220098cd1 from peer 4c8a3af9b5184a51a3c0fe3221c01c75 at {username='slave'} at 127.9.151.130:57223: session id = 4c8a3af9b5184a51a3c0fe3221c01c75-375647a877014ca48ca32ff220098cd1
W20240503 02:14:08.042222 10752 tablet_copy_service.cc:478] P 7cd9a8da65604af78132e9045dac5180: Error handling TabletCopyService RPC request from {username='slave'} at 127.9.151.130:57223: Error beginning tablet copy session for tablet 375647a877014ca48ca32ff220098cd1: Illegal state: The tablet is not in a running state: BOOTSTRAPPING
I20240503 02:14:08.049039 10452 ts_tablet_manager.cc:902] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: Initiating tablet copy from peer 7cd9a8da65604af78132e9045dac5180 (127.9.151.129:33077)
I20240503 02:14:08.050956 10452 tablet_copy_client.cc:308] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: tablet copy: Beginning tablet copy session from remote peer at address 127.9.151.129:33077
I20240503 02:14:08.051811 10752 tablet_copy_service.cc:140] P 7cd9a8da65604af78132e9045dac5180: Received BeginTabletCopySession request for tablet 375647a877014ca48ca32ff220098cd1 from peer 4c8a3af9b5184a51a3c0fe3221c01c75 ({username='slave'} at 127.9.151.130:57223)
I20240503 02:14:08.052045 10752 tablet_copy_service.cc:161] P 7cd9a8da65604af78132e9045dac5180: Beginning new tablet copy session on tablet 375647a877014ca48ca32ff220098cd1 from peer 4c8a3af9b5184a51a3c0fe3221c01c75 at {username='slave'} at 127.9.151.130:57223: session id = 4c8a3af9b5184a51a3c0fe3221c01c75-375647a877014ca48ca32ff220098cd1
W20240503 02:14:08.052238 10752 tablet_copy_service.cc:478] P 7cd9a8da65604af78132e9045dac5180: Error handling TabletCopyService RPC request from {username='slave'} at 127.9.151.130:57223: Error beginning tablet copy session for tablet 375647a877014ca48ca32ff220098cd1: Illegal state: The tablet is not in a running state: BOOTSTRAPPING
I20240503 02:14:08.054718 10452 ts_tablet_manager.cc:902] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: Initiating tablet copy from peer 7cd9a8da65604af78132e9045dac5180 (127.9.151.129:33077)
I20240503 02:14:08.056308 10452 tablet_copy_client.cc:308] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: tablet copy: Beginning tablet copy session from remote peer at address 127.9.151.129:33077
I20240503 02:14:08.057304 10752 tablet_copy_service.cc:140] P 7cd9a8da65604af78132e9045dac5180: Received BeginTabletCopySession request for tablet 375647a877014ca48ca32ff220098cd1 from peer 4c8a3af9b5184a51a3c0fe3221c01c75 ({username='slave'} at 127.9.151.130:57223)
I20240503 02:14:08.057509 10752 tablet_copy_service.cc:161] P 7cd9a8da65604af78132e9045dac5180: Beginning new tablet copy session on tablet 375647a877014ca48ca32ff220098cd1 from peer 4c8a3af9b5184a51a3c0fe3221c01c75 at {username='slave'} at 127.9.151.130:57223: session id = 4c8a3af9b5184a51a3c0fe3221c01c75-375647a877014ca48ca32ff220098cd1
W20240503 02:14:08.057751 10752 tablet_copy_service.cc:478] P 7cd9a8da65604af78132e9045dac5180: Error handling TabletCopyService RPC request from {username='slave'} at 127.9.151.130:57223: Error beginning tablet copy session for tablet 375647a877014ca48ca32ff220098cd1: Illegal state: The tablet is not in a running state: BOOTSTRAPPING
I20240503 02:14:08.068075 10452 ts_tablet_manager.cc:902] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: Initiating tablet copy from peer 7cd9a8da65604af78132e9045dac5180 (127.9.151.129:33077)
I20240503 02:14:08.069368 10452 tablet_copy_client.cc:308] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: tablet copy: Beginning tablet copy session from remote peer at address 127.9.151.129:33077
I20240503 02:14:08.070123 10752 tablet_copy_service.cc:140] P 7cd9a8da65604af78132e9045dac5180: Received BeginTabletCopySession request for tablet 375647a877014ca48ca32ff220098cd1 from peer 4c8a3af9b5184a51a3c0fe3221c01c75 ({username='slave'} at 127.9.151.130:57223)
I20240503 02:14:08.070333 10752 tablet_copy_service.cc:161] P 7cd9a8da65604af78132e9045dac5180: Beginning new tablet copy session on tablet 375647a877014ca48ca32ff220098cd1 from peer 4c8a3af9b5184a51a3c0fe3221c01c75 at {username='slave'} at 127.9.151.130:57223: session id = 4c8a3af9b5184a51a3c0fe3221c01c75-375647a877014ca48ca32ff220098cd1
W20240503 02:14:08.070528 10752 tablet_copy_service.cc:478] P 7cd9a8da65604af78132e9045dac5180: Error handling TabletCopyService RPC request from {username='slave'} at 127.9.151.130:57223: Error beginning tablet copy session for tablet 375647a877014ca48ca32ff220098cd1: Illegal state: The tablet is not in a running state: BOOTSTRAPPING
I20240503 02:14:08.072999 10452 ts_tablet_manager.cc:902] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: Initiating tablet copy from peer 7cd9a8da65604af78132e9045dac5180 (127.9.151.129:33077)
I20240503 02:14:08.075250 10452 tablet_copy_client.cc:308] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: tablet copy: Beginning tablet copy session from remote peer at address 127.9.151.129:33077
I20240503 02:14:08.076009 10752 tablet_copy_service.cc:140] P 7cd9a8da65604af78132e9045dac5180: Received BeginTabletCopySession request for tablet 375647a877014ca48ca32ff220098cd1 from peer 4c8a3af9b5184a51a3c0fe3221c01c75 ({username='slave'} at 127.9.151.130:57223)
I20240503 02:14:08.076157 10752 tablet_copy_service.cc:161] P 7cd9a8da65604af78132e9045dac5180: Beginning new tablet copy session on tablet 375647a877014ca48ca32ff220098cd1 from peer 4c8a3af9b5184a51a3c0fe3221c01c75 at {username='slave'} at 127.9.151.130:57223: session id = 4c8a3af9b5184a51a3c0fe3221c01c75-375647a877014ca48ca32ff220098cd1
W20240503 02:14:08.076351 10752 tablet_copy_service.cc:478] P 7cd9a8da65604af78132e9045dac5180: Error handling TabletCopyService RPC request from {username='slave'} at 127.9.151.130:57223: Error beginning tablet copy session for tablet 375647a877014ca48ca32ff220098cd1: Illegal state: The tablet is not in a running state: BOOTSTRAPPING
I20240503 02:14:08.085032 10452 ts_tablet_manager.cc:902] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: Initiating tablet copy from peer 7cd9a8da65604af78132e9045dac5180 (127.9.151.129:33077)
I20240503 02:14:08.094944 10452 tablet_copy_client.cc:308] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: tablet copy: Beginning tablet copy session from remote peer at address 127.9.151.129:33077
I20240503 02:14:08.095788 10752 tablet_copy_service.cc:140] P 7cd9a8da65604af78132e9045dac5180: Received BeginTabletCopySession request for tablet 375647a877014ca48ca32ff220098cd1 from peer 4c8a3af9b5184a51a3c0fe3221c01c75 ({username='slave'} at 127.9.151.130:57223)
I20240503 02:14:08.096011 10752 tablet_copy_service.cc:161] P 7cd9a8da65604af78132e9045dac5180: Beginning new tablet copy session on tablet 375647a877014ca48ca32ff220098cd1 from peer 4c8a3af9b5184a51a3c0fe3221c01c75 at {username='slave'} at 127.9.151.130:57223: session id = 4c8a3af9b5184a51a3c0fe3221c01c75-375647a877014ca48ca32ff220098cd1
W20240503 02:14:08.096212 10752 tablet_copy_service.cc:478] P 7cd9a8da65604af78132e9045dac5180: Error handling TabletCopyService RPC request from {username='slave'} at 127.9.151.130:57223: Error beginning tablet copy session for tablet 375647a877014ca48ca32ff220098cd1: Illegal state: The tablet is not in a running state: BOOTSTRAPPING
I20240503 02:14:08.101989 10437 heartbeater.cc:503] Master 127.9.151.190:36085 was elected leader, sending a full tablet report...
I20240503 02:14:08.112620 10452 ts_tablet_manager.cc:902] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: Initiating tablet copy from peer 7cd9a8da65604af78132e9045dac5180 (127.9.151.129:33077)
I20240503 02:14:08.113888 10452 tablet_copy_client.cc:308] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: tablet copy: Beginning tablet copy session from remote peer at address 127.9.151.129:33077
I20240503 02:14:08.114910 10752 tablet_copy_service.cc:140] P 7cd9a8da65604af78132e9045dac5180: Received BeginTabletCopySession request for tablet 375647a877014ca48ca32ff220098cd1 from peer 4c8a3af9b5184a51a3c0fe3221c01c75 ({username='slave'} at 127.9.151.130:57223)
I20240503 02:14:08.115103 10752 tablet_copy_service.cc:161] P 7cd9a8da65604af78132e9045dac5180: Beginning new tablet copy session on tablet 375647a877014ca48ca32ff220098cd1 from peer 4c8a3af9b5184a51a3c0fe3221c01c75 at {username='slave'} at 127.9.151.130:57223: session id = 4c8a3af9b5184a51a3c0fe3221c01c75-375647a877014ca48ca32ff220098cd1
W20240503 02:14:08.115330 10752 tablet_copy_service.cc:478] P 7cd9a8da65604af78132e9045dac5180: Error handling TabletCopyService RPC request from {username='slave'} at 127.9.151.130:57223: Error beginning tablet copy session for tablet 375647a877014ca48ca32ff220098cd1: Illegal state: The tablet is not in a running state: BOOTSTRAPPING
I20240503 02:14:08.121052 10452 ts_tablet_manager.cc:902] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: Initiating tablet copy from peer 7cd9a8da65604af78132e9045dac5180 (127.9.151.129:33077)
I20240503 02:14:08.122346 10452 tablet_copy_client.cc:308] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: tablet copy: Beginning tablet copy session from remote peer at address 127.9.151.129:33077
I20240503 02:14:08.123314 10752 tablet_copy_service.cc:140] P 7cd9a8da65604af78132e9045dac5180: Received BeginTabletCopySession request for tablet 375647a877014ca48ca32ff220098cd1 from peer 4c8a3af9b5184a51a3c0fe3221c01c75 ({username='slave'} at 127.9.151.130:57223)
I20240503 02:14:08.123515 10752 tablet_copy_service.cc:161] P 7cd9a8da65604af78132e9045dac5180: Beginning new tablet copy session on tablet 375647a877014ca48ca32ff220098cd1 from peer 4c8a3af9b5184a51a3c0fe3221c01c75 at {username='slave'} at 127.9.151.130:57223: session id = 4c8a3af9b5184a51a3c0fe3221c01c75-375647a877014ca48ca32ff220098cd1
W20240503 02:14:08.123711 10752 tablet_copy_service.cc:478] P 7cd9a8da65604af78132e9045dac5180: Error handling TabletCopyService RPC request from {username='slave'} at 127.9.151.130:57223: Error beginning tablet copy session for tablet 375647a877014ca48ca32ff220098cd1: Illegal state: The tablet is not in a running state: BOOTSTRAPPING
I20240503 02:14:08.127259 10452 ts_tablet_manager.cc:902] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: Initiating tablet copy from peer 7cd9a8da65604af78132e9045dac5180 (127.9.151.129:33077)
I20240503 02:14:08.134893 10452 tablet_copy_client.cc:308] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: tablet copy: Beginning tablet copy session from remote peer at address 127.9.151.129:33077
I20240503 02:14:08.135779 10752 tablet_copy_service.cc:140] P 7cd9a8da65604af78132e9045dac5180: Received BeginTabletCopySession request for tablet 375647a877014ca48ca32ff220098cd1 from peer 4c8a3af9b5184a51a3c0fe3221c01c75 ({username='slave'} at 127.9.151.130:57223)
I20240503 02:14:08.135974 10752 tablet_copy_service.cc:161] P 7cd9a8da65604af78132e9045dac5180: Beginning new tablet copy session on tablet 375647a877014ca48ca32ff220098cd1 from peer 4c8a3af9b5184a51a3c0fe3221c01c75 at {username='slave'} at 127.9.151.130:57223: session id = 4c8a3af9b5184a51a3c0fe3221c01c75-375647a877014ca48ca32ff220098cd1
W20240503 02:14:08.136364 10752 tablet_copy_service.cc:478] P 7cd9a8da65604af78132e9045dac5180: Error handling TabletCopyService RPC request from {username='slave'} at 127.9.151.130:57223: Error beginning tablet copy session for tablet 375647a877014ca48ca32ff220098cd1: Illegal state: The tablet is not in a running state: BOOTSTRAPPING
I20240503 02:14:08.139173 10452 ts_tablet_manager.cc:902] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: Initiating tablet copy from peer 7cd9a8da65604af78132e9045dac5180 (127.9.151.129:33077)
I20240503 02:14:08.140456 10452 tablet_copy_client.cc:308] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: tablet copy: Beginning tablet copy session from remote peer at address 127.9.151.129:33077
I20240503 02:14:08.141392 10752 tablet_copy_service.cc:140] P 7cd9a8da65604af78132e9045dac5180: Received BeginTabletCopySession request for tablet 375647a877014ca48ca32ff220098cd1 from peer 4c8a3af9b5184a51a3c0fe3221c01c75 ({username='slave'} at 127.9.151.130:57223)
I20240503 02:14:08.141635 10752 tablet_copy_service.cc:161] P 7cd9a8da65604af78132e9045dac5180: Beginning new tablet copy session on tablet 375647a877014ca48ca32ff220098cd1 from peer 4c8a3af9b5184a51a3c0fe3221c01c75 at {username='slave'} at 127.9.151.130:57223: session id = 4c8a3af9b5184a51a3c0fe3221c01c75-375647a877014ca48ca32ff220098cd1
W20240503 02:14:08.141862 10752 tablet_copy_service.cc:478] P 7cd9a8da65604af78132e9045dac5180: Error handling TabletCopyService RPC request from {username='slave'} at 127.9.151.130:57223: Error beginning tablet copy session for tablet 375647a877014ca48ca32ff220098cd1: Illegal state: The tablet is not in a running state: BOOTSTRAPPING
I20240503 02:14:08.147264 10452 ts_tablet_manager.cc:902] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: Initiating tablet copy from peer 7cd9a8da65604af78132e9045dac5180 (127.9.151.129:33077)
I20240503 02:14:08.148556 10452 tablet_copy_client.cc:308] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: tablet copy: Beginning tablet copy session from remote peer at address 127.9.151.129:33077
I20240503 02:14:08.149379 10752 tablet_copy_service.cc:140] P 7cd9a8da65604af78132e9045dac5180: Received BeginTabletCopySession request for tablet 375647a877014ca48ca32ff220098cd1 from peer 4c8a3af9b5184a51a3c0fe3221c01c75 ({username='slave'} at 127.9.151.130:57223)
I20240503 02:14:08.149612 10752 tablet_copy_service.cc:161] P 7cd9a8da65604af78132e9045dac5180: Beginning new tablet copy session on tablet 375647a877014ca48ca32ff220098cd1 from peer 4c8a3af9b5184a51a3c0fe3221c01c75 at {username='slave'} at 127.9.151.130:57223: session id = 4c8a3af9b5184a51a3c0fe3221c01c75-375647a877014ca48ca32ff220098cd1
W20240503 02:14:08.149832 10752 tablet_copy_service.cc:478] P 7cd9a8da65604af78132e9045dac5180: Error handling TabletCopyService RPC request from {username='slave'} at 127.9.151.130:57223: Error beginning tablet copy session for tablet 375647a877014ca48ca32ff220098cd1: Illegal state: The tablet is not in a running state: BOOTSTRAPPING
I20240503 02:14:08.171612 10452 ts_tablet_manager.cc:902] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: Initiating tablet copy from peer 7cd9a8da65604af78132e9045dac5180 (127.9.151.129:33077)
I20240503 02:14:08.172973 10452 tablet_copy_client.cc:308] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: tablet copy: Beginning tablet copy session from remote peer at address 127.9.151.129:33077
I20240503 02:14:08.173818 10752 tablet_copy_service.cc:140] P 7cd9a8da65604af78132e9045dac5180: Received BeginTabletCopySession request for tablet 375647a877014ca48ca32ff220098cd1 from peer 4c8a3af9b5184a51a3c0fe3221c01c75 ({username='slave'} at 127.9.151.130:57223)
I20240503 02:14:08.174060 10752 tablet_copy_service.cc:161] P 7cd9a8da65604af78132e9045dac5180: Beginning new tablet copy session on tablet 375647a877014ca48ca32ff220098cd1 from peer 4c8a3af9b5184a51a3c0fe3221c01c75 at {username='slave'} at 127.9.151.130:57223: session id = 4c8a3af9b5184a51a3c0fe3221c01c75-375647a877014ca48ca32ff220098cd1
W20240503 02:14:08.174288 10752 tablet_copy_service.cc:478] P 7cd9a8da65604af78132e9045dac5180: Error handling TabletCopyService RPC request from {username='slave'} at 127.9.151.130:57223: Error beginning tablet copy session for tablet 375647a877014ca48ca32ff220098cd1: Illegal state: The tablet is not in a running state: BOOTSTRAPPING
I20240503 02:14:08.182919 10452 ts_tablet_manager.cc:902] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: Initiating tablet copy from peer 7cd9a8da65604af78132e9045dac5180 (127.9.151.129:33077)
I20240503 02:14:08.184377 10452 tablet_copy_client.cc:308] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: tablet copy: Beginning tablet copy session from remote peer at address 127.9.151.129:33077
I20240503 02:14:08.185209 10752 tablet_copy_service.cc:140] P 7cd9a8da65604af78132e9045dac5180: Received BeginTabletCopySession request for tablet 375647a877014ca48ca32ff220098cd1 from peer 4c8a3af9b5184a51a3c0fe3221c01c75 ({username='slave'} at 127.9.151.130:57223)
I20240503 02:14:08.185410 10752 tablet_copy_service.cc:161] P 7cd9a8da65604af78132e9045dac5180: Beginning new tablet copy session on tablet 375647a877014ca48ca32ff220098cd1 from peer 4c8a3af9b5184a51a3c0fe3221c01c75 at {username='slave'} at 127.9.151.130:57223: session id = 4c8a3af9b5184a51a3c0fe3221c01c75-375647a877014ca48ca32ff220098cd1
W20240503 02:14:08.185614 10752 tablet_copy_service.cc:478] P 7cd9a8da65604af78132e9045dac5180: Error handling TabletCopyService RPC request from {username='slave'} at 127.9.151.130:57223: Error beginning tablet copy session for tablet 375647a877014ca48ca32ff220098cd1: Illegal state: The tablet is not in a running state: BOOTSTRAPPING
I20240503 02:14:08.200042 10452 ts_tablet_manager.cc:902] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: Initiating tablet copy from peer 7cd9a8da65604af78132e9045dac5180 (127.9.151.129:33077)
I20240503 02:14:08.201330 10452 tablet_copy_client.cc:308] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: tablet copy: Beginning tablet copy session from remote peer at address 127.9.151.129:33077
I20240503 02:14:08.202158 10752 tablet_copy_service.cc:140] P 7cd9a8da65604af78132e9045dac5180: Received BeginTabletCopySession request for tablet 375647a877014ca48ca32ff220098cd1 from peer 4c8a3af9b5184a51a3c0fe3221c01c75 ({username='slave'} at 127.9.151.130:57223)
I20240503 02:14:08.202370 10752 tablet_copy_service.cc:161] P 7cd9a8da65604af78132e9045dac5180: Beginning new tablet copy session on tablet 375647a877014ca48ca32ff220098cd1 from peer 4c8a3af9b5184a51a3c0fe3221c01c75 at {username='slave'} at 127.9.151.130:57223: session id = 4c8a3af9b5184a51a3c0fe3221c01c75-375647a877014ca48ca32ff220098cd1
W20240503 02:14:08.202559 10752 tablet_copy_service.cc:478] P 7cd9a8da65604af78132e9045dac5180: Error handling TabletCopyService RPC request from {username='slave'} at 127.9.151.130:57223: Error beginning tablet copy session for tablet 375647a877014ca48ca32ff220098cd1: Illegal state: The tablet is not in a running state: BOOTSTRAPPING
I20240503 02:14:08.205718 10452 ts_tablet_manager.cc:902] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: Initiating tablet copy from peer 7cd9a8da65604af78132e9045dac5180 (127.9.151.129:33077)
I20240503 02:14:08.207000 10452 tablet_copy_client.cc:308] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: tablet copy: Beginning tablet copy session from remote peer at address 127.9.151.129:33077
I20240503 02:14:08.207739 10752 tablet_copy_service.cc:140] P 7cd9a8da65604af78132e9045dac5180: Received BeginTabletCopySession request for tablet 375647a877014ca48ca32ff220098cd1 from peer 4c8a3af9b5184a51a3c0fe3221c01c75 ({username='slave'} at 127.9.151.130:57223)
I20240503 02:14:08.207897 10752 tablet_copy_service.cc:161] P 7cd9a8da65604af78132e9045dac5180: Beginning new tablet copy session on tablet 375647a877014ca48ca32ff220098cd1 from peer 4c8a3af9b5184a51a3c0fe3221c01c75 at {username='slave'} at 127.9.151.130:57223: session id = 4c8a3af9b5184a51a3c0fe3221c01c75-375647a877014ca48ca32ff220098cd1
W20240503 02:14:08.208097 10752 tablet_copy_service.cc:478] P 7cd9a8da65604af78132e9045dac5180: Error handling TabletCopyService RPC request from {username='slave'} at 127.9.151.130:57223: Error beginning tablet copy session for tablet 375647a877014ca48ca32ff220098cd1: Illegal state: The tablet is not in a running state: BOOTSTRAPPING
I20240503 02:14:08.222608 10677 ts_tablet_manager.cc:902] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: Initiating tablet copy from peer 7cd9a8da65604af78132e9045dac5180 (127.9.151.129:33077)
I20240503 02:14:08.230455 10677 tablet_copy_client.cc:308] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: tablet copy: Beginning tablet copy session from remote peer at address 127.9.151.129:33077
I20240503 02:14:08.231483 10752 tablet_copy_service.cc:140] P 7cd9a8da65604af78132e9045dac5180: Received BeginTabletCopySession request for tablet 375647a877014ca48ca32ff220098cd1 from peer 4c8a3af9b5184a51a3c0fe3221c01c75 ({username='slave'} at 127.9.151.130:57223)
I20240503 02:14:08.231686 10752 tablet_copy_service.cc:161] P 7cd9a8da65604af78132e9045dac5180: Beginning new tablet copy session on tablet 375647a877014ca48ca32ff220098cd1 from peer 4c8a3af9b5184a51a3c0fe3221c01c75 at {username='slave'} at 127.9.151.130:57223: session id = 4c8a3af9b5184a51a3c0fe3221c01c75-375647a877014ca48ca32ff220098cd1
W20240503 02:14:08.231891 10752 tablet_copy_service.cc:478] P 7cd9a8da65604af78132e9045dac5180: Error handling TabletCopyService RPC request from {username='slave'} at 127.9.151.130:57223: Error beginning tablet copy session for tablet 375647a877014ca48ca32ff220098cd1: Illegal state: The tablet is not in a running state: BOOTSTRAPPING
I20240503 02:14:08.245220 10677 ts_tablet_manager.cc:902] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: Initiating tablet copy from peer 7cd9a8da65604af78132e9045dac5180 (127.9.151.129:33077)
I20240503 02:14:08.246533 10677 tablet_copy_client.cc:308] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: tablet copy: Beginning tablet copy session from remote peer at address 127.9.151.129:33077
I20240503 02:14:08.247422 10752 tablet_copy_service.cc:140] P 7cd9a8da65604af78132e9045dac5180: Received BeginTabletCopySession request for tablet 375647a877014ca48ca32ff220098cd1 from peer 4c8a3af9b5184a51a3c0fe3221c01c75 ({username='slave'} at 127.9.151.130:57223)
I20240503 02:14:08.247615 10752 tablet_copy_service.cc:161] P 7cd9a8da65604af78132e9045dac5180: Beginning new tablet copy session on tablet 375647a877014ca48ca32ff220098cd1 from peer 4c8a3af9b5184a51a3c0fe3221c01c75 at {username='slave'} at 127.9.151.130:57223: session id = 4c8a3af9b5184a51a3c0fe3221c01c75-375647a877014ca48ca32ff220098cd1
W20240503 02:14:08.247800 10752 tablet_copy_service.cc:478] P 7cd9a8da65604af78132e9045dac5180: Error handling TabletCopyService RPC request from {username='slave'} at 127.9.151.130:57223: Error beginning tablet copy session for tablet 375647a877014ca48ca32ff220098cd1: Illegal state: The tablet is not in a running state: BOOTSTRAPPING
I20240503 02:14:08.250432 10677 ts_tablet_manager.cc:902] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: Initiating tablet copy from peer 7cd9a8da65604af78132e9045dac5180 (127.9.151.129:33077)
I20240503 02:14:08.251667 10677 tablet_copy_client.cc:308] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: tablet copy: Beginning tablet copy session from remote peer at address 127.9.151.129:33077
I20240503 02:14:08.252475 10752 tablet_copy_service.cc:140] P 7cd9a8da65604af78132e9045dac5180: Received BeginTabletCopySession request for tablet 375647a877014ca48ca32ff220098cd1 from peer 4c8a3af9b5184a51a3c0fe3221c01c75 ({username='slave'} at 127.9.151.130:57223)
I20240503 02:14:08.252679 10752 tablet_copy_service.cc:161] P 7cd9a8da65604af78132e9045dac5180: Beginning new tablet copy session on tablet 375647a877014ca48ca32ff220098cd1 from peer 4c8a3af9b5184a51a3c0fe3221c01c75 at {username='slave'} at 127.9.151.130:57223: session id = 4c8a3af9b5184a51a3c0fe3221c01c75-375647a877014ca48ca32ff220098cd1
W20240503 02:14:08.252853 10752 tablet_copy_service.cc:478] P 7cd9a8da65604af78132e9045dac5180: Error handling TabletCopyService RPC request from {username='slave'} at 127.9.151.130:57223: Error beginning tablet copy session for tablet 375647a877014ca48ca32ff220098cd1: Illegal state: The tablet is not in a running state: BOOTSTRAPPING
I20240503 02:14:08.261981 10677 ts_tablet_manager.cc:902] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: Initiating tablet copy from peer 7cd9a8da65604af78132e9045dac5180 (127.9.151.129:33077)
I20240503 02:14:08.263259 10677 tablet_copy_client.cc:308] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: tablet copy: Beginning tablet copy session from remote peer at address 127.9.151.129:33077
I20240503 02:14:08.264124 10752 tablet_copy_service.cc:140] P 7cd9a8da65604af78132e9045dac5180: Received BeginTabletCopySession request for tablet 375647a877014ca48ca32ff220098cd1 from peer 4c8a3af9b5184a51a3c0fe3221c01c75 ({username='slave'} at 127.9.151.130:57223)
I20240503 02:14:08.264293 10752 tablet_copy_service.cc:161] P 7cd9a8da65604af78132e9045dac5180: Beginning new tablet copy session on tablet 375647a877014ca48ca32ff220098cd1 from peer 4c8a3af9b5184a51a3c0fe3221c01c75 at {username='slave'} at 127.9.151.130:57223: session id = 4c8a3af9b5184a51a3c0fe3221c01c75-375647a877014ca48ca32ff220098cd1
W20240503 02:14:08.264469 10752 tablet_copy_service.cc:478] P 7cd9a8da65604af78132e9045dac5180: Error handling TabletCopyService RPC request from {username='slave'} at 127.9.151.130:57223: Error beginning tablet copy session for tablet 375647a877014ca48ca32ff220098cd1: Illegal state: The tablet is not in a running state: BOOTSTRAPPING
I20240503 02:14:08.267798 10677 ts_tablet_manager.cc:902] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: Initiating tablet copy from peer 7cd9a8da65604af78132e9045dac5180 (127.9.151.129:33077)
I20240503 02:14:08.272819 10677 tablet_copy_client.cc:308] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: tablet copy: Beginning tablet copy session from remote peer at address 127.9.151.129:33077
I20240503 02:14:08.273823 10752 tablet_copy_service.cc:140] P 7cd9a8da65604af78132e9045dac5180: Received BeginTabletCopySession request for tablet 375647a877014ca48ca32ff220098cd1 from peer 4c8a3af9b5184a51a3c0fe3221c01c75 ({username='slave'} at 127.9.151.130:57223)
I20240503 02:14:08.274201 10752 tablet_copy_service.cc:161] P 7cd9a8da65604af78132e9045dac5180: Beginning new tablet copy session on tablet 375647a877014ca48ca32ff220098cd1 from peer 4c8a3af9b5184a51a3c0fe3221c01c75 at {username='slave'} at 127.9.151.130:57223: session id = 4c8a3af9b5184a51a3c0fe3221c01c75-375647a877014ca48ca32ff220098cd1
W20240503 02:14:08.274557 10752 tablet_copy_service.cc:478] P 7cd9a8da65604af78132e9045dac5180: Error handling TabletCopyService RPC request from {username='slave'} at 127.9.151.130:57223: Error beginning tablet copy session for tablet 375647a877014ca48ca32ff220098cd1: Illegal state: The tablet is not in a running state: BOOTSTRAPPING
I20240503 02:14:08.283742 10677 ts_tablet_manager.cc:902] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: Initiating tablet copy from peer 7cd9a8da65604af78132e9045dac5180 (127.9.151.129:33077)
I20240503 02:14:08.285082 10677 tablet_copy_client.cc:308] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: tablet copy: Beginning tablet copy session from remote peer at address 127.9.151.129:33077
I20240503 02:14:08.286023 10752 tablet_copy_service.cc:140] P 7cd9a8da65604af78132e9045dac5180: Received BeginTabletCopySession request for tablet 375647a877014ca48ca32ff220098cd1 from peer 4c8a3af9b5184a51a3c0fe3221c01c75 ({username='slave'} at 127.9.151.130:57223)
I20240503 02:14:08.286229 10752 tablet_copy_service.cc:161] P 7cd9a8da65604af78132e9045dac5180: Beginning new tablet copy session on tablet 375647a877014ca48ca32ff220098cd1 from peer 4c8a3af9b5184a51a3c0fe3221c01c75 at {username='slave'} at 127.9.151.130:57223: session id = 4c8a3af9b5184a51a3c0fe3221c01c75-375647a877014ca48ca32ff220098cd1
W20240503 02:14:08.286442 10752 tablet_copy_service.cc:478] P 7cd9a8da65604af78132e9045dac5180: Error handling TabletCopyService RPC request from {username='slave'} at 127.9.151.130:57223: Error beginning tablet copy session for tablet 375647a877014ca48ca32ff220098cd1: Illegal state: The tablet is not in a running state: BOOTSTRAPPING
I20240503 02:14:08.304170 10677 ts_tablet_manager.cc:902] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: Initiating tablet copy from peer 7cd9a8da65604af78132e9045dac5180 (127.9.151.129:33077)
I20240503 02:14:08.308737 10677 tablet_copy_client.cc:308] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: tablet copy: Beginning tablet copy session from remote peer at address 127.9.151.129:33077
I20240503 02:14:08.309653 10752 tablet_copy_service.cc:140] P 7cd9a8da65604af78132e9045dac5180: Received BeginTabletCopySession request for tablet 375647a877014ca48ca32ff220098cd1 from peer 4c8a3af9b5184a51a3c0fe3221c01c75 ({username='slave'} at 127.9.151.130:57223)
I20240503 02:14:08.309867 10752 tablet_copy_service.cc:161] P 7cd9a8da65604af78132e9045dac5180: Beginning new tablet copy session on tablet 375647a877014ca48ca32ff220098cd1 from peer 4c8a3af9b5184a51a3c0fe3221c01c75 at {username='slave'} at 127.9.151.130:57223: session id = 4c8a3af9b5184a51a3c0fe3221c01c75-375647a877014ca48ca32ff220098cd1
W20240503 02:14:08.310060 10752 tablet_copy_service.cc:478] P 7cd9a8da65604af78132e9045dac5180: Error handling TabletCopyService RPC request from {username='slave'} at 127.9.151.130:57223: Error beginning tablet copy session for tablet 375647a877014ca48ca32ff220098cd1: Illegal state: The tablet is not in a running state: BOOTSTRAPPING
I20240503 02:14:08.312330 10452 ts_tablet_manager.cc:902] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: Initiating tablet copy from peer 7cd9a8da65604af78132e9045dac5180 (127.9.151.129:33077)
I20240503 02:14:08.315069 10452 tablet_copy_client.cc:308] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: tablet copy: Beginning tablet copy session from remote peer at address 127.9.151.129:33077
I20240503 02:14:08.315963 10752 tablet_copy_service.cc:140] P 7cd9a8da65604af78132e9045dac5180: Received BeginTabletCopySession request for tablet 375647a877014ca48ca32ff220098cd1 from peer 4c8a3af9b5184a51a3c0fe3221c01c75 ({username='slave'} at 127.9.151.130:57223)
I20240503 02:14:08.316159 10752 tablet_copy_service.cc:161] P 7cd9a8da65604af78132e9045dac5180: Beginning new tablet copy session on tablet 375647a877014ca48ca32ff220098cd1 from peer 4c8a3af9b5184a51a3c0fe3221c01c75 at {username='slave'} at 127.9.151.130:57223: session id = 4c8a3af9b5184a51a3c0fe3221c01c75-375647a877014ca48ca32ff220098cd1
W20240503 02:14:08.316355 10752 tablet_copy_service.cc:478] P 7cd9a8da65604af78132e9045dac5180: Error handling TabletCopyService RPC request from {username='slave'} at 127.9.151.130:57223: Error beginning tablet copy session for tablet 375647a877014ca48ca32ff220098cd1: Illegal state: The tablet is not in a running state: BOOTSTRAPPING
I20240503 02:14:08.321424 10452 ts_tablet_manager.cc:902] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: Initiating tablet copy from peer 7cd9a8da65604af78132e9045dac5180 (127.9.151.129:33077)
I20240503 02:14:08.322657 10452 tablet_copy_client.cc:308] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: tablet copy: Beginning tablet copy session from remote peer at address 127.9.151.129:33077
I20240503 02:14:08.323657 10752 tablet_copy_service.cc:140] P 7cd9a8da65604af78132e9045dac5180: Received BeginTabletCopySession request for tablet 375647a877014ca48ca32ff220098cd1 from peer 4c8a3af9b5184a51a3c0fe3221c01c75 ({username='slave'} at 127.9.151.130:57223)
I20240503 02:14:08.323869 10752 tablet_copy_service.cc:161] P 7cd9a8da65604af78132e9045dac5180: Beginning new tablet copy session on tablet 375647a877014ca48ca32ff220098cd1 from peer 4c8a3af9b5184a51a3c0fe3221c01c75 at {username='slave'} at 127.9.151.130:57223: session id = 4c8a3af9b5184a51a3c0fe3221c01c75-375647a877014ca48ca32ff220098cd1
W20240503 02:14:08.324038 10752 tablet_copy_service.cc:478] P 7cd9a8da65604af78132e9045dac5180: Error handling TabletCopyService RPC request from {username='slave'} at 127.9.151.130:57223: Error beginning tablet copy session for tablet 375647a877014ca48ca32ff220098cd1: Illegal state: The tablet is not in a running state: BOOTSTRAPPING
I20240503 02:14:08.347788 10452 ts_tablet_manager.cc:902] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: Initiating tablet copy from peer 7cd9a8da65604af78132e9045dac5180 (127.9.151.129:33077)
I20240503 02:14:08.349022 10452 tablet_copy_client.cc:308] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: tablet copy: Beginning tablet copy session from remote peer at address 127.9.151.129:33077
I20240503 02:14:08.349777 10752 tablet_copy_service.cc:140] P 7cd9a8da65604af78132e9045dac5180: Received BeginTabletCopySession request for tablet 375647a877014ca48ca32ff220098cd1 from peer 4c8a3af9b5184a51a3c0fe3221c01c75 ({username='slave'} at 127.9.151.130:57223)
I20240503 02:14:08.349927 10752 tablet_copy_service.cc:161] P 7cd9a8da65604af78132e9045dac5180: Beginning new tablet copy session on tablet 375647a877014ca48ca32ff220098cd1 from peer 4c8a3af9b5184a51a3c0fe3221c01c75 at {username='slave'} at 127.9.151.130:57223: session id = 4c8a3af9b5184a51a3c0fe3221c01c75-375647a877014ca48ca32ff220098cd1
W20240503 02:14:08.350136 10752 tablet_copy_service.cc:478] P 7cd9a8da65604af78132e9045dac5180: Error handling TabletCopyService RPC request from {username='slave'} at 127.9.151.130:57223: Error beginning tablet copy session for tablet 375647a877014ca48ca32ff220098cd1: Illegal state: The tablet is not in a running state: BOOTSTRAPPING
I20240503 02:14:08.353586 10452 ts_tablet_manager.cc:902] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: Initiating tablet copy from peer 7cd9a8da65604af78132e9045dac5180 (127.9.151.129:33077)
I20240503 02:14:08.365350 10452 tablet_copy_client.cc:308] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: tablet copy: Beginning tablet copy session from remote peer at address 127.9.151.129:33077
I20240503 02:14:08.366075 10752 tablet_copy_service.cc:140] P 7cd9a8da65604af78132e9045dac5180: Received BeginTabletCopySession request for tablet 375647a877014ca48ca32ff220098cd1 from peer 4c8a3af9b5184a51a3c0fe3221c01c75 ({username='slave'} at 127.9.151.130:57223)
I20240503 02:14:08.366295 10752 tablet_copy_service.cc:161] P 7cd9a8da65604af78132e9045dac5180: Beginning new tablet copy session on tablet 375647a877014ca48ca32ff220098cd1 from peer 4c8a3af9b5184a51a3c0fe3221c01c75 at {username='slave'} at 127.9.151.130:57223: session id = 4c8a3af9b5184a51a3c0fe3221c01c75-375647a877014ca48ca32ff220098cd1
W20240503 02:14:08.366506 10752 tablet_copy_service.cc:478] P 7cd9a8da65604af78132e9045dac5180: Error handling TabletCopyService RPC request from {username='slave'} at 127.9.151.130:57223: Error beginning tablet copy session for tablet 375647a877014ca48ca32ff220098cd1: Illegal state: The tablet is not in a running state: BOOTSTRAPPING
I20240503 02:14:08.383389 10452 ts_tablet_manager.cc:902] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: Initiating tablet copy from peer 7cd9a8da65604af78132e9045dac5180 (127.9.151.129:33077)
I20240503 02:14:08.384879 10452 tablet_copy_client.cc:308] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: tablet copy: Beginning tablet copy session from remote peer at address 127.9.151.129:33077
I20240503 02:14:08.385654 10752 tablet_copy_service.cc:140] P 7cd9a8da65604af78132e9045dac5180: Received BeginTabletCopySession request for tablet 375647a877014ca48ca32ff220098cd1 from peer 4c8a3af9b5184a51a3c0fe3221c01c75 ({username='slave'} at 127.9.151.130:57223)
I20240503 02:14:08.385840 10752 tablet_copy_service.cc:161] P 7cd9a8da65604af78132e9045dac5180: Beginning new tablet copy session on tablet 375647a877014ca48ca32ff220098cd1 from peer 4c8a3af9b5184a51a3c0fe3221c01c75 at {username='slave'} at 127.9.151.130:57223: session id = 4c8a3af9b5184a51a3c0fe3221c01c75-375647a877014ca48ca32ff220098cd1
W20240503 02:14:08.386070 10752 tablet_copy_service.cc:478] P 7cd9a8da65604af78132e9045dac5180: Error handling TabletCopyService RPC request from {username='slave'} at 127.9.151.130:57223: Error beginning tablet copy session for tablet 375647a877014ca48ca32ff220098cd1: Illegal state: The tablet is not in a running state: BOOTSTRAPPING
I20240503 02:14:08.389691 10452 ts_tablet_manager.cc:902] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: Initiating tablet copy from peer 7cd9a8da65604af78132e9045dac5180 (127.9.151.129:33077)
I20240503 02:14:08.398458 10452 tablet_copy_client.cc:308] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: tablet copy: Beginning tablet copy session from remote peer at address 127.9.151.129:33077
I20240503 02:14:08.399411 10752 tablet_copy_service.cc:140] P 7cd9a8da65604af78132e9045dac5180: Received BeginTabletCopySession request for tablet 375647a877014ca48ca32ff220098cd1 from peer 4c8a3af9b5184a51a3c0fe3221c01c75 ({username='slave'} at 127.9.151.130:57223)
I20240503 02:14:08.399730 10752 tablet_copy_service.cc:161] P 7cd9a8da65604af78132e9045dac5180: Beginning new tablet copy session on tablet 375647a877014ca48ca32ff220098cd1 from peer 4c8a3af9b5184a51a3c0fe3221c01c75 at {username='slave'} at 127.9.151.130:57223: session id = 4c8a3af9b5184a51a3c0fe3221c01c75-375647a877014ca48ca32ff220098cd1
W20240503 02:14:08.399942 10752 tablet_copy_service.cc:478] P 7cd9a8da65604af78132e9045dac5180: Error handling TabletCopyService RPC request from {username='slave'} at 127.9.151.130:57223: Error beginning tablet copy session for tablet 375647a877014ca48ca32ff220098cd1: Illegal state: The tablet is not in a running state: BOOTSTRAPPING
I20240503 02:14:08.403795 10452 ts_tablet_manager.cc:902] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: Initiating tablet copy from peer 7cd9a8da65604af78132e9045dac5180 (127.9.151.129:33077)
I20240503 02:14:08.405731 10452 tablet_copy_client.cc:308] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: tablet copy: Beginning tablet copy session from remote peer at address 127.9.151.129:33077
I20240503 02:14:08.406530 10752 tablet_copy_service.cc:140] P 7cd9a8da65604af78132e9045dac5180: Received BeginTabletCopySession request for tablet 375647a877014ca48ca32ff220098cd1 from peer 4c8a3af9b5184a51a3c0fe3221c01c75 ({username='slave'} at 127.9.151.130:57223)
I20240503 02:14:08.406734 10752 tablet_copy_service.cc:161] P 7cd9a8da65604af78132e9045dac5180: Beginning new tablet copy session on tablet 375647a877014ca48ca32ff220098cd1 from peer 4c8a3af9b5184a51a3c0fe3221c01c75 at {username='slave'} at 127.9.151.130:57223: session id = 4c8a3af9b5184a51a3c0fe3221c01c75-375647a877014ca48ca32ff220098cd1
W20240503 02:14:08.406934 10752 tablet_copy_service.cc:478] P 7cd9a8da65604af78132e9045dac5180: Error handling TabletCopyService RPC request from {username='slave'} at 127.9.151.130:57223: Error beginning tablet copy session for tablet 375647a877014ca48ca32ff220098cd1: Illegal state: The tablet is not in a running state: BOOTSTRAPPING
I20240503 02:14:08.412225 10452 ts_tablet_manager.cc:902] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: Initiating tablet copy from peer 7cd9a8da65604af78132e9045dac5180 (127.9.151.129:33077)
I20240503 02:14:08.413416 10452 tablet_copy_client.cc:308] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: tablet copy: Beginning tablet copy session from remote peer at address 127.9.151.129:33077
I20240503 02:14:08.414196 10752 tablet_copy_service.cc:140] P 7cd9a8da65604af78132e9045dac5180: Received BeginTabletCopySession request for tablet 375647a877014ca48ca32ff220098cd1 from peer 4c8a3af9b5184a51a3c0fe3221c01c75 ({username='slave'} at 127.9.151.130:57223)
I20240503 02:14:08.414402 10752 tablet_copy_service.cc:161] P 7cd9a8da65604af78132e9045dac5180: Beginning new tablet copy session on tablet 375647a877014ca48ca32ff220098cd1 from peer 4c8a3af9b5184a51a3c0fe3221c01c75 at {username='slave'} at 127.9.151.130:57223: session id = 4c8a3af9b5184a51a3c0fe3221c01c75-375647a877014ca48ca32ff220098cd1
W20240503 02:14:08.414587 10752 tablet_copy_service.cc:478] P 7cd9a8da65604af78132e9045dac5180: Error handling TabletCopyService RPC request from {username='slave'} at 127.9.151.130:57223: Error beginning tablet copy session for tablet 375647a877014ca48ca32ff220098cd1: Illegal state: The tablet is not in a running state: BOOTSTRAPPING
I20240503 02:14:08.433491 10452 ts_tablet_manager.cc:902] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: Initiating tablet copy from peer 7cd9a8da65604af78132e9045dac5180 (127.9.151.129:33077)
I20240503 02:14:08.435070 10452 tablet_copy_client.cc:308] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: tablet copy: Beginning tablet copy session from remote peer at address 127.9.151.129:33077
I20240503 02:14:08.435869 10752 tablet_copy_service.cc:140] P 7cd9a8da65604af78132e9045dac5180: Received BeginTabletCopySession request for tablet 375647a877014ca48ca32ff220098cd1 from peer 4c8a3af9b5184a51a3c0fe3221c01c75 ({username='slave'} at 127.9.151.130:57223)
I20240503 02:14:08.436045 10752 tablet_copy_service.cc:161] P 7cd9a8da65604af78132e9045dac5180: Beginning new tablet copy session on tablet 375647a877014ca48ca32ff220098cd1 from peer 4c8a3af9b5184a51a3c0fe3221c01c75 at {username='slave'} at 127.9.151.130:57223: session id = 4c8a3af9b5184a51a3c0fe3221c01c75-375647a877014ca48ca32ff220098cd1
W20240503 02:14:08.436271 10752 tablet_copy_service.cc:478] P 7cd9a8da65604af78132e9045dac5180: Error handling TabletCopyService RPC request from {username='slave'} at 127.9.151.130:57223: Error beginning tablet copy session for tablet 375647a877014ca48ca32ff220098cd1: Illegal state: The tablet is not in a running state: BOOTSTRAPPING
I20240503 02:14:08.438346 10452 ts_tablet_manager.cc:902] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: Initiating tablet copy from peer 7cd9a8da65604af78132e9045dac5180 (127.9.151.129:33077)
I20240503 02:14:08.440841 10452 tablet_copy_client.cc:308] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: tablet copy: Beginning tablet copy session from remote peer at address 127.9.151.129:33077
I20240503 02:14:08.441898 10752 tablet_copy_service.cc:140] P 7cd9a8da65604af78132e9045dac5180: Received BeginTabletCopySession request for tablet 375647a877014ca48ca32ff220098cd1 from peer 4c8a3af9b5184a51a3c0fe3221c01c75 ({username='slave'} at 127.9.151.130:57223)
I20240503 02:14:08.442206 10752 tablet_copy_service.cc:161] P 7cd9a8da65604af78132e9045dac5180: Beginning new tablet copy session on tablet 375647a877014ca48ca32ff220098cd1 from peer 4c8a3af9b5184a51a3c0fe3221c01c75 at {username='slave'} at 127.9.151.130:57223: session id = 4c8a3af9b5184a51a3c0fe3221c01c75-375647a877014ca48ca32ff220098cd1
W20240503 02:14:08.442584 10752 tablet_copy_service.cc:478] P 7cd9a8da65604af78132e9045dac5180: Error handling TabletCopyService RPC request from {username='slave'} at 127.9.151.130:57223: Error beginning tablet copy session for tablet 375647a877014ca48ca32ff220098cd1: Illegal state: The tablet is not in a running state: BOOTSTRAPPING
I20240503 02:14:08.448221 10452 ts_tablet_manager.cc:902] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: Initiating tablet copy from peer 7cd9a8da65604af78132e9045dac5180 (127.9.151.129:33077)
I20240503 02:14:08.455564 10452 tablet_copy_client.cc:308] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: tablet copy: Beginning tablet copy session from remote peer at address 127.9.151.129:33077
I20240503 02:14:08.456358 10752 tablet_copy_service.cc:140] P 7cd9a8da65604af78132e9045dac5180: Received BeginTabletCopySession request for tablet 375647a877014ca48ca32ff220098cd1 from peer 4c8a3af9b5184a51a3c0fe3221c01c75 ({username='slave'} at 127.9.151.130:57223)
I20240503 02:14:08.456568 10752 tablet_copy_service.cc:161] P 7cd9a8da65604af78132e9045dac5180: Beginning new tablet copy session on tablet 375647a877014ca48ca32ff220098cd1 from peer 4c8a3af9b5184a51a3c0fe3221c01c75 at {username='slave'} at 127.9.151.130:57223: session id = 4c8a3af9b5184a51a3c0fe3221c01c75-375647a877014ca48ca32ff220098cd1
W20240503 02:14:08.456900 10752 tablet_copy_service.cc:478] P 7cd9a8da65604af78132e9045dac5180: Error handling TabletCopyService RPC request from {username='slave'} at 127.9.151.130:57223: Error beginning tablet copy session for tablet 375647a877014ca48ca32ff220098cd1: Illegal state: The tablet is not in a running state: BOOTSTRAPPING
I20240503 02:14:08.482846 10452 ts_tablet_manager.cc:902] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: Initiating tablet copy from peer 7cd9a8da65604af78132e9045dac5180 (127.9.151.129:33077)
I20240503 02:14:08.484560 10452 tablet_copy_client.cc:308] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: tablet copy: Beginning tablet copy session from remote peer at address 127.9.151.129:33077
I20240503 02:14:08.485347 10752 tablet_copy_service.cc:140] P 7cd9a8da65604af78132e9045dac5180: Received BeginTabletCopySession request for tablet 375647a877014ca48ca32ff220098cd1 from peer 4c8a3af9b5184a51a3c0fe3221c01c75 ({username='slave'} at 127.9.151.130:57223)
I20240503 02:14:08.485548 10752 tablet_copy_service.cc:161] P 7cd9a8da65604af78132e9045dac5180: Beginning new tablet copy session on tablet 375647a877014ca48ca32ff220098cd1 from peer 4c8a3af9b5184a51a3c0fe3221c01c75 at {username='slave'} at 127.9.151.130:57223: session id = 4c8a3af9b5184a51a3c0fe3221c01c75-375647a877014ca48ca32ff220098cd1
W20240503 02:14:08.485746 10752 tablet_copy_service.cc:478] P 7cd9a8da65604af78132e9045dac5180: Error handling TabletCopyService RPC request from {username='slave'} at 127.9.151.130:57223: Error beginning tablet copy session for tablet 375647a877014ca48ca32ff220098cd1: Illegal state: The tablet is not in a running state: BOOTSTRAPPING
I20240503 02:14:08.488492 10452 ts_tablet_manager.cc:902] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: Initiating tablet copy from peer 7cd9a8da65604af78132e9045dac5180 (127.9.151.129:33077)
I20240503 02:14:08.489636 10452 tablet_copy_client.cc:308] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: tablet copy: Beginning tablet copy session from remote peer at address 127.9.151.129:33077
I20240503 02:14:08.490388 10752 tablet_copy_service.cc:140] P 7cd9a8da65604af78132e9045dac5180: Received BeginTabletCopySession request for tablet 375647a877014ca48ca32ff220098cd1 from peer 4c8a3af9b5184a51a3c0fe3221c01c75 ({username='slave'} at 127.9.151.130:57223)
I20240503 02:14:08.490586 10752 tablet_copy_service.cc:161] P 7cd9a8da65604af78132e9045dac5180: Beginning new tablet copy session on tablet 375647a877014ca48ca32ff220098cd1 from peer 4c8a3af9b5184a51a3c0fe3221c01c75 at {username='slave'} at 127.9.151.130:57223: session id = 4c8a3af9b5184a51a3c0fe3221c01c75-375647a877014ca48ca32ff220098cd1
W20240503 02:14:08.490829 10752 tablet_copy_service.cc:478] P 7cd9a8da65604af78132e9045dac5180: Error handling TabletCopyService RPC request from {username='slave'} at 127.9.151.130:57223: Error beginning tablet copy session for tablet 375647a877014ca48ca32ff220098cd1: Illegal state: The tablet is not in a running state: BOOTSTRAPPING
I20240503 02:14:08.496011 10452 ts_tablet_manager.cc:902] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: Initiating tablet copy from peer 7cd9a8da65604af78132e9045dac5180 (127.9.151.129:33077)
I20240503 02:14:08.497427 10452 tablet_copy_client.cc:308] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: tablet copy: Beginning tablet copy session from remote peer at address 127.9.151.129:33077
I20240503 02:14:08.498246 10752 tablet_copy_service.cc:140] P 7cd9a8da65604af78132e9045dac5180: Received BeginTabletCopySession request for tablet 375647a877014ca48ca32ff220098cd1 from peer 4c8a3af9b5184a51a3c0fe3221c01c75 ({username='slave'} at 127.9.151.130:57223)
I20240503 02:14:08.498454 10752 tablet_copy_service.cc:161] P 7cd9a8da65604af78132e9045dac5180: Beginning new tablet copy session on tablet 375647a877014ca48ca32ff220098cd1 from peer 4c8a3af9b5184a51a3c0fe3221c01c75 at {username='slave'} at 127.9.151.130:57223: session id = 4c8a3af9b5184a51a3c0fe3221c01c75-375647a877014ca48ca32ff220098cd1
W20240503 02:14:08.498658 10752 tablet_copy_service.cc:478] P 7cd9a8da65604af78132e9045dac5180: Error handling TabletCopyService RPC request from {username='slave'} at 127.9.151.130:57223: Error beginning tablet copy session for tablet 375647a877014ca48ca32ff220098cd1: Illegal state: The tablet is not in a running state: BOOTSTRAPPING
I20240503 02:14:08.504597 10452 ts_tablet_manager.cc:902] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: Initiating tablet copy from peer 7cd9a8da65604af78132e9045dac5180 (127.9.151.129:33077)
I20240503 02:14:08.507153 10452 tablet_copy_client.cc:308] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: tablet copy: Beginning tablet copy session from remote peer at address 127.9.151.129:33077
I20240503 02:14:08.507990 10752 tablet_copy_service.cc:140] P 7cd9a8da65604af78132e9045dac5180: Received BeginTabletCopySession request for tablet 375647a877014ca48ca32ff220098cd1 from peer 4c8a3af9b5184a51a3c0fe3221c01c75 ({username='slave'} at 127.9.151.130:57223)
I20240503 02:14:08.508175 10752 tablet_copy_service.cc:161] P 7cd9a8da65604af78132e9045dac5180: Beginning new tablet copy session on tablet 375647a877014ca48ca32ff220098cd1 from peer 4c8a3af9b5184a51a3c0fe3221c01c75 at {username='slave'} at 127.9.151.130:57223: session id = 4c8a3af9b5184a51a3c0fe3221c01c75-375647a877014ca48ca32ff220098cd1
W20240503 02:14:08.508371 10752 tablet_copy_service.cc:478] P 7cd9a8da65604af78132e9045dac5180: Error handling TabletCopyService RPC request from {username='slave'} at 127.9.151.130:57223: Error beginning tablet copy session for tablet 375647a877014ca48ca32ff220098cd1: Illegal state: The tablet is not in a running state: BOOTSTRAPPING
I20240503 02:14:08.545681 10452 ts_tablet_manager.cc:902] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: Initiating tablet copy from peer 7cd9a8da65604af78132e9045dac5180 (127.9.151.129:33077)
I20240503 02:14:08.547909 10452 tablet_copy_client.cc:308] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: tablet copy: Beginning tablet copy session from remote peer at address 127.9.151.129:33077
I20240503 02:14:08.548614 10752 tablet_copy_service.cc:140] P 7cd9a8da65604af78132e9045dac5180: Received BeginTabletCopySession request for tablet 375647a877014ca48ca32ff220098cd1 from peer 4c8a3af9b5184a51a3c0fe3221c01c75 ({username='slave'} at 127.9.151.130:57223)
I20240503 02:14:08.548779 10752 tablet_copy_service.cc:161] P 7cd9a8da65604af78132e9045dac5180: Beginning new tablet copy session on tablet 375647a877014ca48ca32ff220098cd1 from peer 4c8a3af9b5184a51a3c0fe3221c01c75 at {username='slave'} at 127.9.151.130:57223: session id = 4c8a3af9b5184a51a3c0fe3221c01c75-375647a877014ca48ca32ff220098cd1
W20240503 02:14:08.548967 10752 tablet_copy_service.cc:478] P 7cd9a8da65604af78132e9045dac5180: Error handling TabletCopyService RPC request from {username='slave'} at 127.9.151.130:57223: Error beginning tablet copy session for tablet 375647a877014ca48ca32ff220098cd1: Illegal state: The tablet is not in a running state: BOOTSTRAPPING
I20240503 02:14:08.563683 10452 ts_tablet_manager.cc:902] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: Initiating tablet copy from peer 7cd9a8da65604af78132e9045dac5180 (127.9.151.129:33077)
I20240503 02:14:08.564973 10452 tablet_copy_client.cc:308] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: tablet copy: Beginning tablet copy session from remote peer at address 127.9.151.129:33077
I20240503 02:14:08.565743 10752 tablet_copy_service.cc:140] P 7cd9a8da65604af78132e9045dac5180: Received BeginTabletCopySession request for tablet 375647a877014ca48ca32ff220098cd1 from peer 4c8a3af9b5184a51a3c0fe3221c01c75 ({username='slave'} at 127.9.151.130:57223)
I20240503 02:14:08.565897 10752 tablet_copy_service.cc:161] P 7cd9a8da65604af78132e9045dac5180: Beginning new tablet copy session on tablet 375647a877014ca48ca32ff220098cd1 from peer 4c8a3af9b5184a51a3c0fe3221c01c75 at {username='slave'} at 127.9.151.130:57223: session id = 4c8a3af9b5184a51a3c0fe3221c01c75-375647a877014ca48ca32ff220098cd1
W20240503 02:14:08.566108 10752 tablet_copy_service.cc:478] P 7cd9a8da65604af78132e9045dac5180: Error handling TabletCopyService RPC request from {username='slave'} at 127.9.151.130:57223: Error beginning tablet copy session for tablet 375647a877014ca48ca32ff220098cd1: Illegal state: The tablet is not in a running state: BOOTSTRAPPING
I20240503 02:14:08.569542 10452 ts_tablet_manager.cc:902] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: Initiating tablet copy from peer 7cd9a8da65604af78132e9045dac5180 (127.9.151.129:33077)
I20240503 02:14:08.570772 10452 tablet_copy_client.cc:308] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: tablet copy: Beginning tablet copy session from remote peer at address 127.9.151.129:33077
I20240503 02:14:08.571524 10752 tablet_copy_service.cc:140] P 7cd9a8da65604af78132e9045dac5180: Received BeginTabletCopySession request for tablet 375647a877014ca48ca32ff220098cd1 from peer 4c8a3af9b5184a51a3c0fe3221c01c75 ({username='slave'} at 127.9.151.130:57223)
I20240503 02:14:08.571810 10752 tablet_copy_service.cc:161] P 7cd9a8da65604af78132e9045dac5180: Beginning new tablet copy session on tablet 375647a877014ca48ca32ff220098cd1 from peer 4c8a3af9b5184a51a3c0fe3221c01c75 at {username='slave'} at 127.9.151.130:57223: session id = 4c8a3af9b5184a51a3c0fe3221c01c75-375647a877014ca48ca32ff220098cd1
W20240503 02:14:08.572063 10752 tablet_copy_service.cc:478] P 7cd9a8da65604af78132e9045dac5180: Error handling TabletCopyService RPC request from {username='slave'} at 127.9.151.130:57223: Error beginning tablet copy session for tablet 375647a877014ca48ca32ff220098cd1: Illegal state: The tablet is not in a running state: BOOTSTRAPPING
I20240503 02:14:08.585058 10452 ts_tablet_manager.cc:902] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: Initiating tablet copy from peer 7cd9a8da65604af78132e9045dac5180 (127.9.151.129:33077)
I20240503 02:14:08.590742 10452 tablet_copy_client.cc:308] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: tablet copy: Beginning tablet copy session from remote peer at address 127.9.151.129:33077
I20240503 02:14:08.591778 10752 tablet_copy_service.cc:140] P 7cd9a8da65604af78132e9045dac5180: Received BeginTabletCopySession request for tablet 375647a877014ca48ca32ff220098cd1 from peer 4c8a3af9b5184a51a3c0fe3221c01c75 ({username='slave'} at 127.9.151.130:57223)
I20240503 02:14:08.592058 10752 tablet_copy_service.cc:161] P 7cd9a8da65604af78132e9045dac5180: Beginning new tablet copy session on tablet 375647a877014ca48ca32ff220098cd1 from peer 4c8a3af9b5184a51a3c0fe3221c01c75 at {username='slave'} at 127.9.151.130:57223: session id = 4c8a3af9b5184a51a3c0fe3221c01c75-375647a877014ca48ca32ff220098cd1
W20240503 02:14:08.592314 10752 tablet_copy_service.cc:478] P 7cd9a8da65604af78132e9045dac5180: Error handling TabletCopyService RPC request from {username='slave'} at 127.9.151.130:57223: Error beginning tablet copy session for tablet 375647a877014ca48ca32ff220098cd1: Illegal state: The tablet is not in a running state: BOOTSTRAPPING
I20240503 02:14:08.600672 10649 tablet_bootstrap.cc:492] T 375647a877014ca48ca32ff220098cd1 P 7cd9a8da65604af78132e9045dac5180: Bootstrap replayed 1/1 log segments. Stats: ops{read=212 overwritten=0 applied=212 ignored=0} inserts{seen=10550 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240503 02:14:08.601276 10649 tablet_bootstrap.cc:492] T 375647a877014ca48ca32ff220098cd1 P 7cd9a8da65604af78132e9045dac5180: Bootstrap complete.
I20240503 02:14:08.602537 10649 ts_tablet_manager.cc:1367] T 375647a877014ca48ca32ff220098cd1 P 7cd9a8da65604af78132e9045dac5180: Time spent bootstrapping tablet: real 0.868s user 0.779s sys 0.072s
I20240503 02:14:08.607636 10649 raft_consensus.cc:348] T 375647a877014ca48ca32ff220098cd1 P 7cd9a8da65604af78132e9045dac5180 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7cd9a8da65604af78132e9045dac5180" member_type: VOTER last_known_addr { host: "127.9.151.129" port: 33077 } }
I20240503 02:14:08.608664 10649 raft_consensus.cc:729] T 375647a877014ca48ca32ff220098cd1 P 7cd9a8da65604af78132e9045dac5180 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7cd9a8da65604af78132e9045dac5180, State: Initialized, Role: FOLLOWER
I20240503 02:14:08.609403 10649 consensus_queue.cc:260] T 375647a877014ca48ca32ff220098cd1 P 7cd9a8da65604af78132e9045dac5180 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 212, Last appended: 1.212, Last appended by leader: 212, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7cd9a8da65604af78132e9045dac5180" member_type: VOTER last_known_addr { host: "127.9.151.129" port: 33077 } }
I20240503 02:14:08.610004 10649 raft_consensus.cc:388] T 375647a877014ca48ca32ff220098cd1 P 7cd9a8da65604af78132e9045dac5180 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20240503 02:14:08.610204 10649 raft_consensus.cc:482] T 375647a877014ca48ca32ff220098cd1 P 7cd9a8da65604af78132e9045dac5180 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20240503 02:14:08.610563 10649 raft_consensus.cc:3037] T 375647a877014ca48ca32ff220098cd1 P 7cd9a8da65604af78132e9045dac5180 [term 1 FOLLOWER]: Advancing to term 2
I20240503 02:14:08.611055 10452 ts_tablet_manager.cc:902] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: Initiating tablet copy from peer 7cd9a8da65604af78132e9045dac5180 (127.9.151.129:33077)
I20240503 02:14:08.612488 10452 tablet_copy_client.cc:308] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: tablet copy: Beginning tablet copy session from remote peer at address 127.9.151.129:33077
I20240503 02:14:08.613416 10752 tablet_copy_service.cc:140] P 7cd9a8da65604af78132e9045dac5180: Received BeginTabletCopySession request for tablet 375647a877014ca48ca32ff220098cd1 from peer 4c8a3af9b5184a51a3c0fe3221c01c75 ({username='slave'} at 127.9.151.130:57223)
I20240503 02:14:08.613641 10752 tablet_copy_service.cc:161] P 7cd9a8da65604af78132e9045dac5180: Beginning new tablet copy session on tablet 375647a877014ca48ca32ff220098cd1 from peer 4c8a3af9b5184a51a3c0fe3221c01c75 at {username='slave'} at 127.9.151.130:57223: session id = 4c8a3af9b5184a51a3c0fe3221c01c75-375647a877014ca48ca32ff220098cd1
W20240503 02:14:08.613860 10752 tablet_copy_service.cc:478] P 7cd9a8da65604af78132e9045dac5180: Error handling TabletCopyService RPC request from {username='slave'} at 127.9.151.130:57223: Error beginning tablet copy session for tablet 375647a877014ca48ca32ff220098cd1: Illegal state: The tablet is not in a running state: BOOTSTRAPPING
I20240503 02:14:08.616410 10649 raft_consensus.cc:504] T 375647a877014ca48ca32ff220098cd1 P 7cd9a8da65604af78132e9045dac5180 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7cd9a8da65604af78132e9045dac5180" member_type: VOTER last_known_addr { host: "127.9.151.129" port: 33077 } }
I20240503 02:14:08.617084 10649 leader_election.cc:302] T 375647a877014ca48ca32ff220098cd1 P 7cd9a8da65604af78132e9045dac5180 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7cd9a8da65604af78132e9045dac5180; no voters:
I20240503 02:14:08.617470 10452 ts_tablet_manager.cc:902] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: Initiating tablet copy from peer 7cd9a8da65604af78132e9045dac5180 (127.9.151.129:33077)
I20240503 02:14:08.617740 10649 leader_election.cc:288] T 375647a877014ca48ca32ff220098cd1 P 7cd9a8da65604af78132e9045dac5180 [CANDIDATE]: Term 2 election: Requested vote from peers
I20240503 02:14:08.617950 10999 raft_consensus.cc:2781] T 375647a877014ca48ca32ff220098cd1 P 7cd9a8da65604af78132e9045dac5180 [term 2 FOLLOWER]: Leader election won for term 2
I20240503 02:14:08.618520 10999 raft_consensus.cc:686] T 375647a877014ca48ca32ff220098cd1 P 7cd9a8da65604af78132e9045dac5180 [term 2 LEADER]: Becoming Leader. State: Replica: 7cd9a8da65604af78132e9045dac5180, State: Running, Role: LEADER
I20240503 02:14:08.619267 10999 consensus_queue.cc:237] T 375647a877014ca48ca32ff220098cd1 P 7cd9a8da65604af78132e9045dac5180 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 212, Committed index: 212, Last appended: 1.212, Last appended by leader: 212, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7cd9a8da65604af78132e9045dac5180" member_type: VOTER last_known_addr { host: "127.9.151.129" port: 33077 } }
I20240503 02:14:08.620559 10793 heartbeater.cc:503] Master 127.9.151.190:36085 was elected leader, sending a full tablet report...
I20240503 02:14:08.624125 10452 tablet_copy_client.cc:308] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: tablet copy: Beginning tablet copy session from remote peer at address 127.9.151.129:33077
I20240503 02:14:08.624421 10649 ts_tablet_manager.cc:1398] T 375647a877014ca48ca32ff220098cd1 P 7cd9a8da65604af78132e9045dac5180: Time spent starting tablet: real 0.022s user 0.011s sys 0.014s
I20240503 02:14:08.625223 10752 tablet_copy_service.cc:140] P 7cd9a8da65604af78132e9045dac5180: Received BeginTabletCopySession request for tablet 375647a877014ca48ca32ff220098cd1 from peer 4c8a3af9b5184a51a3c0fe3221c01c75 ({username='slave'} at 127.9.151.130:57223)
I20240503 02:14:08.625453 10752 tablet_copy_service.cc:161] P 7cd9a8da65604af78132e9045dac5180: Beginning new tablet copy session on tablet 375647a877014ca48ca32ff220098cd1 from peer 4c8a3af9b5184a51a3c0fe3221c01c75 at {username='slave'} at 127.9.151.130:57223: session id = 4c8a3af9b5184a51a3c0fe3221c01c75-375647a877014ca48ca32ff220098cd1
I20240503 02:14:08.628944 10221 catalog_manager.cc:5484] T 375647a877014ca48ca32ff220098cd1 P 7cd9a8da65604af78132e9045dac5180 reported cstate change: term changed from 1 to 2. New cstate: current_term: 2 leader_uuid: "7cd9a8da65604af78132e9045dac5180" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7cd9a8da65604af78132e9045dac5180" member_type: VOTER last_known_addr { host: "127.9.151.129" port: 33077 } health_report { overall_health: HEALTHY } } }
I20240503 02:14:08.629113 10752 tablet_copy_source_session.cc:205] T 375647a877014ca48ca32ff220098cd1 P 7cd9a8da65604af78132e9045dac5180: Tablet Copy: opened 0 blocks and 1 log segments
I20240503 02:14:08.635821 10452 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 375647a877014ca48ca32ff220098cd1. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:14:08.648867 10452 tablet_copy_client.cc:783] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: tablet copy: Starting download of 0 data blocks...
I20240503 02:14:08.649250 10452 tablet_copy_client.cc:647] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: tablet copy: Starting download of 1 WAL segments...
I20240503 02:14:08.654980 10452 tablet_copy_client.cc:523] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240503 02:14:08.659952 10452 tablet_bootstrap.cc:492] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: Bootstrap starting.
I20240503 02:14:08.729543 10452 log.cc:824] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: Log is configured to *not* fsync() on all Append() calls
I20240503 02:14:09.353204 10452 tablet_bootstrap.cc:492] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: Bootstrap replayed 1/1 log segments. Stats: ops{read=213 overwritten=0 applied=212 ignored=0} inserts{seen=10550 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 1 replicates
I20240503 02:14:09.353963 10452 tablet_bootstrap.cc:492] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: Bootstrap complete.
I20240503 02:14:09.354538 10452 ts_tablet_manager.cc:1367] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: Time spent bootstrapping tablet: real 0.695s user 0.611s sys 0.080s
I20240503 02:14:09.359220 10452 raft_consensus.cc:348] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75 [term 2 NON_PARTICIPANT]: Replica starting. Triggering 1 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7cd9a8da65604af78132e9045dac5180" member_type: VOTER last_known_addr { host: "127.9.151.129" port: 33077 } }
I20240503 02:14:09.359846 10452 raft_consensus.cc:729] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75 [term 2 NON_PARTICIPANT]: Becoming Follower/Learner. State: Replica: 4c8a3af9b5184a51a3c0fe3221c01c75, State: Initialized, Role: NON_PARTICIPANT
I20240503 02:14:09.360340 10452 consensus_queue.cc:260] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 212, Last appended: 2.213, Last appended by leader: 213, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7cd9a8da65604af78132e9045dac5180" member_type: VOTER last_known_addr { host: "127.9.151.129" port: 33077 } }
I20240503 02:14:09.361828 10452 ts_tablet_manager.cc:1398] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: Time spent starting tablet: real 0.007s user 0.006s sys 0.005s
I20240503 02:14:09.362973 10752 tablet_copy_service.cc:342] P 7cd9a8da65604af78132e9045dac5180: Request end of tablet copy session 4c8a3af9b5184a51a3c0fe3221c01c75-375647a877014ca48ca32ff220098cd1 received from {username='slave'} at 127.9.151.130:57223
I20240503 02:14:09.363131 10752 tablet_copy_service.cc:433] P 7cd9a8da65604af78132e9045dac5180: ending tablet copy session 4c8a3af9b5184a51a3c0fe3221c01c75-375647a877014ca48ca32ff220098cd1 on tablet 375647a877014ca48ca32ff220098cd1 with peer 4c8a3af9b5184a51a3c0fe3221c01c75
W20240503 02:14:09.365527 10452 ts_tablet_manager.cc:701] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: Tablet Copy: Invalid argument: Leader has replica of tablet 375647a877014ca48ca32ff220098cd1 with term 0, which is lower than last-logged term 2 on local replica. Rejecting tablet copy request
W20240503 02:14:09.378595 10452 ts_tablet_manager.cc:701] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: Tablet Copy: Invalid argument: Leader has replica of tablet 375647a877014ca48ca32ff220098cd1 with term 0, which is lower than last-logged term 2 on local replica. Rejecting tablet copy request
W20240503 02:14:09.381573 10452 ts_tablet_manager.cc:701] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: Tablet Copy: Invalid argument: Leader has replica of tablet 375647a877014ca48ca32ff220098cd1 with term 0, which is lower than last-logged term 2 on local replica. Rejecting tablet copy request
I20240503 02:14:09.384927 10372 tablet_service.cc:1514] Processing DeleteTablet for tablet 375647a877014ca48ca32ff220098cd1 with delete_type TABLET_DATA_DELETED from {username='slave'} at 127.0.0.1:40838
I20240503 02:14:09.385735 11015 tablet_replica.cc:330] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: stopping tablet replica
I20240503 02:14:09.386162 11015 raft_consensus.cc:2229] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75 [term 2 NON_PARTICIPANT]: Raft consensus shutting down.
I20240503 02:14:09.386461 11015 pending_rounds.cc:62] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: Trying to abort 1 pending ops.
I20240503 02:14:09.386579 11015 pending_rounds.cc:68] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: Aborting op as it isn't in flight: id { term: 2 index: 213 } timestamp: 7023421229546078208 op_type: NO_OP noop_request { }
I20240503 02:14:09.386808 11015 raft_consensus.cc:2866] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75 [term 2 NON_PARTICIPANT]: NO_OP replication failed: Aborted: Op aborted
I20240503 02:14:09.386936 11015 raft_consensus.cc:2258] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75 [term 2 NON_PARTICIPANT]: Raft consensus is shut down!
I20240503 02:14:09.388221 11015 ts_tablet_manager.cc:1875] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:14:09.392874 11015 ts_tablet_manager.cc:1888] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 2.213
I20240503 02:14:09.393069 11015 log.cc:1196] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: Deleting WAL directory at /tmp/dist-test-tasknrxqHk/test-tmp/tablet_copy_client_session-itest.0.TabletCopyClientSessionITest.TestStartTabletCopyWhileSourceBootstrapping.1714702442467385-9822-0/minicluster-data/ts-1/wal/wals/375647a877014ca48ca32ff220098cd1
I20240503 02:14:09.393605 11015 ts_tablet_manager.cc:1909] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: Deleting consensus metadata
I20240503 02:14:09.395677 9822 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasknrxqHk/build/asan/bin/kudu pid 10310
I20240503 02:14:09.397058 10432 generic_service.cc:196] Checking for leaks (request via RPC)
I20240503 02:14:09.506141 9822 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknrxqHk/build/asan/bin/kudu with pid 10310
I20240503 02:14:09.529603 9822 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknrxqHk/build/asan/bin/kudu
/tmp/dist-test-tasknrxqHk/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknrxqHk/test-tmp/tablet_copy_client_session-itest.0.TabletCopyClientSessionITest.TestStartTabletCopyWhileSourceBootstrapping.1714702442467385-9822-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasknrxqHk/test-tmp/tablet_copy_client_session-itest.0.TabletCopyClientSessionITest.TestStartTabletCopyWhileSourceBootstrapping.1714702442467385-9822-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknrxqHk/test-tmp/tablet_copy_client_session-itest.0.TabletCopyClientSessionITest.TestStartTabletCopyWhileSourceBootstrapping.1714702442467385-9822-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasknrxqHk/test-tmp/tablet_copy_client_session-itest.0.TabletCopyClientSessionITest.TestStartTabletCopyWhileSourceBootstrapping.1714702442467385-9822-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.151.130:35753
--local_ip_for_outbound_sockets=127.9.151.130
--tserver_master_addrs=127.9.151.190:36085
--webserver_port=42537
--webserver_interface=127.9.151.130
--builtin_ntp_servers=127.9.151.148:38865
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin with env {}
W20240503 02:14:09.886750 11017 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240503 02:14:09.887107 11017 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240503 02:14:09.887274 11017 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240503 02:14:09.897532 11017 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240503 02:14:09.897789 11017 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.151.130
I20240503 02:14:09.908378 11017 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.151.148:38865
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknrxqHk/test-tmp/tablet_copy_client_session-itest.0.TabletCopyClientSessionITest.TestStartTabletCopyWhileSourceBootstrapping.1714702442467385-9822-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasknrxqHk/test-tmp/tablet_copy_client_session-itest.0.TabletCopyClientSessionITest.TestStartTabletCopyWhileSourceBootstrapping.1714702442467385-9822-0/minicluster-data/ts-1/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.151.130:35753
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknrxqHk/test-tmp/tablet_copy_client_session-itest.0.TabletCopyClientSessionITest.TestStartTabletCopyWhileSourceBootstrapping.1714702442467385-9822-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.151.130
--webserver_port=42537
--tserver_master_addrs=127.9.151.190:36085
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.151.130
--log_dir=/tmp/dist-test-tasknrxqHk/test-tmp/tablet_copy_client_session-itest.0.TabletCopyClientSessionITest.TestStartTabletCopyWhileSourceBootstrapping.1714702442467385-9822-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c39e72a98e124e5a12080bac42105b996594d806
build type FASTDEBUG
built by None at 03 May 2024 01:43:22 UTC on efd1337a2aa3
build id 99
ASAN enabled
I20240503 02:14:09.909933 11017 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240503 02:14:09.911899 11017 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240503 02:14:09.926079 11022 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240503 02:14:09.926633 11023 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240503 02:14:09.927193 11025 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240503 02:14:09.927969 11017 server_base.cc:1034] running on GCE node
I20240503 02:14:09.928666 11017 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240503 02:14:09.929700 11017 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240503 02:14:09.930955 11017 hybrid_clock.cc:648] HybridClock initialized: now 1714702449930899 us; error 40 us; skew 500 ppm
I20240503 02:14:09.931305 11017 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20240503 02:14:09.938578 11017 webserver.cc:460] Webserver started at http://127.9.151.130:42537/ using document root <none> and password file <none>
I20240503 02:14:09.939689 11017 fs_manager.cc:359] Metadata directory not provided
I20240503 02:14:09.939857 11017 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240503 02:14:09.946184 11017 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.006s sys 0.000s
I20240503 02:14:09.952124 11031 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240503 02:14:09.953984 11017 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.000s
I20240503 02:14:09.954349 11017 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknrxqHk/test-tmp/tablet_copy_client_session-itest.0.TabletCopyClientSessionITest.TestStartTabletCopyWhileSourceBootstrapping.1714702442467385-9822-0/minicluster-data/ts-1/data,/tmp/dist-test-tasknrxqHk/test-tmp/tablet_copy_client_session-itest.0.TabletCopyClientSessionITest.TestStartTabletCopyWhileSourceBootstrapping.1714702442467385-9822-0/minicluster-data/ts-1/wal
uuid: "4c8a3af9b5184a51a3c0fe3221c01c75"
format_stamp: "Formatted at 2024-05-03 02:14:04 on dist-test-slave-twlh"
I20240503 02:14:09.955385 11017 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknrxqHk/test-tmp/tablet_copy_client_session-itest.0.TabletCopyClientSessionITest.TestStartTabletCopyWhileSourceBootstrapping.1714702442467385-9822-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasknrxqHk/test-tmp/tablet_copy_client_session-itest.0.TabletCopyClientSessionITest.TestStartTabletCopyWhileSourceBootstrapping.1714702442467385-9822-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasknrxqHk/test-tmp/tablet_copy_client_session-itest.0.TabletCopyClientSessionITest.TestStartTabletCopyWhileSourceBootstrapping.1714702442467385-9822-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240503 02:14:09.979763 11017 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240503 02:14:09.980773 11017 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240503 02:14:09.981077 11017 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240503 02:14:09.982215 11017 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240503 02:14:09.983956 11017 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240503 02:14:09.984053 11017 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240503 02:14:09.984143 11017 ts_tablet_manager.cc:590] Registered 0 tablets
I20240503 02:14:09.984202 11017 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240503 02:14:10.020562 11017 rpc_server.cc:307] RPC server started. Bound to: 127.9.151.130:35753
I20240503 02:14:10.020612 11143 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.151.130:35753 every 8 connection(s)
I20240503 02:14:10.022233 11017 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknrxqHk/test-tmp/tablet_copy_client_session-itest.0.TabletCopyClientSessionITest.TestStartTabletCopyWhileSourceBootstrapping.1714702442467385-9822-0/minicluster-data/ts-1/data/info.pb
I20240503 02:14:10.032168 9822 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknrxqHk/build/asan/bin/kudu as pid 11017
I20240503 02:14:10.032503 9822 tablet_copy_client_session-itest.cc:139] Starting loop 1
I20240503 02:14:10.032599 9822 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasknrxqHk/build/asan/bin/kudu pid 10448
I20240503 02:14:10.039351 11144 heartbeater.cc:347] Connected to a master server at 127.9.151.190:36085
I20240503 02:14:10.039790 11144 heartbeater.cc:464] Registering TS with master...
I20240503 02:14:10.040416 10784 generic_service.cc:196] Checking for leaks (request via RPC)
I20240503 02:14:10.041098 11144 heartbeater.cc:511] Master 127.9.151.190:36085 requested a full tablet report, sending...
I20240503 02:14:10.043728 10221 ts_manager.cc:194] Re-registered known tserver with Master: 4c8a3af9b5184a51a3c0fe3221c01c75 (127.9.151.130:35753)
I20240503 02:14:10.045132 10221 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.9.151.130:50823
I20240503 02:14:10.147692 9822 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknrxqHk/build/asan/bin/kudu with pid 10448
I20240503 02:14:10.170262 9822 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknrxqHk/build/asan/bin/kudu
/tmp/dist-test-tasknrxqHk/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknrxqHk/test-tmp/tablet_copy_client_session-itest.0.TabletCopyClientSessionITest.TestStartTabletCopyWhileSourceBootstrapping.1714702442467385-9822-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasknrxqHk/test-tmp/tablet_copy_client_session-itest.0.TabletCopyClientSessionITest.TestStartTabletCopyWhileSourceBootstrapping.1714702442467385-9822-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknrxqHk/test-tmp/tablet_copy_client_session-itest.0.TabletCopyClientSessionITest.TestStartTabletCopyWhileSourceBootstrapping.1714702442467385-9822-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasknrxqHk/test-tmp/tablet_copy_client_session-itest.0.TabletCopyClientSessionITest.TestStartTabletCopyWhileSourceBootstrapping.1714702442467385-9822-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.151.129:33077
--local_ip_for_outbound_sockets=127.9.151.129
--tserver_master_addrs=127.9.151.190:36085
--webserver_port=43885
--webserver_interface=127.9.151.129
--builtin_ntp_servers=127.9.151.148:38865
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin with env {}
I20240503 02:14:10.196620 11160 ts_tablet_manager.cc:902] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: Initiating tablet copy from peer 7cd9a8da65604af78132e9045dac5180 (127.9.151.129:33077)
I20240503 02:14:10.208127 11160 tablet_copy_client.cc:308] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: tablet copy: Beginning tablet copy session from remote peer at address 127.9.151.129:33077
I20240503 02:14:10.226778 11160 ts_tablet_manager.cc:902] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: Initiating tablet copy from peer 7cd9a8da65604af78132e9045dac5180 (127.9.151.129:33077)
I20240503 02:14:10.228276 11160 tablet_copy_client.cc:308] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: tablet copy: Beginning tablet copy session from remote peer at address 127.9.151.129:33077
I20240503 02:14:10.243710 11160 ts_tablet_manager.cc:902] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: Initiating tablet copy from peer 7cd9a8da65604af78132e9045dac5180 (127.9.151.129:33077)
I20240503 02:14:10.244930 11160 tablet_copy_client.cc:308] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: tablet copy: Beginning tablet copy session from remote peer at address 127.9.151.129:33077
I20240503 02:14:10.258540 11160 ts_tablet_manager.cc:902] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: Initiating tablet copy from peer 7cd9a8da65604af78132e9045dac5180 (127.9.151.129:33077)
I20240503 02:14:10.259912 11160 tablet_copy_client.cc:308] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: tablet copy: Beginning tablet copy session from remote peer at address 127.9.151.129:33077
I20240503 02:14:10.268668 11160 ts_tablet_manager.cc:902] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: Initiating tablet copy from peer 7cd9a8da65604af78132e9045dac5180 (127.9.151.129:33077)
I20240503 02:14:10.270136 11160 tablet_copy_client.cc:308] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: tablet copy: Beginning tablet copy session from remote peer at address 127.9.151.129:33077
I20240503 02:14:10.280788 11160 ts_tablet_manager.cc:902] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: Initiating tablet copy from peer 7cd9a8da65604af78132e9045dac5180 (127.9.151.129:33077)
I20240503 02:14:10.282091 11160 tablet_copy_client.cc:308] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: tablet copy: Beginning tablet copy session from remote peer at address 127.9.151.129:33077
I20240503 02:14:10.290877 11160 ts_tablet_manager.cc:902] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: Initiating tablet copy from peer 7cd9a8da65604af78132e9045dac5180 (127.9.151.129:33077)
I20240503 02:14:10.292330 11160 tablet_copy_client.cc:308] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: tablet copy: Beginning tablet copy session from remote peer at address 127.9.151.129:33077
I20240503 02:14:10.317519 11160 ts_tablet_manager.cc:902] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: Initiating tablet copy from peer 7cd9a8da65604af78132e9045dac5180 (127.9.151.129:33077)
I20240503 02:14:10.318918 11160 tablet_copy_client.cc:308] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: tablet copy: Beginning tablet copy session from remote peer at address 127.9.151.129:33077
I20240503 02:14:10.322033 11160 ts_tablet_manager.cc:902] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: Initiating tablet copy from peer 7cd9a8da65604af78132e9045dac5180 (127.9.151.129:33077)
I20240503 02:14:10.323226 11160 tablet_copy_client.cc:308] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: tablet copy: Beginning tablet copy session from remote peer at address 127.9.151.129:33077
I20240503 02:14:10.326587 11160 ts_tablet_manager.cc:902] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: Initiating tablet copy from peer 7cd9a8da65604af78132e9045dac5180 (127.9.151.129:33077)
I20240503 02:14:10.332513 11160 tablet_copy_client.cc:308] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: tablet copy: Beginning tablet copy session from remote peer at address 127.9.151.129:33077
I20240503 02:14:10.335784 11160 ts_tablet_manager.cc:902] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: Initiating tablet copy from peer 7cd9a8da65604af78132e9045dac5180 (127.9.151.129:33077)
I20240503 02:14:10.337127 11160 tablet_copy_client.cc:308] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: tablet copy: Beginning tablet copy session from remote peer at address 127.9.151.129:33077
I20240503 02:14:10.353013 11160 ts_tablet_manager.cc:902] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: Initiating tablet copy from peer 7cd9a8da65604af78132e9045dac5180 (127.9.151.129:33077)
I20240503 02:14:10.354583 11160 tablet_copy_client.cc:308] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: tablet copy: Beginning tablet copy session from remote peer at address 127.9.151.129:33077
I20240503 02:14:10.362555 11160 ts_tablet_manager.cc:902] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: Initiating tablet copy from peer 7cd9a8da65604af78132e9045dac5180 (127.9.151.129:33077)
I20240503 02:14:10.364157 11160 tablet_copy_client.cc:308] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: tablet copy: Beginning tablet copy session from remote peer at address 127.9.151.129:33077
I20240503 02:14:10.367825 11160 ts_tablet_manager.cc:902] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: Initiating tablet copy from peer 7cd9a8da65604af78132e9045dac5180 (127.9.151.129:33077)
I20240503 02:14:10.375134 11160 tablet_copy_client.cc:308] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: tablet copy: Beginning tablet copy session from remote peer at address 127.9.151.129:33077
I20240503 02:14:10.386530 11160 ts_tablet_manager.cc:902] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: Initiating tablet copy from peer 7cd9a8da65604af78132e9045dac5180 (127.9.151.129:33077)
I20240503 02:14:10.387892 11160 tablet_copy_client.cc:308] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: tablet copy: Beginning tablet copy session from remote peer at address 127.9.151.129:33077
I20240503 02:14:10.391906 11160 ts_tablet_manager.cc:902] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: Initiating tablet copy from peer 7cd9a8da65604af78132e9045dac5180 (127.9.151.129:33077)
I20240503 02:14:10.393036 11160 tablet_copy_client.cc:308] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: tablet copy: Beginning tablet copy session from remote peer at address 127.9.151.129:33077
I20240503 02:14:10.406786 11160 ts_tablet_manager.cc:902] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: Initiating tablet copy from peer 7cd9a8da65604af78132e9045dac5180 (127.9.151.129:33077)
I20240503 02:14:10.408102 11160 tablet_copy_client.cc:308] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: tablet copy: Beginning tablet copy session from remote peer at address 127.9.151.129:33077
I20240503 02:14:10.421451 11160 ts_tablet_manager.cc:902] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: Initiating tablet copy from peer 7cd9a8da65604af78132e9045dac5180 (127.9.151.129:33077)
I20240503 02:14:10.422816 11160 tablet_copy_client.cc:308] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: tablet copy: Beginning tablet copy session from remote peer at address 127.9.151.129:33077
I20240503 02:14:10.433534 11160 ts_tablet_manager.cc:902] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: Initiating tablet copy from peer 7cd9a8da65604af78132e9045dac5180 (127.9.151.129:33077)
I20240503 02:14:10.435045 11160 tablet_copy_client.cc:308] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: tablet copy: Beginning tablet copy session from remote peer at address 127.9.151.129:33077
I20240503 02:14:10.439525 11160 ts_tablet_manager.cc:902] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: Initiating tablet copy from peer 7cd9a8da65604af78132e9045dac5180 (127.9.151.129:33077)
I20240503 02:14:10.441080 11160 tablet_copy_client.cc:308] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: tablet copy: Beginning tablet copy session from remote peer at address 127.9.151.129:33077
I20240503 02:14:10.452289 11159 ts_tablet_manager.cc:902] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: Initiating tablet copy from peer 7cd9a8da65604af78132e9045dac5180 (127.9.151.129:33077)
I20240503 02:14:10.454147 11159 tablet_copy_client.cc:308] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: tablet copy: Beginning tablet copy session from remote peer at address 127.9.151.129:33077
I20240503 02:14:10.456796 11159 ts_tablet_manager.cc:902] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: Initiating tablet copy from peer 7cd9a8da65604af78132e9045dac5180 (127.9.151.129:33077)
I20240503 02:14:10.458019 11159 tablet_copy_client.cc:308] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: tablet copy: Beginning tablet copy session from remote peer at address 127.9.151.129:33077
I20240503 02:14:10.476859 11159 ts_tablet_manager.cc:902] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: Initiating tablet copy from peer 7cd9a8da65604af78132e9045dac5180 (127.9.151.129:33077)
I20240503 02:14:10.478158 11159 tablet_copy_client.cc:308] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: tablet copy: Beginning tablet copy session from remote peer at address 127.9.151.129:33077
I20240503 02:14:10.488708 11159 ts_tablet_manager.cc:902] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: Initiating tablet copy from peer 7cd9a8da65604af78132e9045dac5180 (127.9.151.129:33077)
I20240503 02:14:10.490581 11159 tablet_copy_client.cc:308] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: tablet copy: Beginning tablet copy session from remote peer at address 127.9.151.129:33077
I20240503 02:14:10.493875 11159 ts_tablet_manager.cc:902] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: Initiating tablet copy from peer 7cd9a8da65604af78132e9045dac5180 (127.9.151.129:33077)
I20240503 02:14:10.499430 11159 tablet_copy_client.cc:308] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: tablet copy: Beginning tablet copy session from remote peer at address 127.9.151.129:33077
I20240503 02:14:10.519821 11159 ts_tablet_manager.cc:902] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: Initiating tablet copy from peer 7cd9a8da65604af78132e9045dac5180 (127.9.151.129:33077)
I20240503 02:14:10.521404 11159 tablet_copy_client.cc:308] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: tablet copy: Beginning tablet copy session from remote peer at address 127.9.151.129:33077
I20240503 02:14:10.527665 11159 ts_tablet_manager.cc:902] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: Initiating tablet copy from peer 7cd9a8da65604af78132e9045dac5180 (127.9.151.129:33077)
I20240503 02:14:10.529006 11159 tablet_copy_client.cc:308] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: tablet copy: Beginning tablet copy session from remote peer at address 127.9.151.129:33077
I20240503 02:14:10.539400 11159 ts_tablet_manager.cc:902] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: Initiating tablet copy from peer 7cd9a8da65604af78132e9045dac5180 (127.9.151.129:33077)
I20240503 02:14:10.541115 11159 tablet_copy_client.cc:308] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: tablet copy: Beginning tablet copy session from remote peer at address 127.9.151.129:33077
I20240503 02:14:10.546883 11159 ts_tablet_manager.cc:902] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: Initiating tablet copy from peer 7cd9a8da65604af78132e9045dac5180 (127.9.151.129:33077)
I20240503 02:14:10.553119 11159 tablet_copy_client.cc:308] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: tablet copy: Beginning tablet copy session from remote peer at address 127.9.151.129:33077
I20240503 02:14:10.555737 11159 ts_tablet_manager.cc:902] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: Initiating tablet copy from peer 7cd9a8da65604af78132e9045dac5180 (127.9.151.129:33077)
I20240503 02:14:10.557098 11159 tablet_copy_client.cc:308] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: tablet copy: Beginning tablet copy session from remote peer at address 127.9.151.129:33077
I20240503 02:14:10.562307 11159 ts_tablet_manager.cc:902] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: Initiating tablet copy from peer 7cd9a8da65604af78132e9045dac5180 (127.9.151.129:33077)
I20240503 02:14:10.563566 11159 tablet_copy_client.cc:308] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: tablet copy: Beginning tablet copy session from remote peer at address 127.9.151.129:33077
I20240503 02:14:10.568498 11159 ts_tablet_manager.cc:902] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: Initiating tablet copy from peer 7cd9a8da65604af78132e9045dac5180 (127.9.151.129:33077)
I20240503 02:14:10.569765 11159 tablet_copy_client.cc:308] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: tablet copy: Beginning tablet copy session from remote peer at address 127.9.151.129:33077
I20240503 02:14:10.578133 11159 ts_tablet_manager.cc:902] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: Initiating tablet copy from peer 7cd9a8da65604af78132e9045dac5180 (127.9.151.129:33077)
I20240503 02:14:10.579397 11159 tablet_copy_client.cc:308] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: tablet copy: Beginning tablet copy session from remote peer at address 127.9.151.129:33077
W20240503 02:14:10.582226 11156 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240503 02:14:10.582571 11156 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240503 02:14:10.582728 11156 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20240503 02:14:10.587308 11159 ts_tablet_manager.cc:902] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: Initiating tablet copy from peer 7cd9a8da65604af78132e9045dac5180 (127.9.151.129:33077)
I20240503 02:14:10.588544 11159 tablet_copy_client.cc:308] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: tablet copy: Beginning tablet copy session from remote peer at address 127.9.151.129:33077
W20240503 02:14:10.592276 11156 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240503 02:14:10.592514 11156 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.151.129
I20240503 02:14:10.602596 11156 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.151.148:38865
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknrxqHk/test-tmp/tablet_copy_client_session-itest.0.TabletCopyClientSessionITest.TestStartTabletCopyWhileSourceBootstrapping.1714702442467385-9822-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasknrxqHk/test-tmp/tablet_copy_client_session-itest.0.TabletCopyClientSessionITest.TestStartTabletCopyWhileSourceBootstrapping.1714702442467385-9822-0/minicluster-data/ts-0/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.151.129:33077
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknrxqHk/test-tmp/tablet_copy_client_session-itest.0.TabletCopyClientSessionITest.TestStartTabletCopyWhileSourceBootstrapping.1714702442467385-9822-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.151.129
--webserver_port=43885
--tserver_master_addrs=127.9.151.190:36085
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.151.129
--log_dir=/tmp/dist-test-tasknrxqHk/test-tmp/tablet_copy_client_session-itest.0.TabletCopyClientSessionITest.TestStartTabletCopyWhileSourceBootstrapping.1714702442467385-9822-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c39e72a98e124e5a12080bac42105b996594d806
build type FASTDEBUG
built by None at 03 May 2024 01:43:22 UTC on efd1337a2aa3
build id 99
ASAN enabled
I20240503 02:14:10.604509 11159 ts_tablet_manager.cc:902] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: Initiating tablet copy from peer 7cd9a8da65604af78132e9045dac5180 (127.9.151.129:33077)
I20240503 02:14:10.604658 11156 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240503 02:14:10.605808 11159 tablet_copy_client.cc:308] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: tablet copy: Beginning tablet copy session from remote peer at address 127.9.151.129:33077
I20240503 02:14:10.606909 11156 file_cache.cc:492] Constructed file cache file cache with capacity 419430
I20240503 02:14:10.610496 11159 ts_tablet_manager.cc:902] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: Initiating tablet copy from peer 7cd9a8da65604af78132e9045dac5180 (127.9.151.129:33077)
I20240503 02:14:10.611773 11159 tablet_copy_client.cc:308] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: tablet copy: Beginning tablet copy session from remote peer at address 127.9.151.129:33077
I20240503 02:14:10.614276 11159 ts_tablet_manager.cc:902] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: Initiating tablet copy from peer 7cd9a8da65604af78132e9045dac5180 (127.9.151.129:33077)
I20240503 02:14:10.620357 11159 tablet_copy_client.cc:308] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: tablet copy: Beginning tablet copy session from remote peer at address 127.9.151.129:33077
I20240503 02:14:10.624081 11159 ts_tablet_manager.cc:902] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: Initiating tablet copy from peer 7cd9a8da65604af78132e9045dac5180 (127.9.151.129:33077)
I20240503 02:14:10.625494 11159 tablet_copy_client.cc:308] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: tablet copy: Beginning tablet copy session from remote peer at address 127.9.151.129:33077
W20240503 02:14:10.626032 11312 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240503 02:14:10.626435 11311 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240503 02:14:10.626955 11316 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240503 02:14:10.627619 11156 server_base.cc:1034] running on GCE node
I20240503 02:14:10.628429 11156 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240503 02:14:10.628686 11159 ts_tablet_manager.cc:902] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: Initiating tablet copy from peer 7cd9a8da65604af78132e9045dac5180 (127.9.151.129:33077)
I20240503 02:14:10.629557 11156 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240503 02:14:10.629889 11159 tablet_copy_client.cc:308] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: tablet copy: Beginning tablet copy session from remote peer at address 127.9.151.129:33077
I20240503 02:14:10.630806 11156 hybrid_clock.cc:648] HybridClock initialized: now 1714702450630677 us; error 104 us; skew 500 ppm
I20240503 02:14:10.631260 11156 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20240503 02:14:10.632567 11159 ts_tablet_manager.cc:902] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: Initiating tablet copy from peer 7cd9a8da65604af78132e9045dac5180 (127.9.151.129:33077)
I20240503 02:14:10.633651 11159 tablet_copy_client.cc:308] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: tablet copy: Beginning tablet copy session from remote peer at address 127.9.151.129:33077
I20240503 02:14:10.634387 11156 webserver.cc:460] Webserver started at http://127.9.151.129:43885/ using document root <none> and password file <none>
I20240503 02:14:10.635478 11156 fs_manager.cc:359] Metadata directory not provided
I20240503 02:14:10.635668 11156 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240503 02:14:10.642241 11156 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.006s sys 0.000s
I20240503 02:14:10.645898 11334 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240503 02:14:10.648094 11156 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.000s sys 0.002s
I20240503 02:14:10.648680 11159 ts_tablet_manager.cc:902] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: Initiating tablet copy from peer 7cd9a8da65604af78132e9045dac5180 (127.9.151.129:33077)
I20240503 02:14:10.648340 11156 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknrxqHk/test-tmp/tablet_copy_client_session-itest.0.TabletCopyClientSessionITest.TestStartTabletCopyWhileSourceBootstrapping.1714702442467385-9822-0/minicluster-data/ts-0/data,/tmp/dist-test-tasknrxqHk/test-tmp/tablet_copy_client_session-itest.0.TabletCopyClientSessionITest.TestStartTabletCopyWhileSourceBootstrapping.1714702442467385-9822-0/minicluster-data/ts-0/wal
uuid: "7cd9a8da65604af78132e9045dac5180"
format_stamp: "Formatted at 2024-05-03 02:14:03 on dist-test-slave-twlh"
I20240503 02:14:10.649905 11159 tablet_copy_client.cc:308] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: tablet copy: Beginning tablet copy session from remote peer at address 127.9.151.129:33077
I20240503 02:14:10.649816 11156 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknrxqHk/test-tmp/tablet_copy_client_session-itest.0.TabletCopyClientSessionITest.TestStartTabletCopyWhileSourceBootstrapping.1714702442467385-9822-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasknrxqHk/test-tmp/tablet_copy_client_session-itest.0.TabletCopyClientSessionITest.TestStartTabletCopyWhileSourceBootstrapping.1714702442467385-9822-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasknrxqHk/test-tmp/tablet_copy_client_session-itest.0.TabletCopyClientSessionITest.TestStartTabletCopyWhileSourceBootstrapping.1714702442467385-9822-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240503 02:14:10.653374 11159 ts_tablet_manager.cc:902] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: Initiating tablet copy from peer 7cd9a8da65604af78132e9045dac5180 (127.9.151.129:33077)
I20240503 02:14:10.654702 11159 tablet_copy_client.cc:308] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: tablet copy: Beginning tablet copy session from remote peer at address 127.9.151.129:33077
I20240503 02:14:10.666138 11159 ts_tablet_manager.cc:902] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: Initiating tablet copy from peer 7cd9a8da65604af78132e9045dac5180 (127.9.151.129:33077)
I20240503 02:14:10.667330 11159 tablet_copy_client.cc:308] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: tablet copy: Beginning tablet copy session from remote peer at address 127.9.151.129:33077
I20240503 02:14:10.680574 11156 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240503 02:14:10.681520 11156 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240503 02:14:10.681821 11156 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240503 02:14:10.682950 11156 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240503 02:14:10.685186 11159 ts_tablet_manager.cc:902] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: Initiating tablet copy from peer 7cd9a8da65604af78132e9045dac5180 (127.9.151.129:33077)
I20240503 02:14:10.685935 11354 ts_tablet_manager.cc:522] Loading tablet metadata (0/1 complete)
I20240503 02:14:10.686815 11159 tablet_copy_client.cc:308] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: tablet copy: Beginning tablet copy session from remote peer at address 127.9.151.129:33077
I20240503 02:14:10.691630 11159 ts_tablet_manager.cc:902] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: Initiating tablet copy from peer 7cd9a8da65604af78132e9045dac5180 (127.9.151.129:33077)
I20240503 02:14:10.692402 11156 ts_tablet_manager.cc:559] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20240503 02:14:10.692590 11156 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.007s user 0.001s sys 0.000s
I20240503 02:14:10.692754 11156 ts_tablet_manager.cc:574] Registering tablets (0/1 complete)
I20240503 02:14:10.693146 11159 tablet_copy_client.cc:308] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: tablet copy: Beginning tablet copy session from remote peer at address 127.9.151.129:33077
I20240503 02:14:10.697256 11156 ts_tablet_manager.cc:590] Registered 1 tablets
I20240503 02:14:10.697371 11156 ts_tablet_manager.cc:569] Time spent register tablets: real 0.005s user 0.002s sys 0.000s
I20240503 02:14:10.698029 11354 tablet_bootstrap.cc:492] T 375647a877014ca48ca32ff220098cd1 P 7cd9a8da65604af78132e9045dac5180: Bootstrap starting.
I20240503 02:14:10.701587 11159 ts_tablet_manager.cc:902] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: Initiating tablet copy from peer 7cd9a8da65604af78132e9045dac5180 (127.9.151.129:33077)
I20240503 02:14:10.714058 11159 tablet_copy_client.cc:308] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: tablet copy: Beginning tablet copy session from remote peer at address 127.9.151.129:33077
I20240503 02:14:10.724467 11159 ts_tablet_manager.cc:902] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: Initiating tablet copy from peer 7cd9a8da65604af78132e9045dac5180 (127.9.151.129:33077)
I20240503 02:14:10.735738 11159 tablet_copy_client.cc:308] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: tablet copy: Beginning tablet copy session from remote peer at address 127.9.151.129:33077
I20240503 02:14:10.739706 11159 ts_tablet_manager.cc:902] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: Initiating tablet copy from peer 7cd9a8da65604af78132e9045dac5180 (127.9.151.129:33077)
I20240503 02:14:10.744331 11159 tablet_copy_client.cc:308] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: tablet copy: Beginning tablet copy session from remote peer at address 127.9.151.129:33077
I20240503 02:14:10.747502 11159 ts_tablet_manager.cc:902] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: Initiating tablet copy from peer 7cd9a8da65604af78132e9045dac5180 (127.9.151.129:33077)
I20240503 02:14:10.753445 11159 tablet_copy_client.cc:308] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: tablet copy: Beginning tablet copy session from remote peer at address 127.9.151.129:33077
I20240503 02:14:10.762147 11159 ts_tablet_manager.cc:902] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: Initiating tablet copy from peer 7cd9a8da65604af78132e9045dac5180 (127.9.151.129:33077)
I20240503 02:14:10.779856 11159 tablet_copy_client.cc:308] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: tablet copy: Beginning tablet copy session from remote peer at address 127.9.151.129:33077
I20240503 02:14:10.796108 11159 ts_tablet_manager.cc:902] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: Initiating tablet copy from peer 7cd9a8da65604af78132e9045dac5180 (127.9.151.129:33077)
I20240503 02:14:10.803892 11159 tablet_copy_client.cc:308] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: tablet copy: Beginning tablet copy session from remote peer at address 127.9.151.129:33077
I20240503 02:14:10.808126 11159 ts_tablet_manager.cc:902] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: Initiating tablet copy from peer 7cd9a8da65604af78132e9045dac5180 (127.9.151.129:33077)
I20240503 02:14:10.809378 11159 tablet_copy_client.cc:308] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: tablet copy: Beginning tablet copy session from remote peer at address 127.9.151.129:33077
I20240503 02:14:10.813340 11159 ts_tablet_manager.cc:902] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: Initiating tablet copy from peer 7cd9a8da65604af78132e9045dac5180 (127.9.151.129:33077)
I20240503 02:14:10.815034 11159 tablet_copy_client.cc:308] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: tablet copy: Beginning tablet copy session from remote peer at address 127.9.151.129:33077
I20240503 02:14:10.820569 11156 rpc_server.cc:307] RPC server started. Bound to: 127.9.151.129:33077
I20240503 02:14:10.820569 11499 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.151.129:33077 every 8 connection(s)
I20240503 02:14:10.821813 11159 ts_tablet_manager.cc:902] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: Initiating tablet copy from peer 7cd9a8da65604af78132e9045dac5180 (127.9.151.129:33077)
I20240503 02:14:10.822777 11156 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknrxqHk/test-tmp/tablet_copy_client_session-itest.0.TabletCopyClientSessionITest.TestStartTabletCopyWhileSourceBootstrapping.1714702442467385-9822-0/minicluster-data/ts-0/data/info.pb
I20240503 02:14:10.825399 11159 tablet_copy_client.cc:308] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: tablet copy: Beginning tablet copy session from remote peer at address 127.9.151.129:33077
I20240503 02:14:10.828510 9822 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknrxqHk/build/asan/bin/kudu as pid 11156
I20240503 02:14:10.853693 11501 heartbeater.cc:347] Connected to a master server at 127.9.151.190:36085
I20240503 02:14:10.854108 11501 heartbeater.cc:464] Registering TS with master...
I20240503 02:14:10.854063 11463 tablet_copy_service.cc:140] P 7cd9a8da65604af78132e9045dac5180: Received BeginTabletCopySession request for tablet 375647a877014ca48ca32ff220098cd1 from peer 4c8a3af9b5184a51a3c0fe3221c01c75 ({username='slave'} at 127.9.151.130:56721)
I20240503 02:14:10.854421 11463 tablet_copy_service.cc:161] P 7cd9a8da65604af78132e9045dac5180: Beginning new tablet copy session on tablet 375647a877014ca48ca32ff220098cd1 from peer 4c8a3af9b5184a51a3c0fe3221c01c75 at {username='slave'} at 127.9.151.130:56721: session id = 4c8a3af9b5184a51a3c0fe3221c01c75-375647a877014ca48ca32ff220098cd1
I20240503 02:14:10.855170 11354 log.cc:824] T 375647a877014ca48ca32ff220098cd1 P 7cd9a8da65604af78132e9045dac5180: Log is configured to *not* fsync() on all Append() calls
I20240503 02:14:10.855466 11501 heartbeater.cc:511] Master 127.9.151.190:36085 requested a full tablet report, sending...
W20240503 02:14:10.856490 11463 tablet_copy_service.cc:478] P 7cd9a8da65604af78132e9045dac5180: Error handling TabletCopyService RPC request from {username='slave'} at 127.9.151.130:56721: Error beginning tablet copy session for tablet 375647a877014ca48ca32ff220098cd1: Illegal state: The tablet is not in a running state: BOOTSTRAPPING
I20240503 02:14:10.859990 10221 ts_manager.cc:194] Re-registered known tserver with Master: 7cd9a8da65604af78132e9045dac5180 (127.9.151.129:33077)
I20240503 02:14:10.861699 10221 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.9.151.129:34797
I20240503 02:14:10.864094 11159 ts_tablet_manager.cc:902] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: Initiating tablet copy from peer 7cd9a8da65604af78132e9045dac5180 (127.9.151.129:33077)
I20240503 02:14:10.866168 11159 tablet_copy_client.cc:308] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: tablet copy: Beginning tablet copy session from remote peer at address 127.9.151.129:33077
I20240503 02:14:10.866873 11463 tablet_copy_service.cc:140] P 7cd9a8da65604af78132e9045dac5180: Received BeginTabletCopySession request for tablet 375647a877014ca48ca32ff220098cd1 from peer 4c8a3af9b5184a51a3c0fe3221c01c75 ({username='slave'} at 127.9.151.130:56721)
I20240503 02:14:10.867067 11463 tablet_copy_service.cc:161] P 7cd9a8da65604af78132e9045dac5180: Beginning new tablet copy session on tablet 375647a877014ca48ca32ff220098cd1 from peer 4c8a3af9b5184a51a3c0fe3221c01c75 at {username='slave'} at 127.9.151.130:56721: session id = 4c8a3af9b5184a51a3c0fe3221c01c75-375647a877014ca48ca32ff220098cd1
W20240503 02:14:10.867245 11463 tablet_copy_service.cc:478] P 7cd9a8da65604af78132e9045dac5180: Error handling TabletCopyService RPC request from {username='slave'} at 127.9.151.130:56721: Error beginning tablet copy session for tablet 375647a877014ca48ca32ff220098cd1: Illegal state: The tablet is not in a running state: BOOTSTRAPPING
I20240503 02:14:10.869145 11159 ts_tablet_manager.cc:902] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: Initiating tablet copy from peer 7cd9a8da65604af78132e9045dac5180 (127.9.151.129:33077)
I20240503 02:14:10.880831 11159 tablet_copy_client.cc:308] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: tablet copy: Beginning tablet copy session from remote peer at address 127.9.151.129:33077
I20240503 02:14:10.881678 11463 tablet_copy_service.cc:140] P 7cd9a8da65604af78132e9045dac5180: Received BeginTabletCopySession request for tablet 375647a877014ca48ca32ff220098cd1 from peer 4c8a3af9b5184a51a3c0fe3221c01c75 ({username='slave'} at 127.9.151.130:56721)
I20240503 02:14:10.881872 11463 tablet_copy_service.cc:161] P 7cd9a8da65604af78132e9045dac5180: Beginning new tablet copy session on tablet 375647a877014ca48ca32ff220098cd1 from peer 4c8a3af9b5184a51a3c0fe3221c01c75 at {username='slave'} at 127.9.151.130:56721: session id = 4c8a3af9b5184a51a3c0fe3221c01c75-375647a877014ca48ca32ff220098cd1
W20240503 02:14:10.882117 11463 tablet_copy_service.cc:478] P 7cd9a8da65604af78132e9045dac5180: Error handling TabletCopyService RPC request from {username='slave'} at 127.9.151.130:56721: Error beginning tablet copy session for tablet 375647a877014ca48ca32ff220098cd1: Illegal state: The tablet is not in a running state: BOOTSTRAPPING
I20240503 02:14:10.899219 11159 ts_tablet_manager.cc:902] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: Initiating tablet copy from peer 7cd9a8da65604af78132e9045dac5180 (127.9.151.129:33077)
I20240503 02:14:10.900552 11159 tablet_copy_client.cc:308] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: tablet copy: Beginning tablet copy session from remote peer at address 127.9.151.129:33077
I20240503 02:14:10.901683 11463 tablet_copy_service.cc:140] P 7cd9a8da65604af78132e9045dac5180: Received BeginTabletCopySession request for tablet 375647a877014ca48ca32ff220098cd1 from peer 4c8a3af9b5184a51a3c0fe3221c01c75 ({username='slave'} at 127.9.151.130:56721)
I20240503 02:14:10.902135 11463 tablet_copy_service.cc:161] P 7cd9a8da65604af78132e9045dac5180: Beginning new tablet copy session on tablet 375647a877014ca48ca32ff220098cd1 from peer 4c8a3af9b5184a51a3c0fe3221c01c75 at {username='slave'} at 127.9.151.130:56721: session id = 4c8a3af9b5184a51a3c0fe3221c01c75-375647a877014ca48ca32ff220098cd1
W20240503 02:14:10.902393 11463 tablet_copy_service.cc:478] P 7cd9a8da65604af78132e9045dac5180: Error handling TabletCopyService RPC request from {username='slave'} at 127.9.151.130:56721: Error beginning tablet copy session for tablet 375647a877014ca48ca32ff220098cd1: Illegal state: The tablet is not in a running state: BOOTSTRAPPING
I20240503 02:14:10.917326 11159 ts_tablet_manager.cc:902] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: Initiating tablet copy from peer 7cd9a8da65604af78132e9045dac5180 (127.9.151.129:33077)
I20240503 02:14:10.918797 11159 tablet_copy_client.cc:308] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: tablet copy: Beginning tablet copy session from remote peer at address 127.9.151.129:33077
I20240503 02:14:10.919863 11463 tablet_copy_service.cc:140] P 7cd9a8da65604af78132e9045dac5180: Received BeginTabletCopySession request for tablet 375647a877014ca48ca32ff220098cd1 from peer 4c8a3af9b5184a51a3c0fe3221c01c75 ({username='slave'} at 127.9.151.130:56721)
I20240503 02:14:10.920099 11463 tablet_copy_service.cc:161] P 7cd9a8da65604af78132e9045dac5180: Beginning new tablet copy session on tablet 375647a877014ca48ca32ff220098cd1 from peer 4c8a3af9b5184a51a3c0fe3221c01c75 at {username='slave'} at 127.9.151.130:56721: session id = 4c8a3af9b5184a51a3c0fe3221c01c75-375647a877014ca48ca32ff220098cd1
W20240503 02:14:10.920332 11463 tablet_copy_service.cc:478] P 7cd9a8da65604af78132e9045dac5180: Error handling TabletCopyService RPC request from {username='slave'} at 127.9.151.130:56721: Error beginning tablet copy session for tablet 375647a877014ca48ca32ff220098cd1: Illegal state: The tablet is not in a running state: BOOTSTRAPPING
I20240503 02:14:10.924913 11159 ts_tablet_manager.cc:902] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: Initiating tablet copy from peer 7cd9a8da65604af78132e9045dac5180 (127.9.151.129:33077)
I20240503 02:14:10.926352 11159 tablet_copy_client.cc:308] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: tablet copy: Beginning tablet copy session from remote peer at address 127.9.151.129:33077
I20240503 02:14:10.927356 11463 tablet_copy_service.cc:140] P 7cd9a8da65604af78132e9045dac5180: Received BeginTabletCopySession request for tablet 375647a877014ca48ca32ff220098cd1 from peer 4c8a3af9b5184a51a3c0fe3221c01c75 ({username='slave'} at 127.9.151.130:56721)
I20240503 02:14:10.927578 11463 tablet_copy_service.cc:161] P 7cd9a8da65604af78132e9045dac5180: Beginning new tablet copy session on tablet 375647a877014ca48ca32ff220098cd1 from peer 4c8a3af9b5184a51a3c0fe3221c01c75 at {username='slave'} at 127.9.151.130:56721: session id = 4c8a3af9b5184a51a3c0fe3221c01c75-375647a877014ca48ca32ff220098cd1
W20240503 02:14:10.927799 11463 tablet_copy_service.cc:478] P 7cd9a8da65604af78132e9045dac5180: Error handling TabletCopyService RPC request from {username='slave'} at 127.9.151.130:56721: Error beginning tablet copy session for tablet 375647a877014ca48ca32ff220098cd1: Illegal state: The tablet is not in a running state: BOOTSTRAPPING
I20240503 02:14:10.943709 11159 ts_tablet_manager.cc:902] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: Initiating tablet copy from peer 7cd9a8da65604af78132e9045dac5180 (127.9.151.129:33077)
I20240503 02:14:10.945067 11159 tablet_copy_client.cc:308] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: tablet copy: Beginning tablet copy session from remote peer at address 127.9.151.129:33077
I20240503 02:14:10.945976 11463 tablet_copy_service.cc:140] P 7cd9a8da65604af78132e9045dac5180: Received BeginTabletCopySession request for tablet 375647a877014ca48ca32ff220098cd1 from peer 4c8a3af9b5184a51a3c0fe3221c01c75 ({username='slave'} at 127.9.151.130:56721)
I20240503 02:14:10.946152 11463 tablet_copy_service.cc:161] P 7cd9a8da65604af78132e9045dac5180: Beginning new tablet copy session on tablet 375647a877014ca48ca32ff220098cd1 from peer 4c8a3af9b5184a51a3c0fe3221c01c75 at {username='slave'} at 127.9.151.130:56721: session id = 4c8a3af9b5184a51a3c0fe3221c01c75-375647a877014ca48ca32ff220098cd1
W20240503 02:14:10.946381 11463 tablet_copy_service.cc:478] P 7cd9a8da65604af78132e9045dac5180: Error handling TabletCopyService RPC request from {username='slave'} at 127.9.151.130:56721: Error beginning tablet copy session for tablet 375647a877014ca48ca32ff220098cd1: Illegal state: The tablet is not in a running state: BOOTSTRAPPING
I20240503 02:14:10.951675 11159 ts_tablet_manager.cc:902] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: Initiating tablet copy from peer 7cd9a8da65604af78132e9045dac5180 (127.9.151.129:33077)
I20240503 02:14:10.963814 11159 tablet_copy_client.cc:308] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: tablet copy: Beginning tablet copy session from remote peer at address 127.9.151.129:33077
I20240503 02:14:10.964705 11463 tablet_copy_service.cc:140] P 7cd9a8da65604af78132e9045dac5180: Received BeginTabletCopySession request for tablet 375647a877014ca48ca32ff220098cd1 from peer 4c8a3af9b5184a51a3c0fe3221c01c75 ({username='slave'} at 127.9.151.130:56721)
I20240503 02:14:10.964893 11463 tablet_copy_service.cc:161] P 7cd9a8da65604af78132e9045dac5180: Beginning new tablet copy session on tablet 375647a877014ca48ca32ff220098cd1 from peer 4c8a3af9b5184a51a3c0fe3221c01c75 at {username='slave'} at 127.9.151.130:56721: session id = 4c8a3af9b5184a51a3c0fe3221c01c75-375647a877014ca48ca32ff220098cd1
W20240503 02:14:10.965047 11463 tablet_copy_service.cc:478] P 7cd9a8da65604af78132e9045dac5180: Error handling TabletCopyService RPC request from {username='slave'} at 127.9.151.130:56721: Error beginning tablet copy session for tablet 375647a877014ca48ca32ff220098cd1: Illegal state: The tablet is not in a running state: BOOTSTRAPPING
I20240503 02:14:10.968012 11159 ts_tablet_manager.cc:902] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: Initiating tablet copy from peer 7cd9a8da65604af78132e9045dac5180 (127.9.151.129:33077)
I20240503 02:14:10.969192 11159 tablet_copy_client.cc:308] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: tablet copy: Beginning tablet copy session from remote peer at address 127.9.151.129:33077
I20240503 02:14:10.970038 11463 tablet_copy_service.cc:140] P 7cd9a8da65604af78132e9045dac5180: Received BeginTabletCopySession request for tablet 375647a877014ca48ca32ff220098cd1 from peer 4c8a3af9b5184a51a3c0fe3221c01c75 ({username='slave'} at 127.9.151.130:56721)
I20240503 02:14:10.970242 11463 tablet_copy_service.cc:161] P 7cd9a8da65604af78132e9045dac5180: Beginning new tablet copy session on tablet 375647a877014ca48ca32ff220098cd1 from peer 4c8a3af9b5184a51a3c0fe3221c01c75 at {username='slave'} at 127.9.151.130:56721: session id = 4c8a3af9b5184a51a3c0fe3221c01c75-375647a877014ca48ca32ff220098cd1
W20240503 02:14:10.970418 11463 tablet_copy_service.cc:478] P 7cd9a8da65604af78132e9045dac5180: Error handling TabletCopyService RPC request from {username='slave'} at 127.9.151.130:56721: Error beginning tablet copy session for tablet 375647a877014ca48ca32ff220098cd1: Illegal state: The tablet is not in a running state: BOOTSTRAPPING
I20240503 02:14:10.976528 11159 ts_tablet_manager.cc:902] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: Initiating tablet copy from peer 7cd9a8da65604af78132e9045dac5180 (127.9.151.129:33077)
I20240503 02:14:10.978533 11159 tablet_copy_client.cc:308] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: tablet copy: Beginning tablet copy session from remote peer at address 127.9.151.129:33077
I20240503 02:14:10.979405 11463 tablet_copy_service.cc:140] P 7cd9a8da65604af78132e9045dac5180: Received BeginTabletCopySession request for tablet 375647a877014ca48ca32ff220098cd1 from peer 4c8a3af9b5184a51a3c0fe3221c01c75 ({username='slave'} at 127.9.151.130:56721)
I20240503 02:14:10.979604 11463 tablet_copy_service.cc:161] P 7cd9a8da65604af78132e9045dac5180: Beginning new tablet copy session on tablet 375647a877014ca48ca32ff220098cd1 from peer 4c8a3af9b5184a51a3c0fe3221c01c75 at {username='slave'} at 127.9.151.130:56721: session id = 4c8a3af9b5184a51a3c0fe3221c01c75-375647a877014ca48ca32ff220098cd1
W20240503 02:14:10.979816 11463 tablet_copy_service.cc:478] P 7cd9a8da65604af78132e9045dac5180: Error handling TabletCopyService RPC request from {username='slave'} at 127.9.151.130:56721: Error beginning tablet copy session for tablet 375647a877014ca48ca32ff220098cd1: Illegal state: The tablet is not in a running state: BOOTSTRAPPING
I20240503 02:14:10.988746 11159 ts_tablet_manager.cc:902] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: Initiating tablet copy from peer 7cd9a8da65604af78132e9045dac5180 (127.9.151.129:33077)
I20240503 02:14:10.993443 11159 tablet_copy_client.cc:308] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: tablet copy: Beginning tablet copy session from remote peer at address 127.9.151.129:33077
I20240503 02:14:10.994191 11463 tablet_copy_service.cc:140] P 7cd9a8da65604af78132e9045dac5180: Received BeginTabletCopySession request for tablet 375647a877014ca48ca32ff220098cd1 from peer 4c8a3af9b5184a51a3c0fe3221c01c75 ({username='slave'} at 127.9.151.130:56721)
I20240503 02:14:10.994361 11463 tablet_copy_service.cc:161] P 7cd9a8da65604af78132e9045dac5180: Beginning new tablet copy session on tablet 375647a877014ca48ca32ff220098cd1 from peer 4c8a3af9b5184a51a3c0fe3221c01c75 at {username='slave'} at 127.9.151.130:56721: session id = 4c8a3af9b5184a51a3c0fe3221c01c75-375647a877014ca48ca32ff220098cd1
W20240503 02:14:10.994552 11463 tablet_copy_service.cc:478] P 7cd9a8da65604af78132e9045dac5180: Error handling TabletCopyService RPC request from {username='slave'} at 127.9.151.130:56721: Error beginning tablet copy session for tablet 375647a877014ca48ca32ff220098cd1: Illegal state: The tablet is not in a running state: BOOTSTRAPPING
I20240503 02:14:10.996656 11159 ts_tablet_manager.cc:902] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: Initiating tablet copy from peer 7cd9a8da65604af78132e9045dac5180 (127.9.151.129:33077)
I20240503 02:14:11.003217 11159 tablet_copy_client.cc:308] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: tablet copy: Beginning tablet copy session from remote peer at address 127.9.151.129:33077
I20240503 02:14:11.004225 11463 tablet_copy_service.cc:140] P 7cd9a8da65604af78132e9045dac5180: Received BeginTabletCopySession request for tablet 375647a877014ca48ca32ff220098cd1 from peer 4c8a3af9b5184a51a3c0fe3221c01c75 ({username='slave'} at 127.9.151.130:56721)
I20240503 02:14:11.004460 11463 tablet_copy_service.cc:161] P 7cd9a8da65604af78132e9045dac5180: Beginning new tablet copy session on tablet 375647a877014ca48ca32ff220098cd1 from peer 4c8a3af9b5184a51a3c0fe3221c01c75 at {username='slave'} at 127.9.151.130:56721: session id = 4c8a3af9b5184a51a3c0fe3221c01c75-375647a877014ca48ca32ff220098cd1
W20240503 02:14:11.004635 11463 tablet_copy_service.cc:478] P 7cd9a8da65604af78132e9045dac5180: Error handling TabletCopyService RPC request from {username='slave'} at 127.9.151.130:56721: Error beginning tablet copy session for tablet 375647a877014ca48ca32ff220098cd1: Illegal state: The tablet is not in a running state: BOOTSTRAPPING
I20240503 02:14:11.020355 11159 ts_tablet_manager.cc:902] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: Initiating tablet copy from peer 7cd9a8da65604af78132e9045dac5180 (127.9.151.129:33077)
I20240503 02:14:11.022246 11159 tablet_copy_client.cc:308] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: tablet copy: Beginning tablet copy session from remote peer at address 127.9.151.129:33077
I20240503 02:14:11.023569 11463 tablet_copy_service.cc:140] P 7cd9a8da65604af78132e9045dac5180: Received BeginTabletCopySession request for tablet 375647a877014ca48ca32ff220098cd1 from peer 4c8a3af9b5184a51a3c0fe3221c01c75 ({username='slave'} at 127.9.151.130:56721)
I20240503 02:14:11.023787 11463 tablet_copy_service.cc:161] P 7cd9a8da65604af78132e9045dac5180: Beginning new tablet copy session on tablet 375647a877014ca48ca32ff220098cd1 from peer 4c8a3af9b5184a51a3c0fe3221c01c75 at {username='slave'} at 127.9.151.130:56721: session id = 4c8a3af9b5184a51a3c0fe3221c01c75-375647a877014ca48ca32ff220098cd1
W20240503 02:14:11.024024 11463 tablet_copy_service.cc:478] P 7cd9a8da65604af78132e9045dac5180: Error handling TabletCopyService RPC request from {username='slave'} at 127.9.151.130:56721: Error beginning tablet copy session for tablet 375647a877014ca48ca32ff220098cd1: Illegal state: The tablet is not in a running state: BOOTSTRAPPING
I20240503 02:14:11.027503 11159 ts_tablet_manager.cc:902] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: Initiating tablet copy from peer 7cd9a8da65604af78132e9045dac5180 (127.9.151.129:33077)
I20240503 02:14:11.037024 11159 tablet_copy_client.cc:308] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: tablet copy: Beginning tablet copy session from remote peer at address 127.9.151.129:33077
I20240503 02:14:11.038082 11463 tablet_copy_service.cc:140] P 7cd9a8da65604af78132e9045dac5180: Received BeginTabletCopySession request for tablet 375647a877014ca48ca32ff220098cd1 from peer 4c8a3af9b5184a51a3c0fe3221c01c75 ({username='slave'} at 127.9.151.130:56721)
I20240503 02:14:11.038311 11463 tablet_copy_service.cc:161] P 7cd9a8da65604af78132e9045dac5180: Beginning new tablet copy session on tablet 375647a877014ca48ca32ff220098cd1 from peer 4c8a3af9b5184a51a3c0fe3221c01c75 at {username='slave'} at 127.9.151.130:56721: session id = 4c8a3af9b5184a51a3c0fe3221c01c75-375647a877014ca48ca32ff220098cd1
W20240503 02:14:11.038502 11463 tablet_copy_service.cc:478] P 7cd9a8da65604af78132e9045dac5180: Error handling TabletCopyService RPC request from {username='slave'} at 127.9.151.130:56721: Error beginning tablet copy session for tablet 375647a877014ca48ca32ff220098cd1: Illegal state: The tablet is not in a running state: BOOTSTRAPPING
I20240503 02:14:11.048738 11144 heartbeater.cc:503] Master 127.9.151.190:36085 was elected leader, sending a full tablet report...
I20240503 02:14:11.049296 11159 ts_tablet_manager.cc:902] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: Initiating tablet copy from peer 7cd9a8da65604af78132e9045dac5180 (127.9.151.129:33077)
I20240503 02:14:11.050806 11159 tablet_copy_client.cc:308] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: tablet copy: Beginning tablet copy session from remote peer at address 127.9.151.129:33077
I20240503 02:14:11.051649 11463 tablet_copy_service.cc:140] P 7cd9a8da65604af78132e9045dac5180: Received BeginTabletCopySession request for tablet 375647a877014ca48ca32ff220098cd1 from peer 4c8a3af9b5184a51a3c0fe3221c01c75 ({username='slave'} at 127.9.151.130:56721)
I20240503 02:14:11.051849 11463 tablet_copy_service.cc:161] P 7cd9a8da65604af78132e9045dac5180: Beginning new tablet copy session on tablet 375647a877014ca48ca32ff220098cd1 from peer 4c8a3af9b5184a51a3c0fe3221c01c75 at {username='slave'} at 127.9.151.130:56721: session id = 4c8a3af9b5184a51a3c0fe3221c01c75-375647a877014ca48ca32ff220098cd1
W20240503 02:14:11.052064 11463 tablet_copy_service.cc:478] P 7cd9a8da65604af78132e9045dac5180: Error handling TabletCopyService RPC request from {username='slave'} at 127.9.151.130:56721: Error beginning tablet copy session for tablet 375647a877014ca48ca32ff220098cd1: Illegal state: The tablet is not in a running state: BOOTSTRAPPING
I20240503 02:14:11.066637 11159 ts_tablet_manager.cc:902] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: Initiating tablet copy from peer 7cd9a8da65604af78132e9045dac5180 (127.9.151.129:33077)
I20240503 02:14:11.071581 11159 tablet_copy_client.cc:308] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: tablet copy: Beginning tablet copy session from remote peer at address 127.9.151.129:33077
I20240503 02:14:11.072527 11463 tablet_copy_service.cc:140] P 7cd9a8da65604af78132e9045dac5180: Received BeginTabletCopySession request for tablet 375647a877014ca48ca32ff220098cd1 from peer 4c8a3af9b5184a51a3c0fe3221c01c75 ({username='slave'} at 127.9.151.130:56721)
I20240503 02:14:11.072764 11463 tablet_copy_service.cc:161] P 7cd9a8da65604af78132e9045dac5180: Beginning new tablet copy session on tablet 375647a877014ca48ca32ff220098cd1 from peer 4c8a3af9b5184a51a3c0fe3221c01c75 at {username='slave'} at 127.9.151.130:56721: session id = 4c8a3af9b5184a51a3c0fe3221c01c75-375647a877014ca48ca32ff220098cd1
W20240503 02:14:11.073011 11463 tablet_copy_service.cc:478] P 7cd9a8da65604af78132e9045dac5180: Error handling TabletCopyService RPC request from {username='slave'} at 127.9.151.130:56721: Error beginning tablet copy session for tablet 375647a877014ca48ca32ff220098cd1: Illegal state: The tablet is not in a running state: BOOTSTRAPPING
I20240503 02:14:11.079766 11159 ts_tablet_manager.cc:902] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: Initiating tablet copy from peer 7cd9a8da65604af78132e9045dac5180 (127.9.151.129:33077)
I20240503 02:14:11.081652 11159 tablet_copy_client.cc:308] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: tablet copy: Beginning tablet copy session from remote peer at address 127.9.151.129:33077
I20240503 02:14:11.082443 11463 tablet_copy_service.cc:140] P 7cd9a8da65604af78132e9045dac5180: Received BeginTabletCopySession request for tablet 375647a877014ca48ca32ff220098cd1 from peer 4c8a3af9b5184a51a3c0fe3221c01c75 ({username='slave'} at 127.9.151.130:56721)
I20240503 02:14:11.082619 11463 tablet_copy_service.cc:161] P 7cd9a8da65604af78132e9045dac5180: Beginning new tablet copy session on tablet 375647a877014ca48ca32ff220098cd1 from peer 4c8a3af9b5184a51a3c0fe3221c01c75 at {username='slave'} at 127.9.151.130:56721: session id = 4c8a3af9b5184a51a3c0fe3221c01c75-375647a877014ca48ca32ff220098cd1
W20240503 02:14:11.082839 11463 tablet_copy_service.cc:478] P 7cd9a8da65604af78132e9045dac5180: Error handling TabletCopyService RPC request from {username='slave'} at 127.9.151.130:56721: Error beginning tablet copy session for tablet 375647a877014ca48ca32ff220098cd1: Illegal state: The tablet is not in a running state: BOOTSTRAPPING
I20240503 02:14:11.085153 11159 ts_tablet_manager.cc:902] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: Initiating tablet copy from peer 7cd9a8da65604af78132e9045dac5180 (127.9.151.129:33077)
I20240503 02:14:11.086364 11159 tablet_copy_client.cc:308] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: tablet copy: Beginning tablet copy session from remote peer at address 127.9.151.129:33077
I20240503 02:14:11.087069 11463 tablet_copy_service.cc:140] P 7cd9a8da65604af78132e9045dac5180: Received BeginTabletCopySession request for tablet 375647a877014ca48ca32ff220098cd1 from peer 4c8a3af9b5184a51a3c0fe3221c01c75 ({username='slave'} at 127.9.151.130:56721)
I20240503 02:14:11.087245 11463 tablet_copy_service.cc:161] P 7cd9a8da65604af78132e9045dac5180: Beginning new tablet copy session on tablet 375647a877014ca48ca32ff220098cd1 from peer 4c8a3af9b5184a51a3c0fe3221c01c75 at {username='slave'} at 127.9.151.130:56721: session id = 4c8a3af9b5184a51a3c0fe3221c01c75-375647a877014ca48ca32ff220098cd1
W20240503 02:14:11.087415 11463 tablet_copy_service.cc:478] P 7cd9a8da65604af78132e9045dac5180: Error handling TabletCopyService RPC request from {username='slave'} at 127.9.151.130:56721: Error beginning tablet copy session for tablet 375647a877014ca48ca32ff220098cd1: Illegal state: The tablet is not in a running state: BOOTSTRAPPING
I20240503 02:14:11.096403 11159 ts_tablet_manager.cc:902] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: Initiating tablet copy from peer 7cd9a8da65604af78132e9045dac5180 (127.9.151.129:33077)
I20240503 02:14:11.097709 11159 tablet_copy_client.cc:308] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: tablet copy: Beginning tablet copy session from remote peer at address 127.9.151.129:33077
I20240503 02:14:11.098552 11463 tablet_copy_service.cc:140] P 7cd9a8da65604af78132e9045dac5180: Received BeginTabletCopySession request for tablet 375647a877014ca48ca32ff220098cd1 from peer 4c8a3af9b5184a51a3c0fe3221c01c75 ({username='slave'} at 127.9.151.130:56721)
I20240503 02:14:11.098788 11463 tablet_copy_service.cc:161] P 7cd9a8da65604af78132e9045dac5180: Beginning new tablet copy session on tablet 375647a877014ca48ca32ff220098cd1 from peer 4c8a3af9b5184a51a3c0fe3221c01c75 at {username='slave'} at 127.9.151.130:56721: session id = 4c8a3af9b5184a51a3c0fe3221c01c75-375647a877014ca48ca32ff220098cd1
W20240503 02:14:11.099002 11463 tablet_copy_service.cc:478] P 7cd9a8da65604af78132e9045dac5180: Error handling TabletCopyService RPC request from {username='slave'} at 127.9.151.130:56721: Error beginning tablet copy session for tablet 375647a877014ca48ca32ff220098cd1: Illegal state: The tablet is not in a running state: BOOTSTRAPPING
I20240503 02:14:11.104570 11159 ts_tablet_manager.cc:902] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: Initiating tablet copy from peer 7cd9a8da65604af78132e9045dac5180 (127.9.151.129:33077)
I20240503 02:14:11.111568 11159 tablet_copy_client.cc:308] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: tablet copy: Beginning tablet copy session from remote peer at address 127.9.151.129:33077
I20240503 02:14:11.112417 11463 tablet_copy_service.cc:140] P 7cd9a8da65604af78132e9045dac5180: Received BeginTabletCopySession request for tablet 375647a877014ca48ca32ff220098cd1 from peer 4c8a3af9b5184a51a3c0fe3221c01c75 ({username='slave'} at 127.9.151.130:56721)
I20240503 02:14:11.112635 11463 tablet_copy_service.cc:161] P 7cd9a8da65604af78132e9045dac5180: Beginning new tablet copy session on tablet 375647a877014ca48ca32ff220098cd1 from peer 4c8a3af9b5184a51a3c0fe3221c01c75 at {username='slave'} at 127.9.151.130:56721: session id = 4c8a3af9b5184a51a3c0fe3221c01c75-375647a877014ca48ca32ff220098cd1
W20240503 02:14:11.112874 11463 tablet_copy_service.cc:478] P 7cd9a8da65604af78132e9045dac5180: Error handling TabletCopyService RPC request from {username='slave'} at 127.9.151.130:56721: Error beginning tablet copy session for tablet 375647a877014ca48ca32ff220098cd1: Illegal state: The tablet is not in a running state: BOOTSTRAPPING
I20240503 02:14:11.120908 11159 ts_tablet_manager.cc:902] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: Initiating tablet copy from peer 7cd9a8da65604af78132e9045dac5180 (127.9.151.129:33077)
I20240503 02:14:11.122232 11159 tablet_copy_client.cc:308] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: tablet copy: Beginning tablet copy session from remote peer at address 127.9.151.129:33077
I20240503 02:14:11.123095 11463 tablet_copy_service.cc:140] P 7cd9a8da65604af78132e9045dac5180: Received BeginTabletCopySession request for tablet 375647a877014ca48ca32ff220098cd1 from peer 4c8a3af9b5184a51a3c0fe3221c01c75 ({username='slave'} at 127.9.151.130:56721)
I20240503 02:14:11.123258 11463 tablet_copy_service.cc:161] P 7cd9a8da65604af78132e9045dac5180: Beginning new tablet copy session on tablet 375647a877014ca48ca32ff220098cd1 from peer 4c8a3af9b5184a51a3c0fe3221c01c75 at {username='slave'} at 127.9.151.130:56721: session id = 4c8a3af9b5184a51a3c0fe3221c01c75-375647a877014ca48ca32ff220098cd1
W20240503 02:14:11.123440 11463 tablet_copy_service.cc:478] P 7cd9a8da65604af78132e9045dac5180: Error handling TabletCopyService RPC request from {username='slave'} at 127.9.151.130:56721: Error beginning tablet copy session for tablet 375647a877014ca48ca32ff220098cd1: Illegal state: The tablet is not in a running state: BOOTSTRAPPING
I20240503 02:14:11.131987 11159 ts_tablet_manager.cc:902] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: Initiating tablet copy from peer 7cd9a8da65604af78132e9045dac5180 (127.9.151.129:33077)
I20240503 02:14:11.133775 11159 tablet_copy_client.cc:308] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: tablet copy: Beginning tablet copy session from remote peer at address 127.9.151.129:33077
I20240503 02:14:11.134732 11463 tablet_copy_service.cc:140] P 7cd9a8da65604af78132e9045dac5180: Received BeginTabletCopySession request for tablet 375647a877014ca48ca32ff220098cd1 from peer 4c8a3af9b5184a51a3c0fe3221c01c75 ({username='slave'} at 127.9.151.130:56721)
I20240503 02:14:11.134950 11463 tablet_copy_service.cc:161] P 7cd9a8da65604af78132e9045dac5180: Beginning new tablet copy session on tablet 375647a877014ca48ca32ff220098cd1 from peer 4c8a3af9b5184a51a3c0fe3221c01c75 at {username='slave'} at 127.9.151.130:56721: session id = 4c8a3af9b5184a51a3c0fe3221c01c75-375647a877014ca48ca32ff220098cd1
W20240503 02:14:11.135166 11463 tablet_copy_service.cc:478] P 7cd9a8da65604af78132e9045dac5180: Error handling TabletCopyService RPC request from {username='slave'} at 127.9.151.130:56721: Error beginning tablet copy session for tablet 375647a877014ca48ca32ff220098cd1: Illegal state: The tablet is not in a running state: BOOTSTRAPPING
I20240503 02:14:11.141826 11159 ts_tablet_manager.cc:902] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: Initiating tablet copy from peer 7cd9a8da65604af78132e9045dac5180 (127.9.151.129:33077)
I20240503 02:14:11.143333 11159 tablet_copy_client.cc:308] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: tablet copy: Beginning tablet copy session from remote peer at address 127.9.151.129:33077
I20240503 02:14:11.144251 11463 tablet_copy_service.cc:140] P 7cd9a8da65604af78132e9045dac5180: Received BeginTabletCopySession request for tablet 375647a877014ca48ca32ff220098cd1 from peer 4c8a3af9b5184a51a3c0fe3221c01c75 ({username='slave'} at 127.9.151.130:56721)
I20240503 02:14:11.144454 11463 tablet_copy_service.cc:161] P 7cd9a8da65604af78132e9045dac5180: Beginning new tablet copy session on tablet 375647a877014ca48ca32ff220098cd1 from peer 4c8a3af9b5184a51a3c0fe3221c01c75 at {username='slave'} at 127.9.151.130:56721: session id = 4c8a3af9b5184a51a3c0fe3221c01c75-375647a877014ca48ca32ff220098cd1
W20240503 02:14:11.144676 11463 tablet_copy_service.cc:478] P 7cd9a8da65604af78132e9045dac5180: Error handling TabletCopyService RPC request from {username='slave'} at 127.9.151.130:56721: Error beginning tablet copy session for tablet 375647a877014ca48ca32ff220098cd1: Illegal state: The tablet is not in a running state: BOOTSTRAPPING
I20240503 02:14:11.157830 11159 ts_tablet_manager.cc:902] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: Initiating tablet copy from peer 7cd9a8da65604af78132e9045dac5180 (127.9.151.129:33077)
I20240503 02:14:11.167052 11159 tablet_copy_client.cc:308] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: tablet copy: Beginning tablet copy session from remote peer at address 127.9.151.129:33077
I20240503 02:14:11.167838 11463 tablet_copy_service.cc:140] P 7cd9a8da65604af78132e9045dac5180: Received BeginTabletCopySession request for tablet 375647a877014ca48ca32ff220098cd1 from peer 4c8a3af9b5184a51a3c0fe3221c01c75 ({username='slave'} at 127.9.151.130:56721)
I20240503 02:14:11.168033 11463 tablet_copy_service.cc:161] P 7cd9a8da65604af78132e9045dac5180: Beginning new tablet copy session on tablet 375647a877014ca48ca32ff220098cd1 from peer 4c8a3af9b5184a51a3c0fe3221c01c75 at {username='slave'} at 127.9.151.130:56721: session id = 4c8a3af9b5184a51a3c0fe3221c01c75-375647a877014ca48ca32ff220098cd1
W20240503 02:14:11.168208 11463 tablet_copy_service.cc:478] P 7cd9a8da65604af78132e9045dac5180: Error handling TabletCopyService RPC request from {username='slave'} at 127.9.151.130:56721: Error beginning tablet copy session for tablet 375647a877014ca48ca32ff220098cd1: Illegal state: The tablet is not in a running state: BOOTSTRAPPING
I20240503 02:14:11.176839 11159 ts_tablet_manager.cc:902] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: Initiating tablet copy from peer 7cd9a8da65604af78132e9045dac5180 (127.9.151.129:33077)
I20240503 02:14:11.179052 11159 tablet_copy_client.cc:308] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: tablet copy: Beginning tablet copy session from remote peer at address 127.9.151.129:33077
I20240503 02:14:11.179966 11463 tablet_copy_service.cc:140] P 7cd9a8da65604af78132e9045dac5180: Received BeginTabletCopySession request for tablet 375647a877014ca48ca32ff220098cd1 from peer 4c8a3af9b5184a51a3c0fe3221c01c75 ({username='slave'} at 127.9.151.130:56721)
I20240503 02:14:11.180203 11463 tablet_copy_service.cc:161] P 7cd9a8da65604af78132e9045dac5180: Beginning new tablet copy session on tablet 375647a877014ca48ca32ff220098cd1 from peer 4c8a3af9b5184a51a3c0fe3221c01c75 at {username='slave'} at 127.9.151.130:56721: session id = 4c8a3af9b5184a51a3c0fe3221c01c75-375647a877014ca48ca32ff220098cd1
W20240503 02:14:11.180416 11463 tablet_copy_service.cc:478] P 7cd9a8da65604af78132e9045dac5180: Error handling TabletCopyService RPC request from {username='slave'} at 127.9.151.130:56721: Error beginning tablet copy session for tablet 375647a877014ca48ca32ff220098cd1: Illegal state: The tablet is not in a running state: BOOTSTRAPPING
I20240503 02:14:11.185684 11159 ts_tablet_manager.cc:902] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: Initiating tablet copy from peer 7cd9a8da65604af78132e9045dac5180 (127.9.151.129:33077)
I20240503 02:14:11.186966 11159 tablet_copy_client.cc:308] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: tablet copy: Beginning tablet copy session from remote peer at address 127.9.151.129:33077
I20240503 02:14:11.187804 11463 tablet_copy_service.cc:140] P 7cd9a8da65604af78132e9045dac5180: Received BeginTabletCopySession request for tablet 375647a877014ca48ca32ff220098cd1 from peer 4c8a3af9b5184a51a3c0fe3221c01c75 ({username='slave'} at 127.9.151.130:56721)
I20240503 02:14:11.188019 11463 tablet_copy_service.cc:161] P 7cd9a8da65604af78132e9045dac5180: Beginning new tablet copy session on tablet 375647a877014ca48ca32ff220098cd1 from peer 4c8a3af9b5184a51a3c0fe3221c01c75 at {username='slave'} at 127.9.151.130:56721: session id = 4c8a3af9b5184a51a3c0fe3221c01c75-375647a877014ca48ca32ff220098cd1
W20240503 02:14:11.188215 11463 tablet_copy_service.cc:478] P 7cd9a8da65604af78132e9045dac5180: Error handling TabletCopyService RPC request from {username='slave'} at 127.9.151.130:56721: Error beginning tablet copy session for tablet 375647a877014ca48ca32ff220098cd1: Illegal state: The tablet is not in a running state: BOOTSTRAPPING
I20240503 02:14:11.195111 11159 ts_tablet_manager.cc:902] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: Initiating tablet copy from peer 7cd9a8da65604af78132e9045dac5180 (127.9.151.129:33077)
I20240503 02:14:11.199014 11159 tablet_copy_client.cc:308] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: tablet copy: Beginning tablet copy session from remote peer at address 127.9.151.129:33077
I20240503 02:14:11.199894 11463 tablet_copy_service.cc:140] P 7cd9a8da65604af78132e9045dac5180: Received BeginTabletCopySession request for tablet 375647a877014ca48ca32ff220098cd1 from peer 4c8a3af9b5184a51a3c0fe3221c01c75 ({username='slave'} at 127.9.151.130:56721)
I20240503 02:14:11.200073 11463 tablet_copy_service.cc:161] P 7cd9a8da65604af78132e9045dac5180: Beginning new tablet copy session on tablet 375647a877014ca48ca32ff220098cd1 from peer 4c8a3af9b5184a51a3c0fe3221c01c75 at {username='slave'} at 127.9.151.130:56721: session id = 4c8a3af9b5184a51a3c0fe3221c01c75-375647a877014ca48ca32ff220098cd1
W20240503 02:14:11.200249 11463 tablet_copy_service.cc:478] P 7cd9a8da65604af78132e9045dac5180: Error handling TabletCopyService RPC request from {username='slave'} at 127.9.151.130:56721: Error beginning tablet copy session for tablet 375647a877014ca48ca32ff220098cd1: Illegal state: The tablet is not in a running state: BOOTSTRAPPING
I20240503 02:14:11.203182 11160 ts_tablet_manager.cc:902] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: Initiating tablet copy from peer 7cd9a8da65604af78132e9045dac5180 (127.9.151.129:33077)
I20240503 02:14:11.209321 11160 tablet_copy_client.cc:308] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: tablet copy: Beginning tablet copy session from remote peer at address 127.9.151.129:33077
I20240503 02:14:11.210110 11463 tablet_copy_service.cc:140] P 7cd9a8da65604af78132e9045dac5180: Received BeginTabletCopySession request for tablet 375647a877014ca48ca32ff220098cd1 from peer 4c8a3af9b5184a51a3c0fe3221c01c75 ({username='slave'} at 127.9.151.130:56721)
I20240503 02:14:11.210314 11463 tablet_copy_service.cc:161] P 7cd9a8da65604af78132e9045dac5180: Beginning new tablet copy session on tablet 375647a877014ca48ca32ff220098cd1 from peer 4c8a3af9b5184a51a3c0fe3221c01c75 at {username='slave'} at 127.9.151.130:56721: session id = 4c8a3af9b5184a51a3c0fe3221c01c75-375647a877014ca48ca32ff220098cd1
W20240503 02:14:11.210494 11463 tablet_copy_service.cc:478] P 7cd9a8da65604af78132e9045dac5180: Error handling TabletCopyService RPC request from {username='slave'} at 127.9.151.130:56721: Error beginning tablet copy session for tablet 375647a877014ca48ca32ff220098cd1: Illegal state: The tablet is not in a running state: BOOTSTRAPPING
I20240503 02:14:11.218808 11160 ts_tablet_manager.cc:902] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: Initiating tablet copy from peer 7cd9a8da65604af78132e9045dac5180 (127.9.151.129:33077)
I20240503 02:14:11.220101 11160 tablet_copy_client.cc:308] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: tablet copy: Beginning tablet copy session from remote peer at address 127.9.151.129:33077
I20240503 02:14:11.221096 11463 tablet_copy_service.cc:140] P 7cd9a8da65604af78132e9045dac5180: Received BeginTabletCopySession request for tablet 375647a877014ca48ca32ff220098cd1 from peer 4c8a3af9b5184a51a3c0fe3221c01c75 ({username='slave'} at 127.9.151.130:56721)
I20240503 02:14:11.221334 11463 tablet_copy_service.cc:161] P 7cd9a8da65604af78132e9045dac5180: Beginning new tablet copy session on tablet 375647a877014ca48ca32ff220098cd1 from peer 4c8a3af9b5184a51a3c0fe3221c01c75 at {username='slave'} at 127.9.151.130:56721: session id = 4c8a3af9b5184a51a3c0fe3221c01c75-375647a877014ca48ca32ff220098cd1
W20240503 02:14:11.221563 11463 tablet_copy_service.cc:478] P 7cd9a8da65604af78132e9045dac5180: Error handling TabletCopyService RPC request from {username='slave'} at 127.9.151.130:56721: Error beginning tablet copy session for tablet 375647a877014ca48ca32ff220098cd1: Illegal state: The tablet is not in a running state: BOOTSTRAPPING
I20240503 02:14:11.236807 11160 ts_tablet_manager.cc:902] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: Initiating tablet copy from peer 7cd9a8da65604af78132e9045dac5180 (127.9.151.129:33077)
I20240503 02:14:11.238981 11160 tablet_copy_client.cc:308] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: tablet copy: Beginning tablet copy session from remote peer at address 127.9.151.129:33077
I20240503 02:14:11.240114 11463 tablet_copy_service.cc:140] P 7cd9a8da65604af78132e9045dac5180: Received BeginTabletCopySession request for tablet 375647a877014ca48ca32ff220098cd1 from peer 4c8a3af9b5184a51a3c0fe3221c01c75 ({username='slave'} at 127.9.151.130:56721)
I20240503 02:14:11.240392 11463 tablet_copy_service.cc:161] P 7cd9a8da65604af78132e9045dac5180: Beginning new tablet copy session on tablet 375647a877014ca48ca32ff220098cd1 from peer 4c8a3af9b5184a51a3c0fe3221c01c75 at {username='slave'} at 127.9.151.130:56721: session id = 4c8a3af9b5184a51a3c0fe3221c01c75-375647a877014ca48ca32ff220098cd1
W20240503 02:14:11.240608 11463 tablet_copy_service.cc:478] P 7cd9a8da65604af78132e9045dac5180: Error handling TabletCopyService RPC request from {username='slave'} at 127.9.151.130:56721: Error beginning tablet copy session for tablet 375647a877014ca48ca32ff220098cd1: Illegal state: The tablet is not in a running state: BOOTSTRAPPING
I20240503 02:14:11.244071 11160 ts_tablet_manager.cc:902] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: Initiating tablet copy from peer 7cd9a8da65604af78132e9045dac5180 (127.9.151.129:33077)
I20240503 02:14:11.251621 11160 tablet_copy_client.cc:308] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: tablet copy: Beginning tablet copy session from remote peer at address 127.9.151.129:33077
I20240503 02:14:11.252465 11463 tablet_copy_service.cc:140] P 7cd9a8da65604af78132e9045dac5180: Received BeginTabletCopySession request for tablet 375647a877014ca48ca32ff220098cd1 from peer 4c8a3af9b5184a51a3c0fe3221c01c75 ({username='slave'} at 127.9.151.130:56721)
I20240503 02:14:11.252668 11463 tablet_copy_service.cc:161] P 7cd9a8da65604af78132e9045dac5180: Beginning new tablet copy session on tablet 375647a877014ca48ca32ff220098cd1 from peer 4c8a3af9b5184a51a3c0fe3221c01c75 at {username='slave'} at 127.9.151.130:56721: session id = 4c8a3af9b5184a51a3c0fe3221c01c75-375647a877014ca48ca32ff220098cd1
W20240503 02:14:11.252912 11463 tablet_copy_service.cc:478] P 7cd9a8da65604af78132e9045dac5180: Error handling TabletCopyService RPC request from {username='slave'} at 127.9.151.130:56721: Error beginning tablet copy session for tablet 375647a877014ca48ca32ff220098cd1: Illegal state: The tablet is not in a running state: BOOTSTRAPPING
I20240503 02:14:11.270375 11160 ts_tablet_manager.cc:902] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: Initiating tablet copy from peer 7cd9a8da65604af78132e9045dac5180 (127.9.151.129:33077)
I20240503 02:14:11.272318 11160 tablet_copy_client.cc:308] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: tablet copy: Beginning tablet copy session from remote peer at address 127.9.151.129:33077
I20240503 02:14:11.273078 11463 tablet_copy_service.cc:140] P 7cd9a8da65604af78132e9045dac5180: Received BeginTabletCopySession request for tablet 375647a877014ca48ca32ff220098cd1 from peer 4c8a3af9b5184a51a3c0fe3221c01c75 ({username='slave'} at 127.9.151.130:56721)
I20240503 02:14:11.273278 11463 tablet_copy_service.cc:161] P 7cd9a8da65604af78132e9045dac5180: Beginning new tablet copy session on tablet 375647a877014ca48ca32ff220098cd1 from peer 4c8a3af9b5184a51a3c0fe3221c01c75 at {username='slave'} at 127.9.151.130:56721: session id = 4c8a3af9b5184a51a3c0fe3221c01c75-375647a877014ca48ca32ff220098cd1
W20240503 02:14:11.273452 11463 tablet_copy_service.cc:478] P 7cd9a8da65604af78132e9045dac5180: Error handling TabletCopyService RPC request from {username='slave'} at 127.9.151.130:56721: Error beginning tablet copy session for tablet 375647a877014ca48ca32ff220098cd1: Illegal state: The tablet is not in a running state: BOOTSTRAPPING
I20240503 02:14:11.278247 11160 ts_tablet_manager.cc:902] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: Initiating tablet copy from peer 7cd9a8da65604af78132e9045dac5180 (127.9.151.129:33077)
I20240503 02:14:11.279697 11160 tablet_copy_client.cc:308] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: tablet copy: Beginning tablet copy session from remote peer at address 127.9.151.129:33077
I20240503 02:14:11.280738 11463 tablet_copy_service.cc:140] P 7cd9a8da65604af78132e9045dac5180: Received BeginTabletCopySession request for tablet 375647a877014ca48ca32ff220098cd1 from peer 4c8a3af9b5184a51a3c0fe3221c01c75 ({username='slave'} at 127.9.151.130:56721)
I20240503 02:14:11.280992 11463 tablet_copy_service.cc:161] P 7cd9a8da65604af78132e9045dac5180: Beginning new tablet copy session on tablet 375647a877014ca48ca32ff220098cd1 from peer 4c8a3af9b5184a51a3c0fe3221c01c75 at {username='slave'} at 127.9.151.130:56721: session id = 4c8a3af9b5184a51a3c0fe3221c01c75-375647a877014ca48ca32ff220098cd1
W20240503 02:14:11.281208 11463 tablet_copy_service.cc:478] P 7cd9a8da65604af78132e9045dac5180: Error handling TabletCopyService RPC request from {username='slave'} at 127.9.151.130:56721: Error beginning tablet copy session for tablet 375647a877014ca48ca32ff220098cd1: Illegal state: The tablet is not in a running state: BOOTSTRAPPING
I20240503 02:14:11.288336 11160 ts_tablet_manager.cc:902] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: Initiating tablet copy from peer 7cd9a8da65604af78132e9045dac5180 (127.9.151.129:33077)
I20240503 02:14:11.289669 11160 tablet_copy_client.cc:308] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: tablet copy: Beginning tablet copy session from remote peer at address 127.9.151.129:33077
I20240503 02:14:11.290511 11463 tablet_copy_service.cc:140] P 7cd9a8da65604af78132e9045dac5180: Received BeginTabletCopySession request for tablet 375647a877014ca48ca32ff220098cd1 from peer 4c8a3af9b5184a51a3c0fe3221c01c75 ({username='slave'} at 127.9.151.130:56721)
I20240503 02:14:11.290769 11463 tablet_copy_service.cc:161] P 7cd9a8da65604af78132e9045dac5180: Beginning new tablet copy session on tablet 375647a877014ca48ca32ff220098cd1 from peer 4c8a3af9b5184a51a3c0fe3221c01c75 at {username='slave'} at 127.9.151.130:56721: session id = 4c8a3af9b5184a51a3c0fe3221c01c75-375647a877014ca48ca32ff220098cd1
W20240503 02:14:11.291003 11463 tablet_copy_service.cc:478] P 7cd9a8da65604af78132e9045dac5180: Error handling TabletCopyService RPC request from {username='slave'} at 127.9.151.130:56721: Error beginning tablet copy session for tablet 375647a877014ca48ca32ff220098cd1: Illegal state: The tablet is not in a running state: BOOTSTRAPPING
I20240503 02:14:11.302815 11160 ts_tablet_manager.cc:902] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: Initiating tablet copy from peer 7cd9a8da65604af78132e9045dac5180 (127.9.151.129:33077)
I20240503 02:14:11.304170 11160 tablet_copy_client.cc:308] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: tablet copy: Beginning tablet copy session from remote peer at address 127.9.151.129:33077
I20240503 02:14:11.305025 11463 tablet_copy_service.cc:140] P 7cd9a8da65604af78132e9045dac5180: Received BeginTabletCopySession request for tablet 375647a877014ca48ca32ff220098cd1 from peer 4c8a3af9b5184a51a3c0fe3221c01c75 ({username='slave'} at 127.9.151.130:56721)
I20240503 02:14:11.305217 11463 tablet_copy_service.cc:161] P 7cd9a8da65604af78132e9045dac5180: Beginning new tablet copy session on tablet 375647a877014ca48ca32ff220098cd1 from peer 4c8a3af9b5184a51a3c0fe3221c01c75 at {username='slave'} at 127.9.151.130:56721: session id = 4c8a3af9b5184a51a3c0fe3221c01c75-375647a877014ca48ca32ff220098cd1
W20240503 02:14:11.305430 11463 tablet_copy_service.cc:478] P 7cd9a8da65604af78132e9045dac5180: Error handling TabletCopyService RPC request from {username='slave'} at 127.9.151.130:56721: Error beginning tablet copy session for tablet 375647a877014ca48ca32ff220098cd1: Illegal state: The tablet is not in a running state: BOOTSTRAPPING
I20240503 02:14:11.307206 11615 ts_tablet_manager.cc:902] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: Initiating tablet copy from peer 7cd9a8da65604af78132e9045dac5180 (127.9.151.129:33077)
I20240503 02:14:11.322939 11615 tablet_copy_client.cc:308] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: tablet copy: Beginning tablet copy session from remote peer at address 127.9.151.129:33077
I20240503 02:14:11.323757 11463 tablet_copy_service.cc:140] P 7cd9a8da65604af78132e9045dac5180: Received BeginTabletCopySession request for tablet 375647a877014ca48ca32ff220098cd1 from peer 4c8a3af9b5184a51a3c0fe3221c01c75 ({username='slave'} at 127.9.151.130:56721)
I20240503 02:14:11.323963 11463 tablet_copy_service.cc:161] P 7cd9a8da65604af78132e9045dac5180: Beginning new tablet copy session on tablet 375647a877014ca48ca32ff220098cd1 from peer 4c8a3af9b5184a51a3c0fe3221c01c75 at {username='slave'} at 127.9.151.130:56721: session id = 4c8a3af9b5184a51a3c0fe3221c01c75-375647a877014ca48ca32ff220098cd1
W20240503 02:14:11.324141 11463 tablet_copy_service.cc:478] P 7cd9a8da65604af78132e9045dac5180: Error handling TabletCopyService RPC request from {username='slave'} at 127.9.151.130:56721: Error beginning tablet copy session for tablet 375647a877014ca48ca32ff220098cd1: Illegal state: The tablet is not in a running state: BOOTSTRAPPING
I20240503 02:14:11.329355 11615 ts_tablet_manager.cc:902] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: Initiating tablet copy from peer 7cd9a8da65604af78132e9045dac5180 (127.9.151.129:33077)
I20240503 02:14:11.334837 11615 tablet_copy_client.cc:308] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: tablet copy: Beginning tablet copy session from remote peer at address 127.9.151.129:33077
I20240503 02:14:11.335713 11463 tablet_copy_service.cc:140] P 7cd9a8da65604af78132e9045dac5180: Received BeginTabletCopySession request for tablet 375647a877014ca48ca32ff220098cd1 from peer 4c8a3af9b5184a51a3c0fe3221c01c75 ({username='slave'} at 127.9.151.130:56721)
I20240503 02:14:11.335886 11463 tablet_copy_service.cc:161] P 7cd9a8da65604af78132e9045dac5180: Beginning new tablet copy session on tablet 375647a877014ca48ca32ff220098cd1 from peer 4c8a3af9b5184a51a3c0fe3221c01c75 at {username='slave'} at 127.9.151.130:56721: session id = 4c8a3af9b5184a51a3c0fe3221c01c75-375647a877014ca48ca32ff220098cd1
W20240503 02:14:11.336082 11463 tablet_copy_service.cc:478] P 7cd9a8da65604af78132e9045dac5180: Error handling TabletCopyService RPC request from {username='slave'} at 127.9.151.130:56721: Error beginning tablet copy session for tablet 375647a877014ca48ca32ff220098cd1: Illegal state: The tablet is not in a running state: BOOTSTRAPPING
I20240503 02:14:11.352344 11615 ts_tablet_manager.cc:902] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: Initiating tablet copy from peer 7cd9a8da65604af78132e9045dac5180 (127.9.151.129:33077)
I20240503 02:14:11.354328 11615 tablet_copy_client.cc:308] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: tablet copy: Beginning tablet copy session from remote peer at address 127.9.151.129:33077
I20240503 02:14:11.355286 11463 tablet_copy_service.cc:140] P 7cd9a8da65604af78132e9045dac5180: Received BeginTabletCopySession request for tablet 375647a877014ca48ca32ff220098cd1 from peer 4c8a3af9b5184a51a3c0fe3221c01c75 ({username='slave'} at 127.9.151.130:56721)
I20240503 02:14:11.355515 11463 tablet_copy_service.cc:161] P 7cd9a8da65604af78132e9045dac5180: Beginning new tablet copy session on tablet 375647a877014ca48ca32ff220098cd1 from peer 4c8a3af9b5184a51a3c0fe3221c01c75 at {username='slave'} at 127.9.151.130:56721: session id = 4c8a3af9b5184a51a3c0fe3221c01c75-375647a877014ca48ca32ff220098cd1
W20240503 02:14:11.355690 11463 tablet_copy_service.cc:478] P 7cd9a8da65604af78132e9045dac5180: Error handling TabletCopyService RPC request from {username='slave'} at 127.9.151.130:56721: Error beginning tablet copy session for tablet 375647a877014ca48ca32ff220098cd1: Illegal state: The tablet is not in a running state: BOOTSTRAPPING
I20240503 02:14:11.365315 11615 ts_tablet_manager.cc:902] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: Initiating tablet copy from peer 7cd9a8da65604af78132e9045dac5180 (127.9.151.129:33077)
I20240503 02:14:11.366649 11615 tablet_copy_client.cc:308] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: tablet copy: Beginning tablet copy session from remote peer at address 127.9.151.129:33077
I20240503 02:14:11.367511 11463 tablet_copy_service.cc:140] P 7cd9a8da65604af78132e9045dac5180: Received BeginTabletCopySession request for tablet 375647a877014ca48ca32ff220098cd1 from peer 4c8a3af9b5184a51a3c0fe3221c01c75 ({username='slave'} at 127.9.151.130:56721)
I20240503 02:14:11.367700 11463 tablet_copy_service.cc:161] P 7cd9a8da65604af78132e9045dac5180: Beginning new tablet copy session on tablet 375647a877014ca48ca32ff220098cd1 from peer 4c8a3af9b5184a51a3c0fe3221c01c75 at {username='slave'} at 127.9.151.130:56721: session id = 4c8a3af9b5184a51a3c0fe3221c01c75-375647a877014ca48ca32ff220098cd1
W20240503 02:14:11.367928 11463 tablet_copy_service.cc:478] P 7cd9a8da65604af78132e9045dac5180: Error handling TabletCopyService RPC request from {username='slave'} at 127.9.151.130:56721: Error beginning tablet copy session for tablet 375647a877014ca48ca32ff220098cd1: Illegal state: The tablet is not in a running state: BOOTSTRAPPING
I20240503 02:14:11.371402 11615 ts_tablet_manager.cc:902] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: Initiating tablet copy from peer 7cd9a8da65604af78132e9045dac5180 (127.9.151.129:33077)
I20240503 02:14:11.376850 11615 tablet_copy_client.cc:308] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: tablet copy: Beginning tablet copy session from remote peer at address 127.9.151.129:33077
I20240503 02:14:11.377621 11463 tablet_copy_service.cc:140] P 7cd9a8da65604af78132e9045dac5180: Received BeginTabletCopySession request for tablet 375647a877014ca48ca32ff220098cd1 from peer 4c8a3af9b5184a51a3c0fe3221c01c75 ({username='slave'} at 127.9.151.130:56721)
I20240503 02:14:11.377838 11463 tablet_copy_service.cc:161] P 7cd9a8da65604af78132e9045dac5180: Beginning new tablet copy session on tablet 375647a877014ca48ca32ff220098cd1 from peer 4c8a3af9b5184a51a3c0fe3221c01c75 at {username='slave'} at 127.9.151.130:56721: session id = 4c8a3af9b5184a51a3c0fe3221c01c75-375647a877014ca48ca32ff220098cd1
W20240503 02:14:11.378041 11463 tablet_copy_service.cc:478] P 7cd9a8da65604af78132e9045dac5180: Error handling TabletCopyService RPC request from {username='slave'} at 127.9.151.130:56721: Error beginning tablet copy session for tablet 375647a877014ca48ca32ff220098cd1: Illegal state: The tablet is not in a running state: BOOTSTRAPPING
I20240503 02:14:11.379796 11615 ts_tablet_manager.cc:902] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: Initiating tablet copy from peer 7cd9a8da65604af78132e9045dac5180 (127.9.151.129:33077)
I20240503 02:14:11.380893 11615 tablet_copy_client.cc:308] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: tablet copy: Beginning tablet copy session from remote peer at address 127.9.151.129:33077
I20240503 02:14:11.381659 11463 tablet_copy_service.cc:140] P 7cd9a8da65604af78132e9045dac5180: Received BeginTabletCopySession request for tablet 375647a877014ca48ca32ff220098cd1 from peer 4c8a3af9b5184a51a3c0fe3221c01c75 ({username='slave'} at 127.9.151.130:56721)
I20240503 02:14:11.381862 11463 tablet_copy_service.cc:161] P 7cd9a8da65604af78132e9045dac5180: Beginning new tablet copy session on tablet 375647a877014ca48ca32ff220098cd1 from peer 4c8a3af9b5184a51a3c0fe3221c01c75 at {username='slave'} at 127.9.151.130:56721: session id = 4c8a3af9b5184a51a3c0fe3221c01c75-375647a877014ca48ca32ff220098cd1
W20240503 02:14:11.382084 11463 tablet_copy_service.cc:478] P 7cd9a8da65604af78132e9045dac5180: Error handling TabletCopyService RPC request from {username='slave'} at 127.9.151.130:56721: Error beginning tablet copy session for tablet 375647a877014ca48ca32ff220098cd1: Illegal state: The tablet is not in a running state: BOOTSTRAPPING
I20240503 02:14:11.392786 11615 ts_tablet_manager.cc:902] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: Initiating tablet copy from peer 7cd9a8da65604af78132e9045dac5180 (127.9.151.129:33077)
I20240503 02:14:11.394063 11615 tablet_copy_client.cc:308] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: tablet copy: Beginning tablet copy session from remote peer at address 127.9.151.129:33077
I20240503 02:14:11.394867 11463 tablet_copy_service.cc:140] P 7cd9a8da65604af78132e9045dac5180: Received BeginTabletCopySession request for tablet 375647a877014ca48ca32ff220098cd1 from peer 4c8a3af9b5184a51a3c0fe3221c01c75 ({username='slave'} at 127.9.151.130:56721)
I20240503 02:14:11.395097 11463 tablet_copy_service.cc:161] P 7cd9a8da65604af78132e9045dac5180: Beginning new tablet copy session on tablet 375647a877014ca48ca32ff220098cd1 from peer 4c8a3af9b5184a51a3c0fe3221c01c75 at {username='slave'} at 127.9.151.130:56721: session id = 4c8a3af9b5184a51a3c0fe3221c01c75-375647a877014ca48ca32ff220098cd1
W20240503 02:14:11.395291 11463 tablet_copy_service.cc:478] P 7cd9a8da65604af78132e9045dac5180: Error handling TabletCopyService RPC request from {username='slave'} at 127.9.151.130:56721: Error beginning tablet copy session for tablet 375647a877014ca48ca32ff220098cd1: Illegal state: The tablet is not in a running state: BOOTSTRAPPING
I20240503 02:14:11.406530 11615 ts_tablet_manager.cc:902] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: Initiating tablet copy from peer 7cd9a8da65604af78132e9045dac5180 (127.9.151.129:33077)
I20240503 02:14:11.408066 11615 tablet_copy_client.cc:308] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: tablet copy: Beginning tablet copy session from remote peer at address 127.9.151.129:33077
I20240503 02:14:11.408901 11463 tablet_copy_service.cc:140] P 7cd9a8da65604af78132e9045dac5180: Received BeginTabletCopySession request for tablet 375647a877014ca48ca32ff220098cd1 from peer 4c8a3af9b5184a51a3c0fe3221c01c75 ({username='slave'} at 127.9.151.130:56721)
I20240503 02:14:11.409101 11463 tablet_copy_service.cc:161] P 7cd9a8da65604af78132e9045dac5180: Beginning new tablet copy session on tablet 375647a877014ca48ca32ff220098cd1 from peer 4c8a3af9b5184a51a3c0fe3221c01c75 at {username='slave'} at 127.9.151.130:56721: session id = 4c8a3af9b5184a51a3c0fe3221c01c75-375647a877014ca48ca32ff220098cd1
W20240503 02:14:11.409369 11463 tablet_copy_service.cc:478] P 7cd9a8da65604af78132e9045dac5180: Error handling TabletCopyService RPC request from {username='slave'} at 127.9.151.130:56721: Error beginning tablet copy session for tablet 375647a877014ca48ca32ff220098cd1: Illegal state: The tablet is not in a running state: BOOTSTRAPPING
I20240503 02:14:11.416038 11615 ts_tablet_manager.cc:902] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: Initiating tablet copy from peer 7cd9a8da65604af78132e9045dac5180 (127.9.151.129:33077)
I20240503 02:14:11.417577 11615 tablet_copy_client.cc:308] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: tablet copy: Beginning tablet copy session from remote peer at address 127.9.151.129:33077
I20240503 02:14:11.418576 11463 tablet_copy_service.cc:140] P 7cd9a8da65604af78132e9045dac5180: Received BeginTabletCopySession request for tablet 375647a877014ca48ca32ff220098cd1 from peer 4c8a3af9b5184a51a3c0fe3221c01c75 ({username='slave'} at 127.9.151.130:56721)
I20240503 02:14:11.418978 11463 tablet_copy_service.cc:161] P 7cd9a8da65604af78132e9045dac5180: Beginning new tablet copy session on tablet 375647a877014ca48ca32ff220098cd1 from peer 4c8a3af9b5184a51a3c0fe3221c01c75 at {username='slave'} at 127.9.151.130:56721: session id = 4c8a3af9b5184a51a3c0fe3221c01c75-375647a877014ca48ca32ff220098cd1
W20240503 02:14:11.419281 11463 tablet_copy_service.cc:478] P 7cd9a8da65604af78132e9045dac5180: Error handling TabletCopyService RPC request from {username='slave'} at 127.9.151.130:56721: Error beginning tablet copy session for tablet 375647a877014ca48ca32ff220098cd1: Illegal state: The tablet is not in a running state: BOOTSTRAPPING
I20240503 02:14:11.427929 11615 ts_tablet_manager.cc:902] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: Initiating tablet copy from peer 7cd9a8da65604af78132e9045dac5180 (127.9.151.129:33077)
I20240503 02:14:11.429260 11615 tablet_copy_client.cc:308] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: tablet copy: Beginning tablet copy session from remote peer at address 127.9.151.129:33077
I20240503 02:14:11.430127 11463 tablet_copy_service.cc:140] P 7cd9a8da65604af78132e9045dac5180: Received BeginTabletCopySession request for tablet 375647a877014ca48ca32ff220098cd1 from peer 4c8a3af9b5184a51a3c0fe3221c01c75 ({username='slave'} at 127.9.151.130:56721)
I20240503 02:14:11.430301 11463 tablet_copy_service.cc:161] P 7cd9a8da65604af78132e9045dac5180: Beginning new tablet copy session on tablet 375647a877014ca48ca32ff220098cd1 from peer 4c8a3af9b5184a51a3c0fe3221c01c75 at {username='slave'} at 127.9.151.130:56721: session id = 4c8a3af9b5184a51a3c0fe3221c01c75-375647a877014ca48ca32ff220098cd1
W20240503 02:14:11.430496 11463 tablet_copy_service.cc:478] P 7cd9a8da65604af78132e9045dac5180: Error handling TabletCopyService RPC request from {username='slave'} at 127.9.151.130:56721: Error beginning tablet copy session for tablet 375647a877014ca48ca32ff220098cd1: Illegal state: The tablet is not in a running state: BOOTSTRAPPING
I20240503 02:14:11.443032 11615 ts_tablet_manager.cc:902] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: Initiating tablet copy from peer 7cd9a8da65604af78132e9045dac5180 (127.9.151.129:33077)
I20240503 02:14:11.444772 11615 tablet_copy_client.cc:308] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: tablet copy: Beginning tablet copy session from remote peer at address 127.9.151.129:33077
I20240503 02:14:11.445689 11463 tablet_copy_service.cc:140] P 7cd9a8da65604af78132e9045dac5180: Received BeginTabletCopySession request for tablet 375647a877014ca48ca32ff220098cd1 from peer 4c8a3af9b5184a51a3c0fe3221c01c75 ({username='slave'} at 127.9.151.130:56721)
I20240503 02:14:11.445921 11463 tablet_copy_service.cc:161] P 7cd9a8da65604af78132e9045dac5180: Beginning new tablet copy session on tablet 375647a877014ca48ca32ff220098cd1 from peer 4c8a3af9b5184a51a3c0fe3221c01c75 at {username='slave'} at 127.9.151.130:56721: session id = 4c8a3af9b5184a51a3c0fe3221c01c75-375647a877014ca48ca32ff220098cd1
W20240503 02:14:11.446111 11463 tablet_copy_service.cc:478] P 7cd9a8da65604af78132e9045dac5180: Error handling TabletCopyService RPC request from {username='slave'} at 127.9.151.130:56721: Error beginning tablet copy session for tablet 375647a877014ca48ca32ff220098cd1: Illegal state: The tablet is not in a running state: BOOTSTRAPPING
I20240503 02:14:11.452756 11615 ts_tablet_manager.cc:902] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: Initiating tablet copy from peer 7cd9a8da65604af78132e9045dac5180 (127.9.151.129:33077)
I20240503 02:14:11.454198 11615 tablet_copy_client.cc:308] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: tablet copy: Beginning tablet copy session from remote peer at address 127.9.151.129:33077
I20240503 02:14:11.455118 11463 tablet_copy_service.cc:140] P 7cd9a8da65604af78132e9045dac5180: Received BeginTabletCopySession request for tablet 375647a877014ca48ca32ff220098cd1 from peer 4c8a3af9b5184a51a3c0fe3221c01c75 ({username='slave'} at 127.9.151.130:56721)
I20240503 02:14:11.455292 11463 tablet_copy_service.cc:161] P 7cd9a8da65604af78132e9045dac5180: Beginning new tablet copy session on tablet 375647a877014ca48ca32ff220098cd1 from peer 4c8a3af9b5184a51a3c0fe3221c01c75 at {username='slave'} at 127.9.151.130:56721: session id = 4c8a3af9b5184a51a3c0fe3221c01c75-375647a877014ca48ca32ff220098cd1
W20240503 02:14:11.455511 11463 tablet_copy_service.cc:478] P 7cd9a8da65604af78132e9045dac5180: Error handling TabletCopyService RPC request from {username='slave'} at 127.9.151.130:56721: Error beginning tablet copy session for tablet 375647a877014ca48ca32ff220098cd1: Illegal state: The tablet is not in a running state: BOOTSTRAPPING
I20240503 02:14:11.458472 11615 ts_tablet_manager.cc:902] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: Initiating tablet copy from peer 7cd9a8da65604af78132e9045dac5180 (127.9.151.129:33077)
I20240503 02:14:11.459632 11615 tablet_copy_client.cc:308] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: tablet copy: Beginning tablet copy session from remote peer at address 127.9.151.129:33077
I20240503 02:14:11.460434 11463 tablet_copy_service.cc:140] P 7cd9a8da65604af78132e9045dac5180: Received BeginTabletCopySession request for tablet 375647a877014ca48ca32ff220098cd1 from peer 4c8a3af9b5184a51a3c0fe3221c01c75 ({username='slave'} at 127.9.151.130:56721)
I20240503 02:14:11.460633 11463 tablet_copy_service.cc:161] P 7cd9a8da65604af78132e9045dac5180: Beginning new tablet copy session on tablet 375647a877014ca48ca32ff220098cd1 from peer 4c8a3af9b5184a51a3c0fe3221c01c75 at {username='slave'} at 127.9.151.130:56721: session id = 4c8a3af9b5184a51a3c0fe3221c01c75-375647a877014ca48ca32ff220098cd1
W20240503 02:14:11.460803 11463 tablet_copy_service.cc:478] P 7cd9a8da65604af78132e9045dac5180: Error handling TabletCopyService RPC request from {username='slave'} at 127.9.151.130:56721: Error beginning tablet copy session for tablet 375647a877014ca48ca32ff220098cd1: Illegal state: The tablet is not in a running state: BOOTSTRAPPING
I20240503 02:14:11.463129 11615 ts_tablet_manager.cc:902] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: Initiating tablet copy from peer 7cd9a8da65604af78132e9045dac5180 (127.9.151.129:33077)
I20240503 02:14:11.464191 11615 tablet_copy_client.cc:308] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: tablet copy: Beginning tablet copy session from remote peer at address 127.9.151.129:33077
I20240503 02:14:11.464979 11463 tablet_copy_service.cc:140] P 7cd9a8da65604af78132e9045dac5180: Received BeginTabletCopySession request for tablet 375647a877014ca48ca32ff220098cd1 from peer 4c8a3af9b5184a51a3c0fe3221c01c75 ({username='slave'} at 127.9.151.130:56721)
I20240503 02:14:11.465160 11463 tablet_copy_service.cc:161] P 7cd9a8da65604af78132e9045dac5180: Beginning new tablet copy session on tablet 375647a877014ca48ca32ff220098cd1 from peer 4c8a3af9b5184a51a3c0fe3221c01c75 at {username='slave'} at 127.9.151.130:56721: session id = 4c8a3af9b5184a51a3c0fe3221c01c75-375647a877014ca48ca32ff220098cd1
W20240503 02:14:11.465344 11463 tablet_copy_service.cc:478] P 7cd9a8da65604af78132e9045dac5180: Error handling TabletCopyService RPC request from {username='slave'} at 127.9.151.130:56721: Error beginning tablet copy session for tablet 375647a877014ca48ca32ff220098cd1: Illegal state: The tablet is not in a running state: BOOTSTRAPPING
I20240503 02:14:11.467739 11615 ts_tablet_manager.cc:902] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: Initiating tablet copy from peer 7cd9a8da65604af78132e9045dac5180 (127.9.151.129:33077)
I20240503 02:14:11.469095 11615 tablet_copy_client.cc:308] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: tablet copy: Beginning tablet copy session from remote peer at address 127.9.151.129:33077
I20240503 02:14:11.469954 11463 tablet_copy_service.cc:140] P 7cd9a8da65604af78132e9045dac5180: Received BeginTabletCopySession request for tablet 375647a877014ca48ca32ff220098cd1 from peer 4c8a3af9b5184a51a3c0fe3221c01c75 ({username='slave'} at 127.9.151.130:56721)
I20240503 02:14:11.470206 11463 tablet_copy_service.cc:161] P 7cd9a8da65604af78132e9045dac5180: Beginning new tablet copy session on tablet 375647a877014ca48ca32ff220098cd1 from peer 4c8a3af9b5184a51a3c0fe3221c01c75 at {username='slave'} at 127.9.151.130:56721: session id = 4c8a3af9b5184a51a3c0fe3221c01c75-375647a877014ca48ca32ff220098cd1
W20240503 02:14:11.470415 11463 tablet_copy_service.cc:478] P 7cd9a8da65604af78132e9045dac5180: Error handling TabletCopyService RPC request from {username='slave'} at 127.9.151.130:56721: Error beginning tablet copy session for tablet 375647a877014ca48ca32ff220098cd1: Illegal state: The tablet is not in a running state: BOOTSTRAPPING
I20240503 02:14:11.487177 11615 ts_tablet_manager.cc:902] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: Initiating tablet copy from peer 7cd9a8da65604af78132e9045dac5180 (127.9.151.129:33077)
I20240503 02:14:11.488603 11615 tablet_copy_client.cc:308] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: tablet copy: Beginning tablet copy session from remote peer at address 127.9.151.129:33077
I20240503 02:14:11.489729 11463 tablet_copy_service.cc:140] P 7cd9a8da65604af78132e9045dac5180: Received BeginTabletCopySession request for tablet 375647a877014ca48ca32ff220098cd1 from peer 4c8a3af9b5184a51a3c0fe3221c01c75 ({username='slave'} at 127.9.151.130:56721)
I20240503 02:14:11.489948 11463 tablet_copy_service.cc:161] P 7cd9a8da65604af78132e9045dac5180: Beginning new tablet copy session on tablet 375647a877014ca48ca32ff220098cd1 from peer 4c8a3af9b5184a51a3c0fe3221c01c75 at {username='slave'} at 127.9.151.130:56721: session id = 4c8a3af9b5184a51a3c0fe3221c01c75-375647a877014ca48ca32ff220098cd1
W20240503 02:14:11.490124 11463 tablet_copy_service.cc:478] P 7cd9a8da65604af78132e9045dac5180: Error handling TabletCopyService RPC request from {username='slave'} at 127.9.151.130:56721: Error beginning tablet copy session for tablet 375647a877014ca48ca32ff220098cd1: Illegal state: The tablet is not in a running state: BOOTSTRAPPING
I20240503 02:14:11.496382 11615 ts_tablet_manager.cc:902] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: Initiating tablet copy from peer 7cd9a8da65604af78132e9045dac5180 (127.9.151.129:33077)
I20240503 02:14:11.503083 11615 tablet_copy_client.cc:308] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: tablet copy: Beginning tablet copy session from remote peer at address 127.9.151.129:33077
I20240503 02:14:11.504101 11463 tablet_copy_service.cc:140] P 7cd9a8da65604af78132e9045dac5180: Received BeginTabletCopySession request for tablet 375647a877014ca48ca32ff220098cd1 from peer 4c8a3af9b5184a51a3c0fe3221c01c75 ({username='slave'} at 127.9.151.130:56721)
I20240503 02:14:11.504300 11463 tablet_copy_service.cc:161] P 7cd9a8da65604af78132e9045dac5180: Beginning new tablet copy session on tablet 375647a877014ca48ca32ff220098cd1 from peer 4c8a3af9b5184a51a3c0fe3221c01c75 at {username='slave'} at 127.9.151.130:56721: session id = 4c8a3af9b5184a51a3c0fe3221c01c75-375647a877014ca48ca32ff220098cd1
W20240503 02:14:11.504523 11463 tablet_copy_service.cc:478] P 7cd9a8da65604af78132e9045dac5180: Error handling TabletCopyService RPC request from {username='slave'} at 127.9.151.130:56721: Error beginning tablet copy session for tablet 375647a877014ca48ca32ff220098cd1: Illegal state: The tablet is not in a running state: BOOTSTRAPPING
I20240503 02:14:11.513131 11615 ts_tablet_manager.cc:902] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: Initiating tablet copy from peer 7cd9a8da65604af78132e9045dac5180 (127.9.151.129:33077)
I20240503 02:14:11.515163 11615 tablet_copy_client.cc:308] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: tablet copy: Beginning tablet copy session from remote peer at address 127.9.151.129:33077
I20240503 02:14:11.515995 11463 tablet_copy_service.cc:140] P 7cd9a8da65604af78132e9045dac5180: Received BeginTabletCopySession request for tablet 375647a877014ca48ca32ff220098cd1 from peer 4c8a3af9b5184a51a3c0fe3221c01c75 ({username='slave'} at 127.9.151.130:56721)
I20240503 02:14:11.516196 11463 tablet_copy_service.cc:161] P 7cd9a8da65604af78132e9045dac5180: Beginning new tablet copy session on tablet 375647a877014ca48ca32ff220098cd1 from peer 4c8a3af9b5184a51a3c0fe3221c01c75 at {username='slave'} at 127.9.151.130:56721: session id = 4c8a3af9b5184a51a3c0fe3221c01c75-375647a877014ca48ca32ff220098cd1
W20240503 02:14:11.516414 11463 tablet_copy_service.cc:478] P 7cd9a8da65604af78132e9045dac5180: Error handling TabletCopyService RPC request from {username='slave'} at 127.9.151.130:56721: Error beginning tablet copy session for tablet 375647a877014ca48ca32ff220098cd1: Illegal state: The tablet is not in a running state: BOOTSTRAPPING
I20240503 02:14:11.521847 11615 ts_tablet_manager.cc:902] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: Initiating tablet copy from peer 7cd9a8da65604af78132e9045dac5180 (127.9.151.129:33077)
I20240503 02:14:11.523591 11615 tablet_copy_client.cc:308] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: tablet copy: Beginning tablet copy session from remote peer at address 127.9.151.129:33077
I20240503 02:14:11.524683 11463 tablet_copy_service.cc:140] P 7cd9a8da65604af78132e9045dac5180: Received BeginTabletCopySession request for tablet 375647a877014ca48ca32ff220098cd1 from peer 4c8a3af9b5184a51a3c0fe3221c01c75 ({username='slave'} at 127.9.151.130:56721)
I20240503 02:14:11.524952 11463 tablet_copy_service.cc:161] P 7cd9a8da65604af78132e9045dac5180: Beginning new tablet copy session on tablet 375647a877014ca48ca32ff220098cd1 from peer 4c8a3af9b5184a51a3c0fe3221c01c75 at {username='slave'} at 127.9.151.130:56721: session id = 4c8a3af9b5184a51a3c0fe3221c01c75-375647a877014ca48ca32ff220098cd1
W20240503 02:14:11.525171 11463 tablet_copy_service.cc:478] P 7cd9a8da65604af78132e9045dac5180: Error handling TabletCopyService RPC request from {username='slave'} at 127.9.151.130:56721: Error beginning tablet copy session for tablet 375647a877014ca48ca32ff220098cd1: Illegal state: The tablet is not in a running state: BOOTSTRAPPING
I20240503 02:14:11.530582 11615 ts_tablet_manager.cc:902] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: Initiating tablet copy from peer 7cd9a8da65604af78132e9045dac5180 (127.9.151.129:33077)
I20240503 02:14:11.532059 11615 tablet_copy_client.cc:308] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: tablet copy: Beginning tablet copy session from remote peer at address 127.9.151.129:33077
I20240503 02:14:11.532954 11463 tablet_copy_service.cc:140] P 7cd9a8da65604af78132e9045dac5180: Received BeginTabletCopySession request for tablet 375647a877014ca48ca32ff220098cd1 from peer 4c8a3af9b5184a51a3c0fe3221c01c75 ({username='slave'} at 127.9.151.130:56721)
I20240503 02:14:11.533154 11463 tablet_copy_service.cc:161] P 7cd9a8da65604af78132e9045dac5180: Beginning new tablet copy session on tablet 375647a877014ca48ca32ff220098cd1 from peer 4c8a3af9b5184a51a3c0fe3221c01c75 at {username='slave'} at 127.9.151.130:56721: session id = 4c8a3af9b5184a51a3c0fe3221c01c75-375647a877014ca48ca32ff220098cd1
W20240503 02:14:11.533368 11463 tablet_copy_service.cc:478] P 7cd9a8da65604af78132e9045dac5180: Error handling TabletCopyService RPC request from {username='slave'} at 127.9.151.130:56721: Error beginning tablet copy session for tablet 375647a877014ca48ca32ff220098cd1: Illegal state: The tablet is not in a running state: BOOTSTRAPPING
I20240503 02:14:11.536702 11615 ts_tablet_manager.cc:902] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: Initiating tablet copy from peer 7cd9a8da65604af78132e9045dac5180 (127.9.151.129:33077)
I20240503 02:14:11.537922 11615 tablet_copy_client.cc:308] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: tablet copy: Beginning tablet copy session from remote peer at address 127.9.151.129:33077
I20240503 02:14:11.538604 11463 tablet_copy_service.cc:140] P 7cd9a8da65604af78132e9045dac5180: Received BeginTabletCopySession request for tablet 375647a877014ca48ca32ff220098cd1 from peer 4c8a3af9b5184a51a3c0fe3221c01c75 ({username='slave'} at 127.9.151.130:56721)
I20240503 02:14:11.538846 11463 tablet_copy_service.cc:161] P 7cd9a8da65604af78132e9045dac5180: Beginning new tablet copy session on tablet 375647a877014ca48ca32ff220098cd1 from peer 4c8a3af9b5184a51a3c0fe3221c01c75 at {username='slave'} at 127.9.151.130:56721: session id = 4c8a3af9b5184a51a3c0fe3221c01c75-375647a877014ca48ca32ff220098cd1
W20240503 02:14:11.539055 11463 tablet_copy_service.cc:478] P 7cd9a8da65604af78132e9045dac5180: Error handling TabletCopyService RPC request from {username='slave'} at 127.9.151.130:56721: Error beginning tablet copy session for tablet 375647a877014ca48ca32ff220098cd1: Illegal state: The tablet is not in a running state: BOOTSTRAPPING
I20240503 02:14:11.541158 11615 ts_tablet_manager.cc:902] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: Initiating tablet copy from peer 7cd9a8da65604af78132e9045dac5180 (127.9.151.129:33077)
I20240503 02:14:11.542634 11615 tablet_copy_client.cc:308] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: tablet copy: Beginning tablet copy session from remote peer at address 127.9.151.129:33077
I20240503 02:14:11.543435 11463 tablet_copy_service.cc:140] P 7cd9a8da65604af78132e9045dac5180: Received BeginTabletCopySession request for tablet 375647a877014ca48ca32ff220098cd1 from peer 4c8a3af9b5184a51a3c0fe3221c01c75 ({username='slave'} at 127.9.151.130:56721)
I20240503 02:14:11.543632 11463 tablet_copy_service.cc:161] P 7cd9a8da65604af78132e9045dac5180: Beginning new tablet copy session on tablet 375647a877014ca48ca32ff220098cd1 from peer 4c8a3af9b5184a51a3c0fe3221c01c75 at {username='slave'} at 127.9.151.130:56721: session id = 4c8a3af9b5184a51a3c0fe3221c01c75-375647a877014ca48ca32ff220098cd1
W20240503 02:14:11.543818 11463 tablet_copy_service.cc:478] P 7cd9a8da65604af78132e9045dac5180: Error handling TabletCopyService RPC request from {username='slave'} at 127.9.151.130:56721: Error beginning tablet copy session for tablet 375647a877014ca48ca32ff220098cd1: Illegal state: The tablet is not in a running state: BOOTSTRAPPING
I20240503 02:14:11.559551 11354 tablet_bootstrap.cc:492] T 375647a877014ca48ca32ff220098cd1 P 7cd9a8da65604af78132e9045dac5180: Bootstrap replayed 1/1 log segments. Stats: ops{read=213 overwritten=0 applied=213 ignored=0} inserts{seen=10550 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240503 02:14:11.560163 11354 tablet_bootstrap.cc:492] T 375647a877014ca48ca32ff220098cd1 P 7cd9a8da65604af78132e9045dac5180: Bootstrap complete.
I20240503 02:14:11.561488 11615 ts_tablet_manager.cc:902] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: Initiating tablet copy from peer 7cd9a8da65604af78132e9045dac5180 (127.9.151.129:33077)
I20240503 02:14:11.561520 11354 ts_tablet_manager.cc:1367] T 375647a877014ca48ca32ff220098cd1 P 7cd9a8da65604af78132e9045dac5180: Time spent bootstrapping tablet: real 0.864s user 0.772s sys 0.079s
I20240503 02:14:11.566901 11354 raft_consensus.cc:348] T 375647a877014ca48ca32ff220098cd1 P 7cd9a8da65604af78132e9045dac5180 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7cd9a8da65604af78132e9045dac5180" member_type: VOTER last_known_addr { host: "127.9.151.129" port: 33077 } }
I20240503 02:14:11.567939 11354 raft_consensus.cc:729] T 375647a877014ca48ca32ff220098cd1 P 7cd9a8da65604af78132e9045dac5180 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7cd9a8da65604af78132e9045dac5180, State: Initialized, Role: FOLLOWER
I20240503 02:14:11.569079 11354 consensus_queue.cc:260] T 375647a877014ca48ca32ff220098cd1 P 7cd9a8da65604af78132e9045dac5180 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 213, Last appended: 2.213, Last appended by leader: 213, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7cd9a8da65604af78132e9045dac5180" member_type: VOTER last_known_addr { host: "127.9.151.129" port: 33077 } }
I20240503 02:14:11.569481 11354 raft_consensus.cc:388] T 375647a877014ca48ca32ff220098cd1 P 7cd9a8da65604af78132e9045dac5180 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20240503 02:14:11.569598 11615 tablet_copy_client.cc:308] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: tablet copy: Beginning tablet copy session from remote peer at address 127.9.151.129:33077
I20240503 02:14:11.569638 11354 raft_consensus.cc:482] T 375647a877014ca48ca32ff220098cd1 P 7cd9a8da65604af78132e9045dac5180 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20240503 02:14:11.569836 11354 raft_consensus.cc:3037] T 375647a877014ca48ca32ff220098cd1 P 7cd9a8da65604af78132e9045dac5180 [term 2 FOLLOWER]: Advancing to term 3
I20240503 02:14:11.570408 11463 tablet_copy_service.cc:140] P 7cd9a8da65604af78132e9045dac5180: Received BeginTabletCopySession request for tablet 375647a877014ca48ca32ff220098cd1 from peer 4c8a3af9b5184a51a3c0fe3221c01c75 ({username='slave'} at 127.9.151.130:56721)
I20240503 02:14:11.570600 11463 tablet_copy_service.cc:161] P 7cd9a8da65604af78132e9045dac5180: Beginning new tablet copy session on tablet 375647a877014ca48ca32ff220098cd1 from peer 4c8a3af9b5184a51a3c0fe3221c01c75 at {username='slave'} at 127.9.151.130:56721: session id = 4c8a3af9b5184a51a3c0fe3221c01c75-375647a877014ca48ca32ff220098cd1
W20240503 02:14:11.570823 11463 tablet_copy_service.cc:478] P 7cd9a8da65604af78132e9045dac5180: Error handling TabletCopyService RPC request from {username='slave'} at 127.9.151.130:56721: Error beginning tablet copy session for tablet 375647a877014ca48ca32ff220098cd1: Illegal state: The tablet is not in a running state: BOOTSTRAPPING
I20240503 02:14:11.573168 11615 ts_tablet_manager.cc:902] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: Initiating tablet copy from peer 7cd9a8da65604af78132e9045dac5180 (127.9.151.129:33077)
I20240503 02:14:11.574417 11615 tablet_copy_client.cc:308] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: tablet copy: Beginning tablet copy session from remote peer at address 127.9.151.129:33077
I20240503 02:14:11.574626 11354 raft_consensus.cc:504] T 375647a877014ca48ca32ff220098cd1 P 7cd9a8da65604af78132e9045dac5180 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7cd9a8da65604af78132e9045dac5180" member_type: VOTER last_known_addr { host: "127.9.151.129" port: 33077 } }
I20240503 02:14:11.575189 11463 tablet_copy_service.cc:140] P 7cd9a8da65604af78132e9045dac5180: Received BeginTabletCopySession request for tablet 375647a877014ca48ca32ff220098cd1 from peer 4c8a3af9b5184a51a3c0fe3221c01c75 ({username='slave'} at 127.9.151.130:56721)
I20240503 02:14:11.575376 11463 tablet_copy_service.cc:161] P 7cd9a8da65604af78132e9045dac5180: Beginning new tablet copy session on tablet 375647a877014ca48ca32ff220098cd1 from peer 4c8a3af9b5184a51a3c0fe3221c01c75 at {username='slave'} at 127.9.151.130:56721: session id = 4c8a3af9b5184a51a3c0fe3221c01c75-375647a877014ca48ca32ff220098cd1
I20240503 02:14:11.575332 11354 leader_election.cc:302] T 375647a877014ca48ca32ff220098cd1 P 7cd9a8da65604af78132e9045dac5180 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7cd9a8da65604af78132e9045dac5180; no voters:
W20240503 02:14:11.575588 11463 tablet_copy_service.cc:478] P 7cd9a8da65604af78132e9045dac5180: Error handling TabletCopyService RPC request from {username='slave'} at 127.9.151.130:56721: Error beginning tablet copy session for tablet 375647a877014ca48ca32ff220098cd1: Illegal state: The tablet is not in a running state: BOOTSTRAPPING
I20240503 02:14:11.576006 11354 leader_election.cc:288] T 375647a877014ca48ca32ff220098cd1 P 7cd9a8da65604af78132e9045dac5180 [CANDIDATE]: Term 3 election: Requested vote from peers
I20240503 02:14:11.576180 11743 raft_consensus.cc:2781] T 375647a877014ca48ca32ff220098cd1 P 7cd9a8da65604af78132e9045dac5180 [term 3 FOLLOWER]: Leader election won for term 3
I20240503 02:14:11.577071 11501 heartbeater.cc:503] Master 127.9.151.190:36085 was elected leader, sending a full tablet report...
I20240503 02:14:11.577698 11354 ts_tablet_manager.cc:1398] T 375647a877014ca48ca32ff220098cd1 P 7cd9a8da65604af78132e9045dac5180: Time spent starting tablet: real 0.016s user 0.014s sys 0.004s
I20240503 02:14:11.577208 11743 raft_consensus.cc:686] T 375647a877014ca48ca32ff220098cd1 P 7cd9a8da65604af78132e9045dac5180 [term 3 LEADER]: Becoming Leader. State: Replica: 7cd9a8da65604af78132e9045dac5180, State: Running, Role: LEADER
I20240503 02:14:11.578632 11615 ts_tablet_manager.cc:902] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: Initiating tablet copy from peer 7cd9a8da65604af78132e9045dac5180 (127.9.151.129:33077)
I20240503 02:14:11.578539 11743 consensus_queue.cc:237] T 375647a877014ca48ca32ff220098cd1 P 7cd9a8da65604af78132e9045dac5180 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 213, Committed index: 213, Last appended: 2.213, Last appended by leader: 213, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7cd9a8da65604af78132e9045dac5180" member_type: VOTER last_known_addr { host: "127.9.151.129" port: 33077 } }
I20240503 02:14:11.584501 10221 catalog_manager.cc:5484] T 375647a877014ca48ca32ff220098cd1 P 7cd9a8da65604af78132e9045dac5180 reported cstate change: term changed from 2 to 3. New cstate: current_term: 3 leader_uuid: "7cd9a8da65604af78132e9045dac5180" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7cd9a8da65604af78132e9045dac5180" member_type: VOTER last_known_addr { host: "127.9.151.129" port: 33077 } health_report { overall_health: HEALTHY } } }
I20240503 02:14:11.591835 11615 tablet_copy_client.cc:308] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: tablet copy: Beginning tablet copy session from remote peer at address 127.9.151.129:33077
I20240503 02:14:11.592689 11463 tablet_copy_service.cc:140] P 7cd9a8da65604af78132e9045dac5180: Received BeginTabletCopySession request for tablet 375647a877014ca48ca32ff220098cd1 from peer 4c8a3af9b5184a51a3c0fe3221c01c75 ({username='slave'} at 127.9.151.130:56721)
I20240503 02:14:11.592902 11463 tablet_copy_service.cc:161] P 7cd9a8da65604af78132e9045dac5180: Beginning new tablet copy session on tablet 375647a877014ca48ca32ff220098cd1 from peer 4c8a3af9b5184a51a3c0fe3221c01c75 at {username='slave'} at 127.9.151.130:56721: session id = 4c8a3af9b5184a51a3c0fe3221c01c75-375647a877014ca48ca32ff220098cd1
I20240503 02:14:11.595577 11463 tablet_copy_source_session.cc:205] T 375647a877014ca48ca32ff220098cd1 P 7cd9a8da65604af78132e9045dac5180: Tablet Copy: opened 0 blocks and 1 log segments
I20240503 02:14:11.600785 11615 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 375647a877014ca48ca32ff220098cd1. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:14:11.612653 11615 tablet_copy_client.cc:783] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: tablet copy: Starting download of 0 data blocks...
I20240503 02:14:11.613019 11615 tablet_copy_client.cc:647] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: tablet copy: Starting download of 1 WAL segments...
I20240503 02:14:11.618438 11615 tablet_copy_client.cc:523] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240503 02:14:11.622072 11615 tablet_bootstrap.cc:492] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: Bootstrap starting.
I20240503 02:14:11.691442 11615 log.cc:824] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: Log is configured to *not* fsync() on all Append() calls
I20240503 02:14:12.321553 11615 tablet_bootstrap.cc:492] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: Bootstrap replayed 1/1 log segments. Stats: ops{read=214 overwritten=0 applied=214 ignored=0} inserts{seen=10550 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240503 02:14:12.322216 11615 tablet_bootstrap.cc:492] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: Bootstrap complete.
I20240503 02:14:12.322803 11615 ts_tablet_manager.cc:1367] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: Time spent bootstrapping tablet: real 0.701s user 0.633s sys 0.070s
I20240503 02:14:12.328465 11615 raft_consensus.cc:348] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75 [term 3 NON_PARTICIPANT]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7cd9a8da65604af78132e9045dac5180" member_type: VOTER last_known_addr { host: "127.9.151.129" port: 33077 } }
I20240503 02:14:12.328861 11615 raft_consensus.cc:729] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75 [term 3 NON_PARTICIPANT]: Becoming Follower/Learner. State: Replica: 4c8a3af9b5184a51a3c0fe3221c01c75, State: Initialized, Role: NON_PARTICIPANT
I20240503 02:14:12.329412 11615 consensus_queue.cc:260] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 214, Last appended: 3.214, Last appended by leader: 214, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7cd9a8da65604af78132e9045dac5180" member_type: VOTER last_known_addr { host: "127.9.151.129" port: 33077 } }
I20240503 02:14:12.330962 11615 ts_tablet_manager.cc:1398] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: Time spent starting tablet: real 0.008s user 0.007s sys 0.001s
I20240503 02:14:12.331961 11463 tablet_copy_service.cc:342] P 7cd9a8da65604af78132e9045dac5180: Request end of tablet copy session 4c8a3af9b5184a51a3c0fe3221c01c75-375647a877014ca48ca32ff220098cd1 received from {username='slave'} at 127.9.151.130:56721
I20240503 02:14:12.332140 11463 tablet_copy_service.cc:433] P 7cd9a8da65604af78132e9045dac5180: ending tablet copy session 4c8a3af9b5184a51a3c0fe3221c01c75-375647a877014ca48ca32ff220098cd1 on tablet 375647a877014ca48ca32ff220098cd1 with peer 4c8a3af9b5184a51a3c0fe3221c01c75
W20240503 02:14:12.358163 11615 ts_tablet_manager.cc:701] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: Tablet Copy: Invalid argument: Leader has replica of tablet 375647a877014ca48ca32ff220098cd1 with term 0, which is lower than last-logged term 3 on local replica. Rejecting tablet copy request
W20240503 02:14:12.363011 11615 ts_tablet_manager.cc:701] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: Tablet Copy: Invalid argument: Leader has replica of tablet 375647a877014ca48ca32ff220098cd1 with term 0, which is lower than last-logged term 3 on local replica. Rejecting tablet copy request
W20240503 02:14:12.369769 11615 ts_tablet_manager.cc:701] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: Tablet Copy: Invalid argument: Leader has replica of tablet 375647a877014ca48ca32ff220098cd1 with term 0, which is lower than last-logged term 3 on local replica. Rejecting tablet copy request
I20240503 02:14:12.373155 11079 tablet_service.cc:1514] Processing DeleteTablet for tablet 375647a877014ca48ca32ff220098cd1 with delete_type TABLET_DATA_DELETED from {username='slave'} at 127.0.0.1:55312
I20240503 02:14:12.374212 11757 tablet_replica.cc:330] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: stopping tablet replica
I20240503 02:14:12.374962 11757 raft_consensus.cc:2229] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75 [term 3 NON_PARTICIPANT]: Raft consensus shutting down.
I20240503 02:14:12.375324 11757 raft_consensus.cc:2258] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75 [term 3 NON_PARTICIPANT]: Raft consensus is shut down!
I20240503 02:14:12.376595 11757 ts_tablet_manager.cc:1875] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:14:12.381997 11757 ts_tablet_manager.cc:1888] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 3.214
I20240503 02:14:12.382193 11757 log.cc:1196] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: Deleting WAL directory at /tmp/dist-test-tasknrxqHk/test-tmp/tablet_copy_client_session-itest.0.TabletCopyClientSessionITest.TestStartTabletCopyWhileSourceBootstrapping.1714702442467385-9822-0/minicluster-data/ts-1/wal/wals/375647a877014ca48ca32ff220098cd1
I20240503 02:14:12.382735 11757 ts_tablet_manager.cc:1909] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: Deleting consensus metadata
I20240503 02:14:12.385129 9822 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasknrxqHk/build/asan/bin/kudu pid 11017
I20240503 02:14:12.386530 11139 generic_service.cc:196] Checking for leaks (request via RPC)
I20240503 02:14:12.500434 9822 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknrxqHk/build/asan/bin/kudu with pid 11017
I20240503 02:14:12.524991 9822 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknrxqHk/build/asan/bin/kudu
/tmp/dist-test-tasknrxqHk/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknrxqHk/test-tmp/tablet_copy_client_session-itest.0.TabletCopyClientSessionITest.TestStartTabletCopyWhileSourceBootstrapping.1714702442467385-9822-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasknrxqHk/test-tmp/tablet_copy_client_session-itest.0.TabletCopyClientSessionITest.TestStartTabletCopyWhileSourceBootstrapping.1714702442467385-9822-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknrxqHk/test-tmp/tablet_copy_client_session-itest.0.TabletCopyClientSessionITest.TestStartTabletCopyWhileSourceBootstrapping.1714702442467385-9822-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasknrxqHk/test-tmp/tablet_copy_client_session-itest.0.TabletCopyClientSessionITest.TestStartTabletCopyWhileSourceBootstrapping.1714702442467385-9822-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.151.130:35753
--local_ip_for_outbound_sockets=127.9.151.130
--tserver_master_addrs=127.9.151.190:36085
--webserver_port=42537
--webserver_interface=127.9.151.130
--builtin_ntp_servers=127.9.151.148:38865
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin with env {}
W20240503 02:14:12.897486 11759 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240503 02:14:12.897866 11759 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240503 02:14:12.898273 11759 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240503 02:14:12.907541 11759 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240503 02:14:12.907847 11759 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.151.130
I20240503 02:14:12.917541 11759 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.151.148:38865
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknrxqHk/test-tmp/tablet_copy_client_session-itest.0.TabletCopyClientSessionITest.TestStartTabletCopyWhileSourceBootstrapping.1714702442467385-9822-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasknrxqHk/test-tmp/tablet_copy_client_session-itest.0.TabletCopyClientSessionITest.TestStartTabletCopyWhileSourceBootstrapping.1714702442467385-9822-0/minicluster-data/ts-1/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.151.130:35753
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknrxqHk/test-tmp/tablet_copy_client_session-itest.0.TabletCopyClientSessionITest.TestStartTabletCopyWhileSourceBootstrapping.1714702442467385-9822-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.151.130
--webserver_port=42537
--tserver_master_addrs=127.9.151.190:36085
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.151.130
--log_dir=/tmp/dist-test-tasknrxqHk/test-tmp/tablet_copy_client_session-itest.0.TabletCopyClientSessionITest.TestStartTabletCopyWhileSourceBootstrapping.1714702442467385-9822-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c39e72a98e124e5a12080bac42105b996594d806
build type FASTDEBUG
built by None at 03 May 2024 01:43:22 UTC on efd1337a2aa3
build id 99
ASAN enabled
I20240503 02:14:12.919077 11759 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240503 02:14:12.921023 11759 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240503 02:14:12.935204 11764 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240503 02:14:12.935485 11765 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240503 02:14:12.936110 11767 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240503 02:14:12.936942 11759 server_base.cc:1034] running on GCE node
I20240503 02:14:12.937749 11759 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240503 02:14:12.939085 11759 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240503 02:14:12.940495 11759 hybrid_clock.cc:648] HybridClock initialized: now 1714702452940375 us; error 82 us; skew 500 ppm
I20240503 02:14:12.940909 11759 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20240503 02:14:12.943795 11759 webserver.cc:460] Webserver started at http://127.9.151.130:42537/ using document root <none> and password file <none>
I20240503 02:14:12.944645 11759 fs_manager.cc:359] Metadata directory not provided
I20240503 02:14:12.944785 11759 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240503 02:14:12.951678 11759 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.003s sys 0.000s
I20240503 02:14:12.955849 11773 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240503 02:14:12.957841 11759 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.005s sys 0.000s
I20240503 02:14:12.958067 11759 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknrxqHk/test-tmp/tablet_copy_client_session-itest.0.TabletCopyClientSessionITest.TestStartTabletCopyWhileSourceBootstrapping.1714702442467385-9822-0/minicluster-data/ts-1/data,/tmp/dist-test-tasknrxqHk/test-tmp/tablet_copy_client_session-itest.0.TabletCopyClientSessionITest.TestStartTabletCopyWhileSourceBootstrapping.1714702442467385-9822-0/minicluster-data/ts-1/wal
uuid: "4c8a3af9b5184a51a3c0fe3221c01c75"
format_stamp: "Formatted at 2024-05-03 02:14:04 on dist-test-slave-twlh"
I20240503 02:14:12.959214 11759 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknrxqHk/test-tmp/tablet_copy_client_session-itest.0.TabletCopyClientSessionITest.TestStartTabletCopyWhileSourceBootstrapping.1714702442467385-9822-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasknrxqHk/test-tmp/tablet_copy_client_session-itest.0.TabletCopyClientSessionITest.TestStartTabletCopyWhileSourceBootstrapping.1714702442467385-9822-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasknrxqHk/test-tmp/tablet_copy_client_session-itest.0.TabletCopyClientSessionITest.TestStartTabletCopyWhileSourceBootstrapping.1714702442467385-9822-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240503 02:14:12.986974 11759 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240503 02:14:12.987968 11759 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240503 02:14:12.988282 11759 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240503 02:14:12.989344 11759 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240503 02:14:12.991189 11759 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240503 02:14:12.991317 11759 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240503 02:14:12.991438 11759 ts_tablet_manager.cc:590] Registered 0 tablets
I20240503 02:14:12.991514 11759 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240503 02:14:13.029063 11759 rpc_server.cc:307] RPC server started. Bound to: 127.9.151.130:35753
I20240503 02:14:13.029165 11885 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.151.130:35753 every 8 connection(s)
I20240503 02:14:13.030843 11759 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknrxqHk/test-tmp/tablet_copy_client_session-itest.0.TabletCopyClientSessionITest.TestStartTabletCopyWhileSourceBootstrapping.1714702442467385-9822-0/minicluster-data/ts-1/data/info.pb
I20240503 02:14:13.040011 9822 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknrxqHk/build/asan/bin/kudu as pid 11759
I20240503 02:14:13.040272 9822 tablet_copy_client_session-itest.cc:139] Starting loop 2
I20240503 02:14:13.040400 9822 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasknrxqHk/build/asan/bin/kudu pid 11156
I20240503 02:14:13.046553 11886 heartbeater.cc:347] Connected to a master server at 127.9.151.190:36085
I20240503 02:14:13.046953 11886 heartbeater.cc:464] Registering TS with master...
I20240503 02:14:13.047953 11886 heartbeater.cc:511] Master 127.9.151.190:36085 requested a full tablet report, sending...
I20240503 02:14:13.049764 11495 generic_service.cc:196] Checking for leaks (request via RPC)
I20240503 02:14:13.049929 10221 ts_manager.cc:194] Re-registered known tserver with Master: 4c8a3af9b5184a51a3c0fe3221c01c75 (127.9.151.130:35753)
I20240503 02:14:13.051435 10221 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.9.151.130:34317
I20240503 02:14:13.158537 9822 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknrxqHk/build/asan/bin/kudu with pid 11156
I20240503 02:14:13.182909 9822 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknrxqHk/build/asan/bin/kudu
/tmp/dist-test-tasknrxqHk/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknrxqHk/test-tmp/tablet_copy_client_session-itest.0.TabletCopyClientSessionITest.TestStartTabletCopyWhileSourceBootstrapping.1714702442467385-9822-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasknrxqHk/test-tmp/tablet_copy_client_session-itest.0.TabletCopyClientSessionITest.TestStartTabletCopyWhileSourceBootstrapping.1714702442467385-9822-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknrxqHk/test-tmp/tablet_copy_client_session-itest.0.TabletCopyClientSessionITest.TestStartTabletCopyWhileSourceBootstrapping.1714702442467385-9822-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasknrxqHk/test-tmp/tablet_copy_client_session-itest.0.TabletCopyClientSessionITest.TestStartTabletCopyWhileSourceBootstrapping.1714702442467385-9822-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.151.129:33077
--local_ip_for_outbound_sockets=127.9.151.129
--tserver_master_addrs=127.9.151.190:36085
--webserver_port=43885
--webserver_interface=127.9.151.129
--builtin_ntp_servers=127.9.151.148:38865
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin with env {}
I20240503 02:14:13.199318 11899 ts_tablet_manager.cc:902] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: Initiating tablet copy from peer 7cd9a8da65604af78132e9045dac5180 (127.9.151.129:33077)
I20240503 02:14:13.203017 11899 tablet_copy_client.cc:308] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: tablet copy: Beginning tablet copy session from remote peer at address 127.9.151.129:33077
I20240503 02:14:13.207840 11899 ts_tablet_manager.cc:902] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: Initiating tablet copy from peer 7cd9a8da65604af78132e9045dac5180 (127.9.151.129:33077)
I20240503 02:14:13.209273 11899 tablet_copy_client.cc:308] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: tablet copy: Beginning tablet copy session from remote peer at address 127.9.151.129:33077
I20240503 02:14:13.222613 11899 ts_tablet_manager.cc:902] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: Initiating tablet copy from peer 7cd9a8da65604af78132e9045dac5180 (127.9.151.129:33077)
I20240503 02:14:13.224524 11899 tablet_copy_client.cc:308] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: tablet copy: Beginning tablet copy session from remote peer at address 127.9.151.129:33077
I20240503 02:14:13.236428 11899 ts_tablet_manager.cc:902] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: Initiating tablet copy from peer 7cd9a8da65604af78132e9045dac5180 (127.9.151.129:33077)
I20240503 02:14:13.238039 11899 tablet_copy_client.cc:308] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: tablet copy: Beginning tablet copy session from remote peer at address 127.9.151.129:33077
I20240503 02:14:13.241954 11899 ts_tablet_manager.cc:902] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: Initiating tablet copy from peer 7cd9a8da65604af78132e9045dac5180 (127.9.151.129:33077)
I20240503 02:14:13.243531 11899 tablet_copy_client.cc:308] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: tablet copy: Beginning tablet copy session from remote peer at address 127.9.151.129:33077
I20240503 02:14:13.247375 11899 ts_tablet_manager.cc:902] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: Initiating tablet copy from peer 7cd9a8da65604af78132e9045dac5180 (127.9.151.129:33077)
I20240503 02:14:13.248657 11899 tablet_copy_client.cc:308] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: tablet copy: Beginning tablet copy session from remote peer at address 127.9.151.129:33077
I20240503 02:14:13.259306 11899 ts_tablet_manager.cc:902] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: Initiating tablet copy from peer 7cd9a8da65604af78132e9045dac5180 (127.9.151.129:33077)
I20240503 02:14:13.260718 11899 tablet_copy_client.cc:308] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: tablet copy: Beginning tablet copy session from remote peer at address 127.9.151.129:33077
I20240503 02:14:13.274930 11899 ts_tablet_manager.cc:902] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: Initiating tablet copy from peer 7cd9a8da65604af78132e9045dac5180 (127.9.151.129:33077)
I20240503 02:14:13.276319 11899 tablet_copy_client.cc:308] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: tablet copy: Beginning tablet copy session from remote peer at address 127.9.151.129:33077
I20240503 02:14:13.297391 11899 ts_tablet_manager.cc:902] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: Initiating tablet copy from peer 7cd9a8da65604af78132e9045dac5180 (127.9.151.129:33077)
I20240503 02:14:13.298736 11899 tablet_copy_client.cc:308] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: tablet copy: Beginning tablet copy session from remote peer at address 127.9.151.129:33077
I20240503 02:14:13.301077 11899 ts_tablet_manager.cc:902] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: Initiating tablet copy from peer 7cd9a8da65604af78132e9045dac5180 (127.9.151.129:33077)
I20240503 02:14:13.302819 11899 tablet_copy_client.cc:308] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: tablet copy: Beginning tablet copy session from remote peer at address 127.9.151.129:33077
I20240503 02:14:13.318414 11899 ts_tablet_manager.cc:902] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: Initiating tablet copy from peer 7cd9a8da65604af78132e9045dac5180 (127.9.151.129:33077)
I20240503 02:14:13.319870 11899 tablet_copy_client.cc:308] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: tablet copy: Beginning tablet copy session from remote peer at address 127.9.151.129:33077
I20240503 02:14:13.322975 11899 ts_tablet_manager.cc:902] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: Initiating tablet copy from peer 7cd9a8da65604af78132e9045dac5180 (127.9.151.129:33077)
I20240503 02:14:13.324486 11899 tablet_copy_client.cc:308] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: tablet copy: Beginning tablet copy session from remote peer at address 127.9.151.129:33077
I20240503 02:14:13.331670 11899 ts_tablet_manager.cc:902] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: Initiating tablet copy from peer 7cd9a8da65604af78132e9045dac5180 (127.9.151.129:33077)
I20240503 02:14:13.333271 11899 tablet_copy_client.cc:308] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: tablet copy: Beginning tablet copy session from remote peer at address 127.9.151.129:33077
I20240503 02:14:13.340325 11899 ts_tablet_manager.cc:902] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: Initiating tablet copy from peer 7cd9a8da65604af78132e9045dac5180 (127.9.151.129:33077)
I20240503 02:14:13.348219 11899 tablet_copy_client.cc:308] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: tablet copy: Beginning tablet copy session from remote peer at address 127.9.151.129:33077
I20240503 02:14:13.376329 11899 ts_tablet_manager.cc:902] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: Initiating tablet copy from peer 7cd9a8da65604af78132e9045dac5180 (127.9.151.129:33077)
I20240503 02:14:13.381335 11899 tablet_copy_client.cc:308] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: tablet copy: Beginning tablet copy session from remote peer at address 127.9.151.129:33077
I20240503 02:14:13.397096 11899 ts_tablet_manager.cc:902] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: Initiating tablet copy from peer 7cd9a8da65604af78132e9045dac5180 (127.9.151.129:33077)
I20240503 02:14:13.398639 11899 tablet_copy_client.cc:308] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: tablet copy: Beginning tablet copy session from remote peer at address 127.9.151.129:33077
I20240503 02:14:13.406040 11899 ts_tablet_manager.cc:902] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: Initiating tablet copy from peer 7cd9a8da65604af78132e9045dac5180 (127.9.151.129:33077)
I20240503 02:14:13.407930 11899 tablet_copy_client.cc:308] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: tablet copy: Beginning tablet copy session from remote peer at address 127.9.151.129:33077
I20240503 02:14:13.411540 11899 ts_tablet_manager.cc:902] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: Initiating tablet copy from peer 7cd9a8da65604af78132e9045dac5180 (127.9.151.129:33077)
I20240503 02:14:13.415042 11899 tablet_copy_client.cc:308] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: tablet copy: Beginning tablet copy session from remote peer at address 127.9.151.129:33077
I20240503 02:14:13.424057 11899 ts_tablet_manager.cc:902] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: Initiating tablet copy from peer 7cd9a8da65604af78132e9045dac5180 (127.9.151.129:33077)
I20240503 02:14:13.425742 11899 tablet_copy_client.cc:308] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: tablet copy: Beginning tablet copy session from remote peer at address 127.9.151.129:33077
I20240503 02:14:13.435031 11899 ts_tablet_manager.cc:902] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: Initiating tablet copy from peer 7cd9a8da65604af78132e9045dac5180 (127.9.151.129:33077)
I20240503 02:14:13.436722 11899 tablet_copy_client.cc:308] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: tablet copy: Beginning tablet copy session from remote peer at address 127.9.151.129:33077
I20240503 02:14:13.441099 11899 ts_tablet_manager.cc:902] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: Initiating tablet copy from peer 7cd9a8da65604af78132e9045dac5180 (127.9.151.129:33077)
I20240503 02:14:13.445721 11899 tablet_copy_client.cc:308] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: tablet copy: Beginning tablet copy session from remote peer at address 127.9.151.129:33077
I20240503 02:14:13.454483 11899 ts_tablet_manager.cc:902] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: Initiating tablet copy from peer 7cd9a8da65604af78132e9045dac5180 (127.9.151.129:33077)
I20240503 02:14:13.455772 11899 tablet_copy_client.cc:308] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: tablet copy: Beginning tablet copy session from remote peer at address 127.9.151.129:33077
I20240503 02:14:13.465948 11899 ts_tablet_manager.cc:902] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: Initiating tablet copy from peer 7cd9a8da65604af78132e9045dac5180 (127.9.151.129:33077)
I20240503 02:14:13.467621 11899 tablet_copy_client.cc:308] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: tablet copy: Beginning tablet copy session from remote peer at address 127.9.151.129:33077
I20240503 02:14:13.483814 11899 ts_tablet_manager.cc:902] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: Initiating tablet copy from peer 7cd9a8da65604af78132e9045dac5180 (127.9.151.129:33077)
I20240503 02:14:13.485211 11899 tablet_copy_client.cc:308] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: tablet copy: Beginning tablet copy session from remote peer at address 127.9.151.129:33077
I20240503 02:14:13.489159 11899 ts_tablet_manager.cc:902] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: Initiating tablet copy from peer 7cd9a8da65604af78132e9045dac5180 (127.9.151.129:33077)
I20240503 02:14:13.490622 11899 tablet_copy_client.cc:308] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: tablet copy: Beginning tablet copy session from remote peer at address 127.9.151.129:33077
I20240503 02:14:13.494205 11899 ts_tablet_manager.cc:902] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: Initiating tablet copy from peer 7cd9a8da65604af78132e9045dac5180 (127.9.151.129:33077)
I20240503 02:14:13.495792 11899 tablet_copy_client.cc:308] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: tablet copy: Beginning tablet copy session from remote peer at address 127.9.151.129:33077
I20240503 02:14:13.516100 11899 ts_tablet_manager.cc:902] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: Initiating tablet copy from peer 7cd9a8da65604af78132e9045dac5180 (127.9.151.129:33077)
I20240503 02:14:13.519045 11899 tablet_copy_client.cc:308] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: tablet copy: Beginning tablet copy session from remote peer at address 127.9.151.129:33077
I20240503 02:14:13.534785 11899 ts_tablet_manager.cc:902] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: Initiating tablet copy from peer 7cd9a8da65604af78132e9045dac5180 (127.9.151.129:33077)
I20240503 02:14:13.536144 11899 tablet_copy_client.cc:308] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: tablet copy: Beginning tablet copy session from remote peer at address 127.9.151.129:33077
I20240503 02:14:13.540480 11899 ts_tablet_manager.cc:902] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: Initiating tablet copy from peer 7cd9a8da65604af78132e9045dac5180 (127.9.151.129:33077)
I20240503 02:14:13.547981 11899 tablet_copy_client.cc:308] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: tablet copy: Beginning tablet copy session from remote peer at address 127.9.151.129:33077
I20240503 02:14:13.561231 11899 ts_tablet_manager.cc:902] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: Initiating tablet copy from peer 7cd9a8da65604af78132e9045dac5180 (127.9.151.129:33077)
I20240503 02:14:13.562579 11899 tablet_copy_client.cc:308] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: tablet copy: Beginning tablet copy session from remote peer at address 127.9.151.129:33077
I20240503 02:14:13.574925 11899 ts_tablet_manager.cc:902] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: Initiating tablet copy from peer 7cd9a8da65604af78132e9045dac5180 (127.9.151.129:33077)
I20240503 02:14:13.576253 11899 tablet_copy_client.cc:308] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: tablet copy: Beginning tablet copy session from remote peer at address 127.9.151.129:33077
I20240503 02:14:13.580242 11899 ts_tablet_manager.cc:902] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: Initiating tablet copy from peer 7cd9a8da65604af78132e9045dac5180 (127.9.151.129:33077)
I20240503 02:14:13.581969 11899 tablet_copy_client.cc:308] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: tablet copy: Beginning tablet copy session from remote peer at address 127.9.151.129:33077
I20240503 02:14:13.597230 11899 ts_tablet_manager.cc:902] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: Initiating tablet copy from peer 7cd9a8da65604af78132e9045dac5180 (127.9.151.129:33077)
I20240503 02:14:13.599234 11899 tablet_copy_client.cc:308] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: tablet copy: Beginning tablet copy session from remote peer at address 127.9.151.129:33077
I20240503 02:14:13.606994 11899 ts_tablet_manager.cc:902] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: Initiating tablet copy from peer 7cd9a8da65604af78132e9045dac5180 (127.9.151.129:33077)
W20240503 02:14:13.608120 11898 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
I20240503 02:14:13.608502 11899 tablet_copy_client.cc:308] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: tablet copy: Beginning tablet copy session from remote peer at address 127.9.151.129:33077
W20240503 02:14:13.608747 11898 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240503 02:14:13.609086 11898 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20240503 02:14:13.617755 11899 ts_tablet_manager.cc:902] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: Initiating tablet copy from peer 7cd9a8da65604af78132e9045dac5180 (127.9.151.129:33077)
W20240503 02:14:13.618252 11898 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240503 02:14:13.618554 11898 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.151.129
I20240503 02:14:13.619014 11899 tablet_copy_client.cc:308] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: tablet copy: Beginning tablet copy session from remote peer at address 127.9.151.129:33077
I20240503 02:14:13.624791 11899 ts_tablet_manager.cc:902] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: Initiating tablet copy from peer 7cd9a8da65604af78132e9045dac5180 (127.9.151.129:33077)
I20240503 02:14:13.626034 11899 tablet_copy_client.cc:308] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: tablet copy: Beginning tablet copy session from remote peer at address 127.9.151.129:33077
I20240503 02:14:13.630806 11898 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.151.148:38865
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknrxqHk/test-tmp/tablet_copy_client_session-itest.0.TabletCopyClientSessionITest.TestStartTabletCopyWhileSourceBootstrapping.1714702442467385-9822-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasknrxqHk/test-tmp/tablet_copy_client_session-itest.0.TabletCopyClientSessionITest.TestStartTabletCopyWhileSourceBootstrapping.1714702442467385-9822-0/minicluster-data/ts-0/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.151.129:33077
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknrxqHk/test-tmp/tablet_copy_client_session-itest.0.TabletCopyClientSessionITest.TestStartTabletCopyWhileSourceBootstrapping.1714702442467385-9822-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.151.129
--webserver_port=43885
--tserver_master_addrs=127.9.151.190:36085
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.151.129
--log_dir=/tmp/dist-test-tasknrxqHk/test-tmp/tablet_copy_client_session-itest.0.TabletCopyClientSessionITest.TestStartTabletCopyWhileSourceBootstrapping.1714702442467385-9822-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c39e72a98e124e5a12080bac42105b996594d806
build type FASTDEBUG
built by None at 03 May 2024 01:43:22 UTC on efd1337a2aa3
build id 99
ASAN enabled
I20240503 02:14:13.632467 11898 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240503 02:14:13.634378 11898 file_cache.cc:492] Constructed file cache file cache with capacity 419430
I20240503 02:14:13.644654 11899 ts_tablet_manager.cc:902] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: Initiating tablet copy from peer 7cd9a8da65604af78132e9045dac5180 (127.9.151.129:33077)
I20240503 02:14:13.645942 11899 tablet_copy_client.cc:308] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: tablet copy: Beginning tablet copy session from remote peer at address 127.9.151.129:33077
I20240503 02:14:13.649080 11899 ts_tablet_manager.cc:902] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: Initiating tablet copy from peer 7cd9a8da65604af78132e9045dac5180 (127.9.151.129:33077)
I20240503 02:14:13.656335 11899 tablet_copy_client.cc:308] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: tablet copy: Beginning tablet copy session from remote peer at address 127.9.151.129:33077
W20240503 02:14:13.657164 12052 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240503 02:14:13.658322 12051 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240503 02:14:13.659598 11898 server_base.cc:1034] running on GCE node
W20240503 02:14:13.660132 12054 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240503 02:14:13.660670 11899 ts_tablet_manager.cc:902] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: Initiating tablet copy from peer 7cd9a8da65604af78132e9045dac5180 (127.9.151.129:33077)
I20240503 02:14:13.661046 11898 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240503 02:14:13.662353 11898 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240503 02:14:13.662473 11899 tablet_copy_client.cc:308] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: tablet copy: Beginning tablet copy session from remote peer at address 127.9.151.129:33077
I20240503 02:14:13.663648 11898 hybrid_clock.cc:648] HybridClock initialized: now 1714702453663546 us; error 80 us; skew 500 ppm
I20240503 02:14:13.664080 11898 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20240503 02:14:13.667146 11898 webserver.cc:460] Webserver started at http://127.9.151.129:43885/ using document root <none> and password file <none>
I20240503 02:14:13.667692 11899 ts_tablet_manager.cc:902] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: Initiating tablet copy from peer 7cd9a8da65604af78132e9045dac5180 (127.9.151.129:33077)
I20240503 02:14:13.668146 11898 fs_manager.cc:359] Metadata directory not provided
I20240503 02:14:13.668334 11898 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240503 02:14:13.670084 11899 tablet_copy_client.cc:308] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: tablet copy: Beginning tablet copy session from remote peer at address 127.9.151.129:33077
I20240503 02:14:13.672427 11901 ts_tablet_manager.cc:902] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: Initiating tablet copy from peer 7cd9a8da65604af78132e9045dac5180 (127.9.151.129:33077)
I20240503 02:14:13.676178 11898 fs_manager.cc:709] Time spent opening directory manager: real 0.006s user 0.007s sys 0.000s
I20240503 02:14:13.679874 12077 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240503 02:14:13.681136 11901 tablet_copy_client.cc:308] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: tablet copy: Beginning tablet copy session from remote peer at address 127.9.151.129:33077
I20240503 02:14:13.681778 11898 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.005s sys 0.000s
I20240503 02:14:13.681964 11898 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknrxqHk/test-tmp/tablet_copy_client_session-itest.0.TabletCopyClientSessionITest.TestStartTabletCopyWhileSourceBootstrapping.1714702442467385-9822-0/minicluster-data/ts-0/data,/tmp/dist-test-tasknrxqHk/test-tmp/tablet_copy_client_session-itest.0.TabletCopyClientSessionITest.TestStartTabletCopyWhileSourceBootstrapping.1714702442467385-9822-0/minicluster-data/ts-0/wal
uuid: "7cd9a8da65604af78132e9045dac5180"
format_stamp: "Formatted at 2024-05-03 02:14:03 on dist-test-slave-twlh"
I20240503 02:14:13.683272 11898 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknrxqHk/test-tmp/tablet_copy_client_session-itest.0.TabletCopyClientSessionITest.TestStartTabletCopyWhileSourceBootstrapping.1714702442467385-9822-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasknrxqHk/test-tmp/tablet_copy_client_session-itest.0.TabletCopyClientSessionITest.TestStartTabletCopyWhileSourceBootstrapping.1714702442467385-9822-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasknrxqHk/test-tmp/tablet_copy_client_session-itest.0.TabletCopyClientSessionITest.TestStartTabletCopyWhileSourceBootstrapping.1714702442467385-9822-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240503 02:14:13.688786 11901 ts_tablet_manager.cc:902] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: Initiating tablet copy from peer 7cd9a8da65604af78132e9045dac5180 (127.9.151.129:33077)
I20240503 02:14:13.690315 11901 tablet_copy_client.cc:308] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: tablet copy: Beginning tablet copy session from remote peer at address 127.9.151.129:33077
I20240503 02:14:13.703625 11901 ts_tablet_manager.cc:902] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: Initiating tablet copy from peer 7cd9a8da65604af78132e9045dac5180 (127.9.151.129:33077)
I20240503 02:14:13.705497 11901 tablet_copy_client.cc:308] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: tablet copy: Beginning tablet copy session from remote peer at address 127.9.151.129:33077
I20240503 02:14:13.717715 11898 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240503 02:14:13.718859 11898 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240503 02:14:13.719161 11898 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240503 02:14:13.720327 11898 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240503 02:14:13.722990 11901 ts_tablet_manager.cc:902] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: Initiating tablet copy from peer 7cd9a8da65604af78132e9045dac5180 (127.9.151.129:33077)
I20240503 02:14:13.723106 12095 ts_tablet_manager.cc:522] Loading tablet metadata (0/1 complete)
I20240503 02:14:13.724860 11901 tablet_copy_client.cc:308] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: tablet copy: Beginning tablet copy session from remote peer at address 127.9.151.129:33077
I20240503 02:14:13.730650 11898 ts_tablet_manager.cc:559] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20240503 02:14:13.730868 11898 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.008s user 0.001s sys 0.000s
I20240503 02:14:13.730979 11898 ts_tablet_manager.cc:574] Registering tablets (0/1 complete)
I20240503 02:14:13.734134 11898 ts_tablet_manager.cc:590] Registered 1 tablets
I20240503 02:14:13.734274 11898 ts_tablet_manager.cc:569] Time spent register tablets: real 0.003s user 0.000s sys 0.000s
I20240503 02:14:13.734755 12095 tablet_bootstrap.cc:492] T 375647a877014ca48ca32ff220098cd1 P 7cd9a8da65604af78132e9045dac5180: Bootstrap starting.
I20240503 02:14:13.736964 11901 ts_tablet_manager.cc:902] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: Initiating tablet copy from peer 7cd9a8da65604af78132e9045dac5180 (127.9.151.129:33077)
I20240503 02:14:13.738312 11901 tablet_copy_client.cc:308] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: tablet copy: Beginning tablet copy session from remote peer at address 127.9.151.129:33077
I20240503 02:14:13.748148 11901 ts_tablet_manager.cc:902] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: Initiating tablet copy from peer 7cd9a8da65604af78132e9045dac5180 (127.9.151.129:33077)
I20240503 02:14:13.763329 11901 tablet_copy_client.cc:308] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: tablet copy: Beginning tablet copy session from remote peer at address 127.9.151.129:33077
I20240503 02:14:13.767525 11901 ts_tablet_manager.cc:902] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: Initiating tablet copy from peer 7cd9a8da65604af78132e9045dac5180 (127.9.151.129:33077)
I20240503 02:14:13.774582 11901 tablet_copy_client.cc:308] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: tablet copy: Beginning tablet copy session from remote peer at address 127.9.151.129:33077
I20240503 02:14:13.784950 11901 ts_tablet_manager.cc:902] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: Initiating tablet copy from peer 7cd9a8da65604af78132e9045dac5180 (127.9.151.129:33077)
I20240503 02:14:13.795107 11901 tablet_copy_client.cc:308] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: tablet copy: Beginning tablet copy session from remote peer at address 127.9.151.129:33077
I20240503 02:14:13.803309 11901 ts_tablet_manager.cc:902] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: Initiating tablet copy from peer 7cd9a8da65604af78132e9045dac5180 (127.9.151.129:33077)
I20240503 02:14:13.805895 11901 tablet_copy_client.cc:308] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: tablet copy: Beginning tablet copy session from remote peer at address 127.9.151.129:33077
I20240503 02:14:13.812136 11901 ts_tablet_manager.cc:902] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: Initiating tablet copy from peer 7cd9a8da65604af78132e9045dac5180 (127.9.151.129:33077)
I20240503 02:14:13.813406 11901 tablet_copy_client.cc:308] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: tablet copy: Beginning tablet copy session from remote peer at address 127.9.151.129:33077
I20240503 02:14:13.828040 11901 ts_tablet_manager.cc:902] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: Initiating tablet copy from peer 7cd9a8da65604af78132e9045dac5180 (127.9.151.129:33077)
I20240503 02:14:13.837790 11901 tablet_copy_client.cc:308] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: tablet copy: Beginning tablet copy session from remote peer at address 127.9.151.129:33077
I20240503 02:14:13.841001 11901 ts_tablet_manager.cc:902] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: Initiating tablet copy from peer 7cd9a8da65604af78132e9045dac5180 (127.9.151.129:33077)
I20240503 02:14:13.844838 11901 tablet_copy_client.cc:308] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: tablet copy: Beginning tablet copy session from remote peer at address 127.9.151.129:33077
I20240503 02:14:13.848764 11901 ts_tablet_manager.cc:902] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: Initiating tablet copy from peer 7cd9a8da65604af78132e9045dac5180 (127.9.151.129:33077)
I20240503 02:14:13.859516 11901 tablet_copy_client.cc:308] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: tablet copy: Beginning tablet copy session from remote peer at address 127.9.151.129:33077
I20240503 02:14:13.863554 11898 rpc_server.cc:307] RPC server started. Bound to: 127.9.151.129:33077
I20240503 02:14:13.863579 12241 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.151.129:33077 every 8 connection(s)
I20240503 02:14:13.865556 11898 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknrxqHk/test-tmp/tablet_copy_client_session-itest.0.TabletCopyClientSessionITest.TestStartTabletCopyWhileSourceBootstrapping.1714702442467385-9822-0/minicluster-data/ts-0/data/info.pb
I20240503 02:14:13.872880 9822 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknrxqHk/build/asan/bin/kudu as pid 11898
I20240503 02:14:13.882344 12242 heartbeater.cc:347] Connected to a master server at 127.9.151.190:36085
I20240503 02:14:13.882755 12242 heartbeater.cc:464] Registering TS with master...
I20240503 02:14:13.883778 12242 heartbeater.cc:511] Master 127.9.151.190:36085 requested a full tablet report, sending...
I20240503 02:14:13.886826 10221 ts_manager.cc:194] Re-registered known tserver with Master: 7cd9a8da65604af78132e9045dac5180 (127.9.151.129:33077)
I20240503 02:14:13.888039 10221 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.9.151.129:42055
I20240503 02:14:13.889298 12095 log.cc:824] T 375647a877014ca48ca32ff220098cd1 P 7cd9a8da65604af78132e9045dac5180: Log is configured to *not* fsync() on all Append() calls
I20240503 02:14:13.891184 11901 ts_tablet_manager.cc:902] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: Initiating tablet copy from peer 7cd9a8da65604af78132e9045dac5180 (127.9.151.129:33077)
I20240503 02:14:13.892480 11901 tablet_copy_client.cc:308] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: tablet copy: Beginning tablet copy session from remote peer at address 127.9.151.129:33077
I20240503 02:14:13.904875 12205 tablet_copy_service.cc:140] P 7cd9a8da65604af78132e9045dac5180: Received BeginTabletCopySession request for tablet 375647a877014ca48ca32ff220098cd1 from peer 4c8a3af9b5184a51a3c0fe3221c01c75 ({username='slave'} at 127.9.151.130:58285)
I20240503 02:14:13.905305 12205 tablet_copy_service.cc:161] P 7cd9a8da65604af78132e9045dac5180: Beginning new tablet copy session on tablet 375647a877014ca48ca32ff220098cd1 from peer 4c8a3af9b5184a51a3c0fe3221c01c75 at {username='slave'} at 127.9.151.130:58285: session id = 4c8a3af9b5184a51a3c0fe3221c01c75-375647a877014ca48ca32ff220098cd1
W20240503 02:14:13.907629 12205 tablet_copy_service.cc:478] P 7cd9a8da65604af78132e9045dac5180: Error handling TabletCopyService RPC request from {username='slave'} at 127.9.151.130:58285: Error beginning tablet copy session for tablet 375647a877014ca48ca32ff220098cd1: Illegal state: The tablet is not in a running state: BOOTSTRAPPING
I20240503 02:14:13.924479 11901 ts_tablet_manager.cc:902] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: Initiating tablet copy from peer 7cd9a8da65604af78132e9045dac5180 (127.9.151.129:33077)
I20240503 02:14:13.925886 11901 tablet_copy_client.cc:308] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: tablet copy: Beginning tablet copy session from remote peer at address 127.9.151.129:33077
I20240503 02:14:13.926810 12205 tablet_copy_service.cc:140] P 7cd9a8da65604af78132e9045dac5180: Received BeginTabletCopySession request for tablet 375647a877014ca48ca32ff220098cd1 from peer 4c8a3af9b5184a51a3c0fe3221c01c75 ({username='slave'} at 127.9.151.130:58285)
I20240503 02:14:13.927038 12205 tablet_copy_service.cc:161] P 7cd9a8da65604af78132e9045dac5180: Beginning new tablet copy session on tablet 375647a877014ca48ca32ff220098cd1 from peer 4c8a3af9b5184a51a3c0fe3221c01c75 at {username='slave'} at 127.9.151.130:58285: session id = 4c8a3af9b5184a51a3c0fe3221c01c75-375647a877014ca48ca32ff220098cd1
W20240503 02:14:13.927248 12205 tablet_copy_service.cc:478] P 7cd9a8da65604af78132e9045dac5180: Error handling TabletCopyService RPC request from {username='slave'} at 127.9.151.130:58285: Error beginning tablet copy session for tablet 375647a877014ca48ca32ff220098cd1: Illegal state: The tablet is not in a running state: BOOTSTRAPPING
I20240503 02:14:13.929880 11901 ts_tablet_manager.cc:902] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: Initiating tablet copy from peer 7cd9a8da65604af78132e9045dac5180 (127.9.151.129:33077)
I20240503 02:14:13.939106 11901 tablet_copy_client.cc:308] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: tablet copy: Beginning tablet copy session from remote peer at address 127.9.151.129:33077
I20240503 02:14:13.940177 12205 tablet_copy_service.cc:140] P 7cd9a8da65604af78132e9045dac5180: Received BeginTabletCopySession request for tablet 375647a877014ca48ca32ff220098cd1 from peer 4c8a3af9b5184a51a3c0fe3221c01c75 ({username='slave'} at 127.9.151.130:58285)
I20240503 02:14:13.940368 12205 tablet_copy_service.cc:161] P 7cd9a8da65604af78132e9045dac5180: Beginning new tablet copy session on tablet 375647a877014ca48ca32ff220098cd1 from peer 4c8a3af9b5184a51a3c0fe3221c01c75 at {username='slave'} at 127.9.151.130:58285: session id = 4c8a3af9b5184a51a3c0fe3221c01c75-375647a877014ca48ca32ff220098cd1
W20240503 02:14:13.940568 12205 tablet_copy_service.cc:478] P 7cd9a8da65604af78132e9045dac5180: Error handling TabletCopyService RPC request from {username='slave'} at 127.9.151.130:58285: Error beginning tablet copy session for tablet 375647a877014ca48ca32ff220098cd1: Illegal state: The tablet is not in a running state: BOOTSTRAPPING
I20240503 02:14:13.946008 11901 ts_tablet_manager.cc:902] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: Initiating tablet copy from peer 7cd9a8da65604af78132e9045dac5180 (127.9.151.129:33077)
I20240503 02:14:13.947299 11901 tablet_copy_client.cc:308] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: tablet copy: Beginning tablet copy session from remote peer at address 127.9.151.129:33077
I20240503 02:14:13.948100 12205 tablet_copy_service.cc:140] P 7cd9a8da65604af78132e9045dac5180: Received BeginTabletCopySession request for tablet 375647a877014ca48ca32ff220098cd1 from peer 4c8a3af9b5184a51a3c0fe3221c01c75 ({username='slave'} at 127.9.151.130:58285)
I20240503 02:14:13.948325 12205 tablet_copy_service.cc:161] P 7cd9a8da65604af78132e9045dac5180: Beginning new tablet copy session on tablet 375647a877014ca48ca32ff220098cd1 from peer 4c8a3af9b5184a51a3c0fe3221c01c75 at {username='slave'} at 127.9.151.130:58285: session id = 4c8a3af9b5184a51a3c0fe3221c01c75-375647a877014ca48ca32ff220098cd1
W20240503 02:14:13.948520 12205 tablet_copy_service.cc:478] P 7cd9a8da65604af78132e9045dac5180: Error handling TabletCopyService RPC request from {username='slave'} at 127.9.151.130:58285: Error beginning tablet copy session for tablet 375647a877014ca48ca32ff220098cd1: Illegal state: The tablet is not in a running state: BOOTSTRAPPING
I20240503 02:14:13.960106 11901 ts_tablet_manager.cc:902] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: Initiating tablet copy from peer 7cd9a8da65604af78132e9045dac5180 (127.9.151.129:33077)
I20240503 02:14:13.967056 11901 tablet_copy_client.cc:308] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: tablet copy: Beginning tablet copy session from remote peer at address 127.9.151.129:33077
I20240503 02:14:13.967876 12205 tablet_copy_service.cc:140] P 7cd9a8da65604af78132e9045dac5180: Received BeginTabletCopySession request for tablet 375647a877014ca48ca32ff220098cd1 from peer 4c8a3af9b5184a51a3c0fe3221c01c75 ({username='slave'} at 127.9.151.130:58285)
I20240503 02:14:13.968080 12205 tablet_copy_service.cc:161] P 7cd9a8da65604af78132e9045dac5180: Beginning new tablet copy session on tablet 375647a877014ca48ca32ff220098cd1 from peer 4c8a3af9b5184a51a3c0fe3221c01c75 at {username='slave'} at 127.9.151.130:58285: session id = 4c8a3af9b5184a51a3c0fe3221c01c75-375647a877014ca48ca32ff220098cd1
W20240503 02:14:13.968310 12205 tablet_copy_service.cc:478] P 7cd9a8da65604af78132e9045dac5180: Error handling TabletCopyService RPC request from {username='slave'} at 127.9.151.130:58285: Error beginning tablet copy session for tablet 375647a877014ca48ca32ff220098cd1: Illegal state: The tablet is not in a running state: BOOTSTRAPPING
I20240503 02:14:13.986925 11899 ts_tablet_manager.cc:902] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: Initiating tablet copy from peer 7cd9a8da65604af78132e9045dac5180 (127.9.151.129:33077)
I20240503 02:14:13.988349 11899 tablet_copy_client.cc:308] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: tablet copy: Beginning tablet copy session from remote peer at address 127.9.151.129:33077
I20240503 02:14:13.989055 12205 tablet_copy_service.cc:140] P 7cd9a8da65604af78132e9045dac5180: Received BeginTabletCopySession request for tablet 375647a877014ca48ca32ff220098cd1 from peer 4c8a3af9b5184a51a3c0fe3221c01c75 ({username='slave'} at 127.9.151.130:58285)
I20240503 02:14:13.989271 12205 tablet_copy_service.cc:161] P 7cd9a8da65604af78132e9045dac5180: Beginning new tablet copy session on tablet 375647a877014ca48ca32ff220098cd1 from peer 4c8a3af9b5184a51a3c0fe3221c01c75 at {username='slave'} at 127.9.151.130:58285: session id = 4c8a3af9b5184a51a3c0fe3221c01c75-375647a877014ca48ca32ff220098cd1
W20240503 02:14:13.989485 12205 tablet_copy_service.cc:478] P 7cd9a8da65604af78132e9045dac5180: Error handling TabletCopyService RPC request from {username='slave'} at 127.9.151.130:58285: Error beginning tablet copy session for tablet 375647a877014ca48ca32ff220098cd1: Illegal state: The tablet is not in a running state: BOOTSTRAPPING
I20240503 02:14:14.003334 11899 ts_tablet_manager.cc:902] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: Initiating tablet copy from peer 7cd9a8da65604af78132e9045dac5180 (127.9.151.129:33077)
I20240503 02:14:14.005057 11899 tablet_copy_client.cc:308] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: tablet copy: Beginning tablet copy session from remote peer at address 127.9.151.129:33077
I20240503 02:14:14.006083 12205 tablet_copy_service.cc:140] P 7cd9a8da65604af78132e9045dac5180: Received BeginTabletCopySession request for tablet 375647a877014ca48ca32ff220098cd1 from peer 4c8a3af9b5184a51a3c0fe3221c01c75 ({username='slave'} at 127.9.151.130:58285)
I20240503 02:14:14.006330 12205 tablet_copy_service.cc:161] P 7cd9a8da65604af78132e9045dac5180: Beginning new tablet copy session on tablet 375647a877014ca48ca32ff220098cd1 from peer 4c8a3af9b5184a51a3c0fe3221c01c75 at {username='slave'} at 127.9.151.130:58285: session id = 4c8a3af9b5184a51a3c0fe3221c01c75-375647a877014ca48ca32ff220098cd1
W20240503 02:14:14.006551 12205 tablet_copy_service.cc:478] P 7cd9a8da65604af78132e9045dac5180: Error handling TabletCopyService RPC request from {username='slave'} at 127.9.151.130:58285: Error beginning tablet copy session for tablet 375647a877014ca48ca32ff220098cd1: Illegal state: The tablet is not in a running state: BOOTSTRAPPING
I20240503 02:14:14.012348 11899 ts_tablet_manager.cc:902] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: Initiating tablet copy from peer 7cd9a8da65604af78132e9045dac5180 (127.9.151.129:33077)
I20240503 02:14:14.015049 11899 tablet_copy_client.cc:308] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: tablet copy: Beginning tablet copy session from remote peer at address 127.9.151.129:33077
I20240503 02:14:14.015915 12205 tablet_copy_service.cc:140] P 7cd9a8da65604af78132e9045dac5180: Received BeginTabletCopySession request for tablet 375647a877014ca48ca32ff220098cd1 from peer 4c8a3af9b5184a51a3c0fe3221c01c75 ({username='slave'} at 127.9.151.130:58285)
I20240503 02:14:14.016155 12205 tablet_copy_service.cc:161] P 7cd9a8da65604af78132e9045dac5180: Beginning new tablet copy session on tablet 375647a877014ca48ca32ff220098cd1 from peer 4c8a3af9b5184a51a3c0fe3221c01c75 at {username='slave'} at 127.9.151.130:58285: session id = 4c8a3af9b5184a51a3c0fe3221c01c75-375647a877014ca48ca32ff220098cd1
W20240503 02:14:14.016359 12205 tablet_copy_service.cc:478] P 7cd9a8da65604af78132e9045dac5180: Error handling TabletCopyService RPC request from {username='slave'} at 127.9.151.130:58285: Error beginning tablet copy session for tablet 375647a877014ca48ca32ff220098cd1: Illegal state: The tablet is not in a running state: BOOTSTRAPPING
I20240503 02:14:14.022662 11899 ts_tablet_manager.cc:902] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: Initiating tablet copy from peer 7cd9a8da65604af78132e9045dac5180 (127.9.151.129:33077)
I20240503 02:14:14.025125 11899 tablet_copy_client.cc:308] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: tablet copy: Beginning tablet copy session from remote peer at address 127.9.151.129:33077
I20240503 02:14:14.026085 12205 tablet_copy_service.cc:140] P 7cd9a8da65604af78132e9045dac5180: Received BeginTabletCopySession request for tablet 375647a877014ca48ca32ff220098cd1 from peer 4c8a3af9b5184a51a3c0fe3221c01c75 ({username='slave'} at 127.9.151.130:58285)
I20240503 02:14:14.026273 12205 tablet_copy_service.cc:161] P 7cd9a8da65604af78132e9045dac5180: Beginning new tablet copy session on tablet 375647a877014ca48ca32ff220098cd1 from peer 4c8a3af9b5184a51a3c0fe3221c01c75 at {username='slave'} at 127.9.151.130:58285: session id = 4c8a3af9b5184a51a3c0fe3221c01c75-375647a877014ca48ca32ff220098cd1
W20240503 02:14:14.026651 12205 tablet_copy_service.cc:478] P 7cd9a8da65604af78132e9045dac5180: Error handling TabletCopyService RPC request from {username='slave'} at 127.9.151.130:58285: Error beginning tablet copy session for tablet 375647a877014ca48ca32ff220098cd1: Illegal state: The tablet is not in a running state: BOOTSTRAPPING
I20240503 02:14:14.035599 11899 ts_tablet_manager.cc:902] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: Initiating tablet copy from peer 7cd9a8da65604af78132e9045dac5180 (127.9.151.129:33077)
I20240503 02:14:14.037575 11899 tablet_copy_client.cc:308] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: tablet copy: Beginning tablet copy session from remote peer at address 127.9.151.129:33077
I20240503 02:14:14.038431 12205 tablet_copy_service.cc:140] P 7cd9a8da65604af78132e9045dac5180: Received BeginTabletCopySession request for tablet 375647a877014ca48ca32ff220098cd1 from peer 4c8a3af9b5184a51a3c0fe3221c01c75 ({username='slave'} at 127.9.151.130:58285)
I20240503 02:14:14.038655 12205 tablet_copy_service.cc:161] P 7cd9a8da65604af78132e9045dac5180: Beginning new tablet copy session on tablet 375647a877014ca48ca32ff220098cd1 from peer 4c8a3af9b5184a51a3c0fe3221c01c75 at {username='slave'} at 127.9.151.130:58285: session id = 4c8a3af9b5184a51a3c0fe3221c01c75-375647a877014ca48ca32ff220098cd1
W20240503 02:14:14.038931 12205 tablet_copy_service.cc:478] P 7cd9a8da65604af78132e9045dac5180: Error handling TabletCopyService RPC request from {username='slave'} at 127.9.151.130:58285: Error beginning tablet copy session for tablet 375647a877014ca48ca32ff220098cd1: Illegal state: The tablet is not in a running state: BOOTSTRAPPING
I20240503 02:14:14.050429 11899 ts_tablet_manager.cc:902] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: Initiating tablet copy from peer 7cd9a8da65604af78132e9045dac5180 (127.9.151.129:33077)
I20240503 02:14:14.051801 11899 tablet_copy_client.cc:308] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: tablet copy: Beginning tablet copy session from remote peer at address 127.9.151.129:33077
I20240503 02:14:14.052697 12205 tablet_copy_service.cc:140] P 7cd9a8da65604af78132e9045dac5180: Received BeginTabletCopySession request for tablet 375647a877014ca48ca32ff220098cd1 from peer 4c8a3af9b5184a51a3c0fe3221c01c75 ({username='slave'} at 127.9.151.130:58285)
I20240503 02:14:14.052865 12205 tablet_copy_service.cc:161] P 7cd9a8da65604af78132e9045dac5180: Beginning new tablet copy session on tablet 375647a877014ca48ca32ff220098cd1 from peer 4c8a3af9b5184a51a3c0fe3221c01c75 at {username='slave'} at 127.9.151.130:58285: session id = 4c8a3af9b5184a51a3c0fe3221c01c75-375647a877014ca48ca32ff220098cd1
W20240503 02:14:14.053105 12205 tablet_copy_service.cc:478] P 7cd9a8da65604af78132e9045dac5180: Error handling TabletCopyService RPC request from {username='slave'} at 127.9.151.130:58285: Error beginning tablet copy session for tablet 375647a877014ca48ca32ff220098cd1: Illegal state: The tablet is not in a running state: BOOTSTRAPPING
I20240503 02:14:14.055541 11886 heartbeater.cc:503] Master 127.9.151.190:36085 was elected leader, sending a full tablet report...
I20240503 02:14:14.056818 11899 ts_tablet_manager.cc:902] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: Initiating tablet copy from peer 7cd9a8da65604af78132e9045dac5180 (127.9.151.129:33077)
I20240503 02:14:14.058557 11899 tablet_copy_client.cc:308] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: tablet copy: Beginning tablet copy session from remote peer at address 127.9.151.129:33077
I20240503 02:14:14.059509 12205 tablet_copy_service.cc:140] P 7cd9a8da65604af78132e9045dac5180: Received BeginTabletCopySession request for tablet 375647a877014ca48ca32ff220098cd1 from peer 4c8a3af9b5184a51a3c0fe3221c01c75 ({username='slave'} at 127.9.151.130:58285)
I20240503 02:14:14.059902 12205 tablet_copy_service.cc:161] P 7cd9a8da65604af78132e9045dac5180: Beginning new tablet copy session on tablet 375647a877014ca48ca32ff220098cd1 from peer 4c8a3af9b5184a51a3c0fe3221c01c75 at {username='slave'} at 127.9.151.130:58285: session id = 4c8a3af9b5184a51a3c0fe3221c01c75-375647a877014ca48ca32ff220098cd1
W20240503 02:14:14.060180 12205 tablet_copy_service.cc:478] P 7cd9a8da65604af78132e9045dac5180: Error handling TabletCopyService RPC request from {username='slave'} at 127.9.151.130:58285: Error beginning tablet copy session for tablet 375647a877014ca48ca32ff220098cd1: Illegal state: The tablet is not in a running state: BOOTSTRAPPING
I20240503 02:14:14.067497 11899 ts_tablet_manager.cc:902] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: Initiating tablet copy from peer 7cd9a8da65604af78132e9045dac5180 (127.9.151.129:33077)
I20240503 02:14:14.075204 11899 tablet_copy_client.cc:308] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: tablet copy: Beginning tablet copy session from remote peer at address 127.9.151.129:33077
I20240503 02:14:14.076311 12205 tablet_copy_service.cc:140] P 7cd9a8da65604af78132e9045dac5180: Received BeginTabletCopySession request for tablet 375647a877014ca48ca32ff220098cd1 from peer 4c8a3af9b5184a51a3c0fe3221c01c75 ({username='slave'} at 127.9.151.130:58285)
I20240503 02:14:14.076552 12205 tablet_copy_service.cc:161] P 7cd9a8da65604af78132e9045dac5180: Beginning new tablet copy session on tablet 375647a877014ca48ca32ff220098cd1 from peer 4c8a3af9b5184a51a3c0fe3221c01c75 at {username='slave'} at 127.9.151.130:58285: session id = 4c8a3af9b5184a51a3c0fe3221c01c75-375647a877014ca48ca32ff220098cd1
W20240503 02:14:14.076786 12205 tablet_copy_service.cc:478] P 7cd9a8da65604af78132e9045dac5180: Error handling TabletCopyService RPC request from {username='slave'} at 127.9.151.130:58285: Error beginning tablet copy session for tablet 375647a877014ca48ca32ff220098cd1: Illegal state: The tablet is not in a running state: BOOTSTRAPPING
I20240503 02:14:14.083237 11899 ts_tablet_manager.cc:902] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: Initiating tablet copy from peer 7cd9a8da65604af78132e9045dac5180 (127.9.151.129:33077)
I20240503 02:14:14.085052 11899 tablet_copy_client.cc:308] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: tablet copy: Beginning tablet copy session from remote peer at address 127.9.151.129:33077
I20240503 02:14:14.085930 12205 tablet_copy_service.cc:140] P 7cd9a8da65604af78132e9045dac5180: Received BeginTabletCopySession request for tablet 375647a877014ca48ca32ff220098cd1 from peer 4c8a3af9b5184a51a3c0fe3221c01c75 ({username='slave'} at 127.9.151.130:58285)
I20240503 02:14:14.086164 12205 tablet_copy_service.cc:161] P 7cd9a8da65604af78132e9045dac5180: Beginning new tablet copy session on tablet 375647a877014ca48ca32ff220098cd1 from peer 4c8a3af9b5184a51a3c0fe3221c01c75 at {username='slave'} at 127.9.151.130:58285: session id = 4c8a3af9b5184a51a3c0fe3221c01c75-375647a877014ca48ca32ff220098cd1
W20240503 02:14:14.086360 12205 tablet_copy_service.cc:478] P 7cd9a8da65604af78132e9045dac5180: Error handling TabletCopyService RPC request from {username='slave'} at 127.9.151.130:58285: Error beginning tablet copy session for tablet 375647a877014ca48ca32ff220098cd1: Illegal state: The tablet is not in a running state: BOOTSTRAPPING
I20240503 02:14:14.093680 11899 ts_tablet_manager.cc:902] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: Initiating tablet copy from peer 7cd9a8da65604af78132e9045dac5180 (127.9.151.129:33077)
I20240503 02:14:14.095382 11899 tablet_copy_client.cc:308] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: tablet copy: Beginning tablet copy session from remote peer at address 127.9.151.129:33077
I20240503 02:14:14.096249 12205 tablet_copy_service.cc:140] P 7cd9a8da65604af78132e9045dac5180: Received BeginTabletCopySession request for tablet 375647a877014ca48ca32ff220098cd1 from peer 4c8a3af9b5184a51a3c0fe3221c01c75 ({username='slave'} at 127.9.151.130:58285)
I20240503 02:14:14.096436 12205 tablet_copy_service.cc:161] P 7cd9a8da65604af78132e9045dac5180: Beginning new tablet copy session on tablet 375647a877014ca48ca32ff220098cd1 from peer 4c8a3af9b5184a51a3c0fe3221c01c75 at {username='slave'} at 127.9.151.130:58285: session id = 4c8a3af9b5184a51a3c0fe3221c01c75-375647a877014ca48ca32ff220098cd1
W20240503 02:14:14.096706 12205 tablet_copy_service.cc:478] P 7cd9a8da65604af78132e9045dac5180: Error handling TabletCopyService RPC request from {username='slave'} at 127.9.151.130:58285: Error beginning tablet copy session for tablet 375647a877014ca48ca32ff220098cd1: Illegal state: The tablet is not in a running state: BOOTSTRAPPING
I20240503 02:14:14.105856 11899 ts_tablet_manager.cc:902] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: Initiating tablet copy from peer 7cd9a8da65604af78132e9045dac5180 (127.9.151.129:33077)
I20240503 02:14:14.107199 11899 tablet_copy_client.cc:308] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: tablet copy: Beginning tablet copy session from remote peer at address 127.9.151.129:33077
I20240503 02:14:14.108047 12205 tablet_copy_service.cc:140] P 7cd9a8da65604af78132e9045dac5180: Received BeginTabletCopySession request for tablet 375647a877014ca48ca32ff220098cd1 from peer 4c8a3af9b5184a51a3c0fe3221c01c75 ({username='slave'} at 127.9.151.130:58285)
I20240503 02:14:14.108297 12205 tablet_copy_service.cc:161] P 7cd9a8da65604af78132e9045dac5180: Beginning new tablet copy session on tablet 375647a877014ca48ca32ff220098cd1 from peer 4c8a3af9b5184a51a3c0fe3221c01c75 at {username='slave'} at 127.9.151.130:58285: session id = 4c8a3af9b5184a51a3c0fe3221c01c75-375647a877014ca48ca32ff220098cd1
W20240503 02:14:14.108553 12205 tablet_copy_service.cc:478] P 7cd9a8da65604af78132e9045dac5180: Error handling TabletCopyService RPC request from {username='slave'} at 127.9.151.130:58285: Error beginning tablet copy session for tablet 375647a877014ca48ca32ff220098cd1: Illegal state: The tablet is not in a running state: BOOTSTRAPPING
I20240503 02:14:14.111490 11899 ts_tablet_manager.cc:902] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: Initiating tablet copy from peer 7cd9a8da65604af78132e9045dac5180 (127.9.151.129:33077)
I20240503 02:14:14.119558 11899 tablet_copy_client.cc:308] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: tablet copy: Beginning tablet copy session from remote peer at address 127.9.151.129:33077
I20240503 02:14:14.120563 12205 tablet_copy_service.cc:140] P 7cd9a8da65604af78132e9045dac5180: Received BeginTabletCopySession request for tablet 375647a877014ca48ca32ff220098cd1 from peer 4c8a3af9b5184a51a3c0fe3221c01c75 ({username='slave'} at 127.9.151.130:58285)
I20240503 02:14:14.120736 12205 tablet_copy_service.cc:161] P 7cd9a8da65604af78132e9045dac5180: Beginning new tablet copy session on tablet 375647a877014ca48ca32ff220098cd1 from peer 4c8a3af9b5184a51a3c0fe3221c01c75 at {username='slave'} at 127.9.151.130:58285: session id = 4c8a3af9b5184a51a3c0fe3221c01c75-375647a877014ca48ca32ff220098cd1
W20240503 02:14:14.121096 12205 tablet_copy_service.cc:478] P 7cd9a8da65604af78132e9045dac5180: Error handling TabletCopyService RPC request from {username='slave'} at 127.9.151.130:58285: Error beginning tablet copy session for tablet 375647a877014ca48ca32ff220098cd1: Illegal state: The tablet is not in a running state: BOOTSTRAPPING
I20240503 02:14:14.134145 11899 ts_tablet_manager.cc:902] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: Initiating tablet copy from peer 7cd9a8da65604af78132e9045dac5180 (127.9.151.129:33077)
I20240503 02:14:14.135529 11899 tablet_copy_client.cc:308] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: tablet copy: Beginning tablet copy session from remote peer at address 127.9.151.129:33077
I20240503 02:14:14.136411 12205 tablet_copy_service.cc:140] P 7cd9a8da65604af78132e9045dac5180: Received BeginTabletCopySession request for tablet 375647a877014ca48ca32ff220098cd1 from peer 4c8a3af9b5184a51a3c0fe3221c01c75 ({username='slave'} at 127.9.151.130:58285)
I20240503 02:14:14.136631 12205 tablet_copy_service.cc:161] P 7cd9a8da65604af78132e9045dac5180: Beginning new tablet copy session on tablet 375647a877014ca48ca32ff220098cd1 from peer 4c8a3af9b5184a51a3c0fe3221c01c75 at {username='slave'} at 127.9.151.130:58285: session id = 4c8a3af9b5184a51a3c0fe3221c01c75-375647a877014ca48ca32ff220098cd1
W20240503 02:14:14.136850 12205 tablet_copy_service.cc:478] P 7cd9a8da65604af78132e9045dac5180: Error handling TabletCopyService RPC request from {username='slave'} at 127.9.151.130:58285: Error beginning tablet copy session for tablet 375647a877014ca48ca32ff220098cd1: Illegal state: The tablet is not in a running state: BOOTSTRAPPING
I20240503 02:14:14.139259 11899 ts_tablet_manager.cc:902] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: Initiating tablet copy from peer 7cd9a8da65604af78132e9045dac5180 (127.9.151.129:33077)
I20240503 02:14:14.142976 11899 tablet_copy_client.cc:308] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: tablet copy: Beginning tablet copy session from remote peer at address 127.9.151.129:33077
I20240503 02:14:14.144088 12205 tablet_copy_service.cc:140] P 7cd9a8da65604af78132e9045dac5180: Received BeginTabletCopySession request for tablet 375647a877014ca48ca32ff220098cd1 from peer 4c8a3af9b5184a51a3c0fe3221c01c75 ({username='slave'} at 127.9.151.130:58285)
I20240503 02:14:14.144331 12205 tablet_copy_service.cc:161] P 7cd9a8da65604af78132e9045dac5180: Beginning new tablet copy session on tablet 375647a877014ca48ca32ff220098cd1 from peer 4c8a3af9b5184a51a3c0fe3221c01c75 at {username='slave'} at 127.9.151.130:58285: session id = 4c8a3af9b5184a51a3c0fe3221c01c75-375647a877014ca48ca32ff220098cd1
W20240503 02:14:14.144547 12205 tablet_copy_service.cc:478] P 7cd9a8da65604af78132e9045dac5180: Error handling TabletCopyService RPC request from {username='slave'} at 127.9.151.130:58285: Error beginning tablet copy session for tablet 375647a877014ca48ca32ff220098cd1: Illegal state: The tablet is not in a running state: BOOTSTRAPPING
I20240503 02:14:14.159799 11899 ts_tablet_manager.cc:902] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: Initiating tablet copy from peer 7cd9a8da65604af78132e9045dac5180 (127.9.151.129:33077)
I20240503 02:14:14.161525 11899 tablet_copy_client.cc:308] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: tablet copy: Beginning tablet copy session from remote peer at address 127.9.151.129:33077
I20240503 02:14:14.162477 12205 tablet_copy_service.cc:140] P 7cd9a8da65604af78132e9045dac5180: Received BeginTabletCopySession request for tablet 375647a877014ca48ca32ff220098cd1 from peer 4c8a3af9b5184a51a3c0fe3221c01c75 ({username='slave'} at 127.9.151.130:58285)
I20240503 02:14:14.162804 12205 tablet_copy_service.cc:161] P 7cd9a8da65604af78132e9045dac5180: Beginning new tablet copy session on tablet 375647a877014ca48ca32ff220098cd1 from peer 4c8a3af9b5184a51a3c0fe3221c01c75 at {username='slave'} at 127.9.151.130:58285: session id = 4c8a3af9b5184a51a3c0fe3221c01c75-375647a877014ca48ca32ff220098cd1
W20240503 02:14:14.163050 12205 tablet_copy_service.cc:478] P 7cd9a8da65604af78132e9045dac5180: Error handling TabletCopyService RPC request from {username='slave'} at 127.9.151.130:58285: Error beginning tablet copy session for tablet 375647a877014ca48ca32ff220098cd1: Illegal state: The tablet is not in a running state: BOOTSTRAPPING
I20240503 02:14:14.168071 11899 ts_tablet_manager.cc:902] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: Initiating tablet copy from peer 7cd9a8da65604af78132e9045dac5180 (127.9.151.129:33077)
I20240503 02:14:14.169375 11899 tablet_copy_client.cc:308] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: tablet copy: Beginning tablet copy session from remote peer at address 127.9.151.129:33077
I20240503 02:14:14.170226 12205 tablet_copy_service.cc:140] P 7cd9a8da65604af78132e9045dac5180: Received BeginTabletCopySession request for tablet 375647a877014ca48ca32ff220098cd1 from peer 4c8a3af9b5184a51a3c0fe3221c01c75 ({username='slave'} at 127.9.151.130:58285)
I20240503 02:14:14.170447 12205 tablet_copy_service.cc:161] P 7cd9a8da65604af78132e9045dac5180: Beginning new tablet copy session on tablet 375647a877014ca48ca32ff220098cd1 from peer 4c8a3af9b5184a51a3c0fe3221c01c75 at {username='slave'} at 127.9.151.130:58285: session id = 4c8a3af9b5184a51a3c0fe3221c01c75-375647a877014ca48ca32ff220098cd1
W20240503 02:14:14.170660 12205 tablet_copy_service.cc:478] P 7cd9a8da65604af78132e9045dac5180: Error handling TabletCopyService RPC request from {username='slave'} at 127.9.151.130:58285: Error beginning tablet copy session for tablet 375647a877014ca48ca32ff220098cd1: Illegal state: The tablet is not in a running state: BOOTSTRAPPING
I20240503 02:14:14.173298 11899 ts_tablet_manager.cc:902] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: Initiating tablet copy from peer 7cd9a8da65604af78132e9045dac5180 (127.9.151.129:33077)
I20240503 02:14:14.174847 11899 tablet_copy_client.cc:308] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: tablet copy: Beginning tablet copy session from remote peer at address 127.9.151.129:33077
I20240503 02:14:14.175827 12205 tablet_copy_service.cc:140] P 7cd9a8da65604af78132e9045dac5180: Received BeginTabletCopySession request for tablet 375647a877014ca48ca32ff220098cd1 from peer 4c8a3af9b5184a51a3c0fe3221c01c75 ({username='slave'} at 127.9.151.130:58285)
I20240503 02:14:14.176045 12205 tablet_copy_service.cc:161] P 7cd9a8da65604af78132e9045dac5180: Beginning new tablet copy session on tablet 375647a877014ca48ca32ff220098cd1 from peer 4c8a3af9b5184a51a3c0fe3221c01c75 at {username='slave'} at 127.9.151.130:58285: session id = 4c8a3af9b5184a51a3c0fe3221c01c75-375647a877014ca48ca32ff220098cd1
W20240503 02:14:14.176252 12205 tablet_copy_service.cc:478] P 7cd9a8da65604af78132e9045dac5180: Error handling TabletCopyService RPC request from {username='slave'} at 127.9.151.130:58285: Error beginning tablet copy session for tablet 375647a877014ca48ca32ff220098cd1: Illegal state: The tablet is not in a running state: BOOTSTRAPPING
I20240503 02:14:14.181901 11899 ts_tablet_manager.cc:902] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: Initiating tablet copy from peer 7cd9a8da65604af78132e9045dac5180 (127.9.151.129:33077)
I20240503 02:14:14.187147 11899 tablet_copy_client.cc:308] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: tablet copy: Beginning tablet copy session from remote peer at address 127.9.151.129:33077
I20240503 02:14:14.188071 12205 tablet_copy_service.cc:140] P 7cd9a8da65604af78132e9045dac5180: Received BeginTabletCopySession request for tablet 375647a877014ca48ca32ff220098cd1 from peer 4c8a3af9b5184a51a3c0fe3221c01c75 ({username='slave'} at 127.9.151.130:58285)
I20240503 02:14:14.188323 12205 tablet_copy_service.cc:161] P 7cd9a8da65604af78132e9045dac5180: Beginning new tablet copy session on tablet 375647a877014ca48ca32ff220098cd1 from peer 4c8a3af9b5184a51a3c0fe3221c01c75 at {username='slave'} at 127.9.151.130:58285: session id = 4c8a3af9b5184a51a3c0fe3221c01c75-375647a877014ca48ca32ff220098cd1
W20240503 02:14:14.188582 12205 tablet_copy_service.cc:478] P 7cd9a8da65604af78132e9045dac5180: Error handling TabletCopyService RPC request from {username='slave'} at 127.9.151.130:58285: Error beginning tablet copy session for tablet 375647a877014ca48ca32ff220098cd1: Illegal state: The tablet is not in a running state: BOOTSTRAPPING
I20240503 02:14:14.192286 11899 ts_tablet_manager.cc:902] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: Initiating tablet copy from peer 7cd9a8da65604af78132e9045dac5180 (127.9.151.129:33077)
I20240503 02:14:14.199815 11899 tablet_copy_client.cc:308] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: tablet copy: Beginning tablet copy session from remote peer at address 127.9.151.129:33077
I20240503 02:14:14.200616 12205 tablet_copy_service.cc:140] P 7cd9a8da65604af78132e9045dac5180: Received BeginTabletCopySession request for tablet 375647a877014ca48ca32ff220098cd1 from peer 4c8a3af9b5184a51a3c0fe3221c01c75 ({username='slave'} at 127.9.151.130:58285)
I20240503 02:14:14.200845 12205 tablet_copy_service.cc:161] P 7cd9a8da65604af78132e9045dac5180: Beginning new tablet copy session on tablet 375647a877014ca48ca32ff220098cd1 from peer 4c8a3af9b5184a51a3c0fe3221c01c75 at {username='slave'} at 127.9.151.130:58285: session id = 4c8a3af9b5184a51a3c0fe3221c01c75-375647a877014ca48ca32ff220098cd1
W20240503 02:14:14.201071 12205 tablet_copy_service.cc:478] P 7cd9a8da65604af78132e9045dac5180: Error handling TabletCopyService RPC request from {username='slave'} at 127.9.151.130:58285: Error beginning tablet copy session for tablet 375647a877014ca48ca32ff220098cd1: Illegal state: The tablet is not in a running state: BOOTSTRAPPING
I20240503 02:14:14.219321 11899 ts_tablet_manager.cc:902] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: Initiating tablet copy from peer 7cd9a8da65604af78132e9045dac5180 (127.9.151.129:33077)
I20240503 02:14:14.220520 11899 tablet_copy_client.cc:308] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: tablet copy: Beginning tablet copy session from remote peer at address 127.9.151.129:33077
I20240503 02:14:14.221531 12205 tablet_copy_service.cc:140] P 7cd9a8da65604af78132e9045dac5180: Received BeginTabletCopySession request for tablet 375647a877014ca48ca32ff220098cd1 from peer 4c8a3af9b5184a51a3c0fe3221c01c75 ({username='slave'} at 127.9.151.130:58285)
I20240503 02:14:14.221729 12205 tablet_copy_service.cc:161] P 7cd9a8da65604af78132e9045dac5180: Beginning new tablet copy session on tablet 375647a877014ca48ca32ff220098cd1 from peer 4c8a3af9b5184a51a3c0fe3221c01c75 at {username='slave'} at 127.9.151.130:58285: session id = 4c8a3af9b5184a51a3c0fe3221c01c75-375647a877014ca48ca32ff220098cd1
W20240503 02:14:14.221949 12205 tablet_copy_service.cc:478] P 7cd9a8da65604af78132e9045dac5180: Error handling TabletCopyService RPC request from {username='slave'} at 127.9.151.130:58285: Error beginning tablet copy session for tablet 375647a877014ca48ca32ff220098cd1: Illegal state: The tablet is not in a running state: BOOTSTRAPPING
I20240503 02:14:14.226218 11899 ts_tablet_manager.cc:902] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: Initiating tablet copy from peer 7cd9a8da65604af78132e9045dac5180 (127.9.151.129:33077)
I20240503 02:14:14.228101 11899 tablet_copy_client.cc:308] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: tablet copy: Beginning tablet copy session from remote peer at address 127.9.151.129:33077
I20240503 02:14:14.229139 12205 tablet_copy_service.cc:140] P 7cd9a8da65604af78132e9045dac5180: Received BeginTabletCopySession request for tablet 375647a877014ca48ca32ff220098cd1 from peer 4c8a3af9b5184a51a3c0fe3221c01c75 ({username='slave'} at 127.9.151.130:58285)
I20240503 02:14:14.229358 12205 tablet_copy_service.cc:161] P 7cd9a8da65604af78132e9045dac5180: Beginning new tablet copy session on tablet 375647a877014ca48ca32ff220098cd1 from peer 4c8a3af9b5184a51a3c0fe3221c01c75 at {username='slave'} at 127.9.151.130:58285: session id = 4c8a3af9b5184a51a3c0fe3221c01c75-375647a877014ca48ca32ff220098cd1
W20240503 02:14:14.229560 12205 tablet_copy_service.cc:478] P 7cd9a8da65604af78132e9045dac5180: Error handling TabletCopyService RPC request from {username='slave'} at 127.9.151.130:58285: Error beginning tablet copy session for tablet 375647a877014ca48ca32ff220098cd1: Illegal state: The tablet is not in a running state: BOOTSTRAPPING
I20240503 02:14:14.235263 11899 ts_tablet_manager.cc:902] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: Initiating tablet copy from peer 7cd9a8da65604af78132e9045dac5180 (127.9.151.129:33077)
I20240503 02:14:14.238304 11899 tablet_copy_client.cc:308] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: tablet copy: Beginning tablet copy session from remote peer at address 127.9.151.129:33077
I20240503 02:14:14.239277 12205 tablet_copy_service.cc:140] P 7cd9a8da65604af78132e9045dac5180: Received BeginTabletCopySession request for tablet 375647a877014ca48ca32ff220098cd1 from peer 4c8a3af9b5184a51a3c0fe3221c01c75 ({username='slave'} at 127.9.151.130:58285)
I20240503 02:14:14.239503 12205 tablet_copy_service.cc:161] P 7cd9a8da65604af78132e9045dac5180: Beginning new tablet copy session on tablet 375647a877014ca48ca32ff220098cd1 from peer 4c8a3af9b5184a51a3c0fe3221c01c75 at {username='slave'} at 127.9.151.130:58285: session id = 4c8a3af9b5184a51a3c0fe3221c01c75-375647a877014ca48ca32ff220098cd1
W20240503 02:14:14.239742 12205 tablet_copy_service.cc:478] P 7cd9a8da65604af78132e9045dac5180: Error handling TabletCopyService RPC request from {username='slave'} at 127.9.151.130:58285: Error beginning tablet copy session for tablet 375647a877014ca48ca32ff220098cd1: Illegal state: The tablet is not in a running state: BOOTSTRAPPING
I20240503 02:14:14.244966 11899 ts_tablet_manager.cc:902] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: Initiating tablet copy from peer 7cd9a8da65604af78132e9045dac5180 (127.9.151.129:33077)
I20240503 02:14:14.246173 11899 tablet_copy_client.cc:308] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: tablet copy: Beginning tablet copy session from remote peer at address 127.9.151.129:33077
I20240503 02:14:14.247087 12205 tablet_copy_service.cc:140] P 7cd9a8da65604af78132e9045dac5180: Received BeginTabletCopySession request for tablet 375647a877014ca48ca32ff220098cd1 from peer 4c8a3af9b5184a51a3c0fe3221c01c75 ({username='slave'} at 127.9.151.130:58285)
I20240503 02:14:14.247313 12205 tablet_copy_service.cc:161] P 7cd9a8da65604af78132e9045dac5180: Beginning new tablet copy session on tablet 375647a877014ca48ca32ff220098cd1 from peer 4c8a3af9b5184a51a3c0fe3221c01c75 at {username='slave'} at 127.9.151.130:58285: session id = 4c8a3af9b5184a51a3c0fe3221c01c75-375647a877014ca48ca32ff220098cd1
W20240503 02:14:14.247502 12205 tablet_copy_service.cc:478] P 7cd9a8da65604af78132e9045dac5180: Error handling TabletCopyService RPC request from {username='slave'} at 127.9.151.130:58285: Error beginning tablet copy session for tablet 375647a877014ca48ca32ff220098cd1: Illegal state: The tablet is not in a running state: BOOTSTRAPPING
I20240503 02:14:14.266009 11899 ts_tablet_manager.cc:902] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: Initiating tablet copy from peer 7cd9a8da65604af78132e9045dac5180 (127.9.151.129:33077)
I20240503 02:14:14.267306 11899 tablet_copy_client.cc:308] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: tablet copy: Beginning tablet copy session from remote peer at address 127.9.151.129:33077
I20240503 02:14:14.268450 12205 tablet_copy_service.cc:140] P 7cd9a8da65604af78132e9045dac5180: Received BeginTabletCopySession request for tablet 375647a877014ca48ca32ff220098cd1 from peer 4c8a3af9b5184a51a3c0fe3221c01c75 ({username='slave'} at 127.9.151.130:58285)
I20240503 02:14:14.268646 12205 tablet_copy_service.cc:161] P 7cd9a8da65604af78132e9045dac5180: Beginning new tablet copy session on tablet 375647a877014ca48ca32ff220098cd1 from peer 4c8a3af9b5184a51a3c0fe3221c01c75 at {username='slave'} at 127.9.151.130:58285: session id = 4c8a3af9b5184a51a3c0fe3221c01c75-375647a877014ca48ca32ff220098cd1
W20240503 02:14:14.268873 12205 tablet_copy_service.cc:478] P 7cd9a8da65604af78132e9045dac5180: Error handling TabletCopyService RPC request from {username='slave'} at 127.9.151.130:58285: Error beginning tablet copy session for tablet 375647a877014ca48ca32ff220098cd1: Illegal state: The tablet is not in a running state: BOOTSTRAPPING
I20240503 02:14:14.271081 11899 ts_tablet_manager.cc:902] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: Initiating tablet copy from peer 7cd9a8da65604af78132e9045dac5180 (127.9.151.129:33077)
I20240503 02:14:14.275306 11899 tablet_copy_client.cc:308] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: tablet copy: Beginning tablet copy session from remote peer at address 127.9.151.129:33077
I20240503 02:14:14.276316 12205 tablet_copy_service.cc:140] P 7cd9a8da65604af78132e9045dac5180: Received BeginTabletCopySession request for tablet 375647a877014ca48ca32ff220098cd1 from peer 4c8a3af9b5184a51a3c0fe3221c01c75 ({username='slave'} at 127.9.151.130:58285)
I20240503 02:14:14.276667 12205 tablet_copy_service.cc:161] P 7cd9a8da65604af78132e9045dac5180: Beginning new tablet copy session on tablet 375647a877014ca48ca32ff220098cd1 from peer 4c8a3af9b5184a51a3c0fe3221c01c75 at {username='slave'} at 127.9.151.130:58285: session id = 4c8a3af9b5184a51a3c0fe3221c01c75-375647a877014ca48ca32ff220098cd1
W20240503 02:14:14.276885 12205 tablet_copy_service.cc:478] P 7cd9a8da65604af78132e9045dac5180: Error handling TabletCopyService RPC request from {username='slave'} at 127.9.151.130:58285: Error beginning tablet copy session for tablet 375647a877014ca48ca32ff220098cd1: Illegal state: The tablet is not in a running state: BOOTSTRAPPING
I20240503 02:14:14.280983 11899 ts_tablet_manager.cc:902] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: Initiating tablet copy from peer 7cd9a8da65604af78132e9045dac5180 (127.9.151.129:33077)
I20240503 02:14:14.282531 11899 tablet_copy_client.cc:308] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: tablet copy: Beginning tablet copy session from remote peer at address 127.9.151.129:33077
I20240503 02:14:14.283531 12205 tablet_copy_service.cc:140] P 7cd9a8da65604af78132e9045dac5180: Received BeginTabletCopySession request for tablet 375647a877014ca48ca32ff220098cd1 from peer 4c8a3af9b5184a51a3c0fe3221c01c75 ({username='slave'} at 127.9.151.130:58285)
I20240503 02:14:14.283712 12205 tablet_copy_service.cc:161] P 7cd9a8da65604af78132e9045dac5180: Beginning new tablet copy session on tablet 375647a877014ca48ca32ff220098cd1 from peer 4c8a3af9b5184a51a3c0fe3221c01c75 at {username='slave'} at 127.9.151.130:58285: session id = 4c8a3af9b5184a51a3c0fe3221c01c75-375647a877014ca48ca32ff220098cd1
W20240503 02:14:14.283954 12205 tablet_copy_service.cc:478] P 7cd9a8da65604af78132e9045dac5180: Error handling TabletCopyService RPC request from {username='slave'} at 127.9.151.130:58285: Error beginning tablet copy session for tablet 375647a877014ca48ca32ff220098cd1: Illegal state: The tablet is not in a running state: BOOTSTRAPPING
I20240503 02:14:14.288446 11899 ts_tablet_manager.cc:902] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: Initiating tablet copy from peer 7cd9a8da65604af78132e9045dac5180 (127.9.151.129:33077)
I20240503 02:14:14.290195 11899 tablet_copy_client.cc:308] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: tablet copy: Beginning tablet copy session from remote peer at address 127.9.151.129:33077
I20240503 02:14:14.290968 12205 tablet_copy_service.cc:140] P 7cd9a8da65604af78132e9045dac5180: Received BeginTabletCopySession request for tablet 375647a877014ca48ca32ff220098cd1 from peer 4c8a3af9b5184a51a3c0fe3221c01c75 ({username='slave'} at 127.9.151.130:58285)
I20240503 02:14:14.291149 12205 tablet_copy_service.cc:161] P 7cd9a8da65604af78132e9045dac5180: Beginning new tablet copy session on tablet 375647a877014ca48ca32ff220098cd1 from peer 4c8a3af9b5184a51a3c0fe3221c01c75 at {username='slave'} at 127.9.151.130:58285: session id = 4c8a3af9b5184a51a3c0fe3221c01c75-375647a877014ca48ca32ff220098cd1
W20240503 02:14:14.291352 12205 tablet_copy_service.cc:478] P 7cd9a8da65604af78132e9045dac5180: Error handling TabletCopyService RPC request from {username='slave'} at 127.9.151.130:58285: Error beginning tablet copy session for tablet 375647a877014ca48ca32ff220098cd1: Illegal state: The tablet is not in a running state: BOOTSTRAPPING
I20240503 02:14:14.306634 11899 ts_tablet_manager.cc:902] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: Initiating tablet copy from peer 7cd9a8da65604af78132e9045dac5180 (127.9.151.129:33077)
I20240503 02:14:14.308104 11899 tablet_copy_client.cc:308] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: tablet copy: Beginning tablet copy session from remote peer at address 127.9.151.129:33077
I20240503 02:14:14.309103 12205 tablet_copy_service.cc:140] P 7cd9a8da65604af78132e9045dac5180: Received BeginTabletCopySession request for tablet 375647a877014ca48ca32ff220098cd1 from peer 4c8a3af9b5184a51a3c0fe3221c01c75 ({username='slave'} at 127.9.151.130:58285)
I20240503 02:14:14.309305 12205 tablet_copy_service.cc:161] P 7cd9a8da65604af78132e9045dac5180: Beginning new tablet copy session on tablet 375647a877014ca48ca32ff220098cd1 from peer 4c8a3af9b5184a51a3c0fe3221c01c75 at {username='slave'} at 127.9.151.130:58285: session id = 4c8a3af9b5184a51a3c0fe3221c01c75-375647a877014ca48ca32ff220098cd1
W20240503 02:14:14.309557 12205 tablet_copy_service.cc:478] P 7cd9a8da65604af78132e9045dac5180: Error handling TabletCopyService RPC request from {username='slave'} at 127.9.151.130:58285: Error beginning tablet copy session for tablet 375647a877014ca48ca32ff220098cd1: Illegal state: The tablet is not in a running state: BOOTSTRAPPING
I20240503 02:14:14.327291 11899 ts_tablet_manager.cc:902] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: Initiating tablet copy from peer 7cd9a8da65604af78132e9045dac5180 (127.9.151.129:33077)
I20240503 02:14:14.328850 11899 tablet_copy_client.cc:308] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: tablet copy: Beginning tablet copy session from remote peer at address 127.9.151.129:33077
I20240503 02:14:14.329924 12205 tablet_copy_service.cc:140] P 7cd9a8da65604af78132e9045dac5180: Received BeginTabletCopySession request for tablet 375647a877014ca48ca32ff220098cd1 from peer 4c8a3af9b5184a51a3c0fe3221c01c75 ({username='slave'} at 127.9.151.130:58285)
I20240503 02:14:14.330209 12205 tablet_copy_service.cc:161] P 7cd9a8da65604af78132e9045dac5180: Beginning new tablet copy session on tablet 375647a877014ca48ca32ff220098cd1 from peer 4c8a3af9b5184a51a3c0fe3221c01c75 at {username='slave'} at 127.9.151.130:58285: session id = 4c8a3af9b5184a51a3c0fe3221c01c75-375647a877014ca48ca32ff220098cd1
W20240503 02:14:14.330469 12205 tablet_copy_service.cc:478] P 7cd9a8da65604af78132e9045dac5180: Error handling TabletCopyService RPC request from {username='slave'} at 127.9.151.130:58285: Error beginning tablet copy session for tablet 375647a877014ca48ca32ff220098cd1: Illegal state: The tablet is not in a running state: BOOTSTRAPPING
I20240503 02:14:14.333905 11899 ts_tablet_manager.cc:902] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: Initiating tablet copy from peer 7cd9a8da65604af78132e9045dac5180 (127.9.151.129:33077)
I20240503 02:14:14.335307 11899 tablet_copy_client.cc:308] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: tablet copy: Beginning tablet copy session from remote peer at address 127.9.151.129:33077
I20240503 02:14:14.336151 12205 tablet_copy_service.cc:140] P 7cd9a8da65604af78132e9045dac5180: Received BeginTabletCopySession request for tablet 375647a877014ca48ca32ff220098cd1 from peer 4c8a3af9b5184a51a3c0fe3221c01c75 ({username='slave'} at 127.9.151.130:58285)
I20240503 02:14:14.336360 12205 tablet_copy_service.cc:161] P 7cd9a8da65604af78132e9045dac5180: Beginning new tablet copy session on tablet 375647a877014ca48ca32ff220098cd1 from peer 4c8a3af9b5184a51a3c0fe3221c01c75 at {username='slave'} at 127.9.151.130:58285: session id = 4c8a3af9b5184a51a3c0fe3221c01c75-375647a877014ca48ca32ff220098cd1
W20240503 02:14:14.336598 12205 tablet_copy_service.cc:478] P 7cd9a8da65604af78132e9045dac5180: Error handling TabletCopyService RPC request from {username='slave'} at 127.9.151.130:58285: Error beginning tablet copy session for tablet 375647a877014ca48ca32ff220098cd1: Illegal state: The tablet is not in a running state: BOOTSTRAPPING
I20240503 02:14:14.352303 11899 ts_tablet_manager.cc:902] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: Initiating tablet copy from peer 7cd9a8da65604af78132e9045dac5180 (127.9.151.129:33077)
I20240503 02:14:14.353760 11899 tablet_copy_client.cc:308] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: tablet copy: Beginning tablet copy session from remote peer at address 127.9.151.129:33077
I20240503 02:14:14.354499 12205 tablet_copy_service.cc:140] P 7cd9a8da65604af78132e9045dac5180: Received BeginTabletCopySession request for tablet 375647a877014ca48ca32ff220098cd1 from peer 4c8a3af9b5184a51a3c0fe3221c01c75 ({username='slave'} at 127.9.151.130:58285)
I20240503 02:14:14.354751 12205 tablet_copy_service.cc:161] P 7cd9a8da65604af78132e9045dac5180: Beginning new tablet copy session on tablet 375647a877014ca48ca32ff220098cd1 from peer 4c8a3af9b5184a51a3c0fe3221c01c75 at {username='slave'} at 127.9.151.130:58285: session id = 4c8a3af9b5184a51a3c0fe3221c01c75-375647a877014ca48ca32ff220098cd1
W20240503 02:14:14.354943 12205 tablet_copy_service.cc:478] P 7cd9a8da65604af78132e9045dac5180: Error handling TabletCopyService RPC request from {username='slave'} at 127.9.151.130:58285: Error beginning tablet copy session for tablet 375647a877014ca48ca32ff220098cd1: Illegal state: The tablet is not in a running state: BOOTSTRAPPING
I20240503 02:14:14.356953 11899 ts_tablet_manager.cc:902] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: Initiating tablet copy from peer 7cd9a8da65604af78132e9045dac5180 (127.9.151.129:33077)
I20240503 02:14:14.358916 11899 tablet_copy_client.cc:308] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: tablet copy: Beginning tablet copy session from remote peer at address 127.9.151.129:33077
I20240503 02:14:14.360136 12205 tablet_copy_service.cc:140] P 7cd9a8da65604af78132e9045dac5180: Received BeginTabletCopySession request for tablet 375647a877014ca48ca32ff220098cd1 from peer 4c8a3af9b5184a51a3c0fe3221c01c75 ({username='slave'} at 127.9.151.130:58285)
I20240503 02:14:14.360332 12205 tablet_copy_service.cc:161] P 7cd9a8da65604af78132e9045dac5180: Beginning new tablet copy session on tablet 375647a877014ca48ca32ff220098cd1 from peer 4c8a3af9b5184a51a3c0fe3221c01c75 at {username='slave'} at 127.9.151.130:58285: session id = 4c8a3af9b5184a51a3c0fe3221c01c75-375647a877014ca48ca32ff220098cd1
W20240503 02:14:14.360543 12205 tablet_copy_service.cc:478] P 7cd9a8da65604af78132e9045dac5180: Error handling TabletCopyService RPC request from {username='slave'} at 127.9.151.130:58285: Error beginning tablet copy session for tablet 375647a877014ca48ca32ff220098cd1: Illegal state: The tablet is not in a running state: BOOTSTRAPPING
I20240503 02:14:14.367496 11899 ts_tablet_manager.cc:902] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: Initiating tablet copy from peer 7cd9a8da65604af78132e9045dac5180 (127.9.151.129:33077)
I20240503 02:14:14.375128 11899 tablet_copy_client.cc:308] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: tablet copy: Beginning tablet copy session from remote peer at address 127.9.151.129:33077
I20240503 02:14:14.376009 12205 tablet_copy_service.cc:140] P 7cd9a8da65604af78132e9045dac5180: Received BeginTabletCopySession request for tablet 375647a877014ca48ca32ff220098cd1 from peer 4c8a3af9b5184a51a3c0fe3221c01c75 ({username='slave'} at 127.9.151.130:58285)
I20240503 02:14:14.376233 12205 tablet_copy_service.cc:161] P 7cd9a8da65604af78132e9045dac5180: Beginning new tablet copy session on tablet 375647a877014ca48ca32ff220098cd1 from peer 4c8a3af9b5184a51a3c0fe3221c01c75 at {username='slave'} at 127.9.151.130:58285: session id = 4c8a3af9b5184a51a3c0fe3221c01c75-375647a877014ca48ca32ff220098cd1
W20240503 02:14:14.376451 12205 tablet_copy_service.cc:478] P 7cd9a8da65604af78132e9045dac5180: Error handling TabletCopyService RPC request from {username='slave'} at 127.9.151.130:58285: Error beginning tablet copy session for tablet 375647a877014ca48ca32ff220098cd1: Illegal state: The tablet is not in a running state: BOOTSTRAPPING
I20240503 02:14:14.379626 11899 ts_tablet_manager.cc:902] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: Initiating tablet copy from peer 7cd9a8da65604af78132e9045dac5180 (127.9.151.129:33077)
I20240503 02:14:14.380841 11899 tablet_copy_client.cc:308] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: tablet copy: Beginning tablet copy session from remote peer at address 127.9.151.129:33077
I20240503 02:14:14.381584 12205 tablet_copy_service.cc:140] P 7cd9a8da65604af78132e9045dac5180: Received BeginTabletCopySession request for tablet 375647a877014ca48ca32ff220098cd1 from peer 4c8a3af9b5184a51a3c0fe3221c01c75 ({username='slave'} at 127.9.151.130:58285)
I20240503 02:14:14.381758 12205 tablet_copy_service.cc:161] P 7cd9a8da65604af78132e9045dac5180: Beginning new tablet copy session on tablet 375647a877014ca48ca32ff220098cd1 from peer 4c8a3af9b5184a51a3c0fe3221c01c75 at {username='slave'} at 127.9.151.130:58285: session id = 4c8a3af9b5184a51a3c0fe3221c01c75-375647a877014ca48ca32ff220098cd1
W20240503 02:14:14.381966 12205 tablet_copy_service.cc:478] P 7cd9a8da65604af78132e9045dac5180: Error handling TabletCopyService RPC request from {username='slave'} at 127.9.151.130:58285: Error beginning tablet copy session for tablet 375647a877014ca48ca32ff220098cd1: Illegal state: The tablet is not in a running state: BOOTSTRAPPING
I20240503 02:14:14.392874 11899 ts_tablet_manager.cc:902] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: Initiating tablet copy from peer 7cd9a8da65604af78132e9045dac5180 (127.9.151.129:33077)
I20240503 02:14:14.395155 11899 tablet_copy_client.cc:308] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: tablet copy: Beginning tablet copy session from remote peer at address 127.9.151.129:33077
I20240503 02:14:14.395929 12205 tablet_copy_service.cc:140] P 7cd9a8da65604af78132e9045dac5180: Received BeginTabletCopySession request for tablet 375647a877014ca48ca32ff220098cd1 from peer 4c8a3af9b5184a51a3c0fe3221c01c75 ({username='slave'} at 127.9.151.130:58285)
I20240503 02:14:14.396231 12205 tablet_copy_service.cc:161] P 7cd9a8da65604af78132e9045dac5180: Beginning new tablet copy session on tablet 375647a877014ca48ca32ff220098cd1 from peer 4c8a3af9b5184a51a3c0fe3221c01c75 at {username='slave'} at 127.9.151.130:58285: session id = 4c8a3af9b5184a51a3c0fe3221c01c75-375647a877014ca48ca32ff220098cd1
W20240503 02:14:14.396422 12205 tablet_copy_service.cc:478] P 7cd9a8da65604af78132e9045dac5180: Error handling TabletCopyService RPC request from {username='slave'} at 127.9.151.130:58285: Error beginning tablet copy session for tablet 375647a877014ca48ca32ff220098cd1: Illegal state: The tablet is not in a running state: BOOTSTRAPPING
I20240503 02:14:14.398232 11899 ts_tablet_manager.cc:902] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: Initiating tablet copy from peer 7cd9a8da65604af78132e9045dac5180 (127.9.151.129:33077)
I20240503 02:14:14.399464 11899 tablet_copy_client.cc:308] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: tablet copy: Beginning tablet copy session from remote peer at address 127.9.151.129:33077
I20240503 02:14:14.400156 12205 tablet_copy_service.cc:140] P 7cd9a8da65604af78132e9045dac5180: Received BeginTabletCopySession request for tablet 375647a877014ca48ca32ff220098cd1 from peer 4c8a3af9b5184a51a3c0fe3221c01c75 ({username='slave'} at 127.9.151.130:58285)
I20240503 02:14:14.400337 12205 tablet_copy_service.cc:161] P 7cd9a8da65604af78132e9045dac5180: Beginning new tablet copy session on tablet 375647a877014ca48ca32ff220098cd1 from peer 4c8a3af9b5184a51a3c0fe3221c01c75 at {username='slave'} at 127.9.151.130:58285: session id = 4c8a3af9b5184a51a3c0fe3221c01c75-375647a877014ca48ca32ff220098cd1
W20240503 02:14:14.400538 12205 tablet_copy_service.cc:478] P 7cd9a8da65604af78132e9045dac5180: Error handling TabletCopyService RPC request from {username='slave'} at 127.9.151.130:58285: Error beginning tablet copy session for tablet 375647a877014ca48ca32ff220098cd1: Illegal state: The tablet is not in a running state: BOOTSTRAPPING
I20240503 02:14:14.406656 11899 ts_tablet_manager.cc:902] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: Initiating tablet copy from peer 7cd9a8da65604af78132e9045dac5180 (127.9.151.129:33077)
I20240503 02:14:14.408524 11899 tablet_copy_client.cc:308] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: tablet copy: Beginning tablet copy session from remote peer at address 127.9.151.129:33077
I20240503 02:14:14.409271 12205 tablet_copy_service.cc:140] P 7cd9a8da65604af78132e9045dac5180: Received BeginTabletCopySession request for tablet 375647a877014ca48ca32ff220098cd1 from peer 4c8a3af9b5184a51a3c0fe3221c01c75 ({username='slave'} at 127.9.151.130:58285)
I20240503 02:14:14.409521 12205 tablet_copy_service.cc:161] P 7cd9a8da65604af78132e9045dac5180: Beginning new tablet copy session on tablet 375647a877014ca48ca32ff220098cd1 from peer 4c8a3af9b5184a51a3c0fe3221c01c75 at {username='slave'} at 127.9.151.130:58285: session id = 4c8a3af9b5184a51a3c0fe3221c01c75-375647a877014ca48ca32ff220098cd1
W20240503 02:14:14.409761 12205 tablet_copy_service.cc:478] P 7cd9a8da65604af78132e9045dac5180: Error handling TabletCopyService RPC request from {username='slave'} at 127.9.151.130:58285: Error beginning tablet copy session for tablet 375647a877014ca48ca32ff220098cd1: Illegal state: The tablet is not in a running state: BOOTSTRAPPING
I20240503 02:14:14.416520 11899 ts_tablet_manager.cc:902] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: Initiating tablet copy from peer 7cd9a8da65604af78132e9045dac5180 (127.9.151.129:33077)
I20240503 02:14:14.418183 11899 tablet_copy_client.cc:308] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: tablet copy: Beginning tablet copy session from remote peer at address 127.9.151.129:33077
I20240503 02:14:14.418936 12205 tablet_copy_service.cc:140] P 7cd9a8da65604af78132e9045dac5180: Received BeginTabletCopySession request for tablet 375647a877014ca48ca32ff220098cd1 from peer 4c8a3af9b5184a51a3c0fe3221c01c75 ({username='slave'} at 127.9.151.130:58285)
I20240503 02:14:14.419123 12205 tablet_copy_service.cc:161] P 7cd9a8da65604af78132e9045dac5180: Beginning new tablet copy session on tablet 375647a877014ca48ca32ff220098cd1 from peer 4c8a3af9b5184a51a3c0fe3221c01c75 at {username='slave'} at 127.9.151.130:58285: session id = 4c8a3af9b5184a51a3c0fe3221c01c75-375647a877014ca48ca32ff220098cd1
W20240503 02:14:14.419337 12205 tablet_copy_service.cc:478] P 7cd9a8da65604af78132e9045dac5180: Error handling TabletCopyService RPC request from {username='slave'} at 127.9.151.130:58285: Error beginning tablet copy session for tablet 375647a877014ca48ca32ff220098cd1: Illegal state: The tablet is not in a running state: BOOTSTRAPPING
I20240503 02:14:14.423777 11899 ts_tablet_manager.cc:902] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: Initiating tablet copy from peer 7cd9a8da65604af78132e9045dac5180 (127.9.151.129:33077)
I20240503 02:14:14.425562 11899 tablet_copy_client.cc:308] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: tablet copy: Beginning tablet copy session from remote peer at address 127.9.151.129:33077
I20240503 02:14:14.426355 12205 tablet_copy_service.cc:140] P 7cd9a8da65604af78132e9045dac5180: Received BeginTabletCopySession request for tablet 375647a877014ca48ca32ff220098cd1 from peer 4c8a3af9b5184a51a3c0fe3221c01c75 ({username='slave'} at 127.9.151.130:58285)
I20240503 02:14:14.426545 12205 tablet_copy_service.cc:161] P 7cd9a8da65604af78132e9045dac5180: Beginning new tablet copy session on tablet 375647a877014ca48ca32ff220098cd1 from peer 4c8a3af9b5184a51a3c0fe3221c01c75 at {username='slave'} at 127.9.151.130:58285: session id = 4c8a3af9b5184a51a3c0fe3221c01c75-375647a877014ca48ca32ff220098cd1
W20240503 02:14:14.426779 12205 tablet_copy_service.cc:478] P 7cd9a8da65604af78132e9045dac5180: Error handling TabletCopyService RPC request from {username='slave'} at 127.9.151.130:58285: Error beginning tablet copy session for tablet 375647a877014ca48ca32ff220098cd1: Illegal state: The tablet is not in a running state: BOOTSTRAPPING
I20240503 02:14:14.442247 11899 ts_tablet_manager.cc:902] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: Initiating tablet copy from peer 7cd9a8da65604af78132e9045dac5180 (127.9.151.129:33077)
I20240503 02:14:14.443651 11899 tablet_copy_client.cc:308] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: tablet copy: Beginning tablet copy session from remote peer at address 127.9.151.129:33077
I20240503 02:14:14.444366 12205 tablet_copy_service.cc:140] P 7cd9a8da65604af78132e9045dac5180: Received BeginTabletCopySession request for tablet 375647a877014ca48ca32ff220098cd1 from peer 4c8a3af9b5184a51a3c0fe3221c01c75 ({username='slave'} at 127.9.151.130:58285)
I20240503 02:14:14.444521 12205 tablet_copy_service.cc:161] P 7cd9a8da65604af78132e9045dac5180: Beginning new tablet copy session on tablet 375647a877014ca48ca32ff220098cd1 from peer 4c8a3af9b5184a51a3c0fe3221c01c75 at {username='slave'} at 127.9.151.130:58285: session id = 4c8a3af9b5184a51a3c0fe3221c01c75-375647a877014ca48ca32ff220098cd1
W20240503 02:14:14.444846 12205 tablet_copy_service.cc:478] P 7cd9a8da65604af78132e9045dac5180: Error handling TabletCopyService RPC request from {username='slave'} at 127.9.151.130:58285: Error beginning tablet copy session for tablet 375647a877014ca48ca32ff220098cd1: Illegal state: The tablet is not in a running state: BOOTSTRAPPING
I20240503 02:14:14.459790 11899 ts_tablet_manager.cc:902] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: Initiating tablet copy from peer 7cd9a8da65604af78132e9045dac5180 (127.9.151.129:33077)
I20240503 02:14:14.461278 11899 tablet_copy_client.cc:308] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: tablet copy: Beginning tablet copy session from remote peer at address 127.9.151.129:33077
I20240503 02:14:14.461967 12205 tablet_copy_service.cc:140] P 7cd9a8da65604af78132e9045dac5180: Received BeginTabletCopySession request for tablet 375647a877014ca48ca32ff220098cd1 from peer 4c8a3af9b5184a51a3c0fe3221c01c75 ({username='slave'} at 127.9.151.130:58285)
I20240503 02:14:14.462144 12205 tablet_copy_service.cc:161] P 7cd9a8da65604af78132e9045dac5180: Beginning new tablet copy session on tablet 375647a877014ca48ca32ff220098cd1 from peer 4c8a3af9b5184a51a3c0fe3221c01c75 at {username='slave'} at 127.9.151.130:58285: session id = 4c8a3af9b5184a51a3c0fe3221c01c75-375647a877014ca48ca32ff220098cd1
W20240503 02:14:14.462330 12205 tablet_copy_service.cc:478] P 7cd9a8da65604af78132e9045dac5180: Error handling TabletCopyService RPC request from {username='slave'} at 127.9.151.130:58285: Error beginning tablet copy session for tablet 375647a877014ca48ca32ff220098cd1: Illegal state: The tablet is not in a running state: BOOTSTRAPPING
I20240503 02:14:14.474438 11899 ts_tablet_manager.cc:902] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: Initiating tablet copy from peer 7cd9a8da65604af78132e9045dac5180 (127.9.151.129:33077)
I20240503 02:14:14.476604 11899 tablet_copy_client.cc:308] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: tablet copy: Beginning tablet copy session from remote peer at address 127.9.151.129:33077
I20240503 02:14:14.477370 12205 tablet_copy_service.cc:140] P 7cd9a8da65604af78132e9045dac5180: Received BeginTabletCopySession request for tablet 375647a877014ca48ca32ff220098cd1 from peer 4c8a3af9b5184a51a3c0fe3221c01c75 ({username='slave'} at 127.9.151.130:58285)
I20240503 02:14:14.477532 12205 tablet_copy_service.cc:161] P 7cd9a8da65604af78132e9045dac5180: Beginning new tablet copy session on tablet 375647a877014ca48ca32ff220098cd1 from peer 4c8a3af9b5184a51a3c0fe3221c01c75 at {username='slave'} at 127.9.151.130:58285: session id = 4c8a3af9b5184a51a3c0fe3221c01c75-375647a877014ca48ca32ff220098cd1
W20240503 02:14:14.477736 12205 tablet_copy_service.cc:478] P 7cd9a8da65604af78132e9045dac5180: Error handling TabletCopyService RPC request from {username='slave'} at 127.9.151.130:58285: Error beginning tablet copy session for tablet 375647a877014ca48ca32ff220098cd1: Illegal state: The tablet is not in a running state: BOOTSTRAPPING
I20240503 02:14:14.481017 11899 ts_tablet_manager.cc:902] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: Initiating tablet copy from peer 7cd9a8da65604af78132e9045dac5180 (127.9.151.129:33077)
I20240503 02:14:14.482215 11899 tablet_copy_client.cc:308] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: tablet copy: Beginning tablet copy session from remote peer at address 127.9.151.129:33077
I20240503 02:14:14.482945 12205 tablet_copy_service.cc:140] P 7cd9a8da65604af78132e9045dac5180: Received BeginTabletCopySession request for tablet 375647a877014ca48ca32ff220098cd1 from peer 4c8a3af9b5184a51a3c0fe3221c01c75 ({username='slave'} at 127.9.151.130:58285)
I20240503 02:14:14.483124 12205 tablet_copy_service.cc:161] P 7cd9a8da65604af78132e9045dac5180: Beginning new tablet copy session on tablet 375647a877014ca48ca32ff220098cd1 from peer 4c8a3af9b5184a51a3c0fe3221c01c75 at {username='slave'} at 127.9.151.130:58285: session id = 4c8a3af9b5184a51a3c0fe3221c01c75-375647a877014ca48ca32ff220098cd1
W20240503 02:14:14.483335 12205 tablet_copy_service.cc:478] P 7cd9a8da65604af78132e9045dac5180: Error handling TabletCopyService RPC request from {username='slave'} at 127.9.151.130:58285: Error beginning tablet copy session for tablet 375647a877014ca48ca32ff220098cd1: Illegal state: The tablet is not in a running state: BOOTSTRAPPING
I20240503 02:14:14.488706 11899 ts_tablet_manager.cc:902] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: Initiating tablet copy from peer 7cd9a8da65604af78132e9045dac5180 (127.9.151.129:33077)
I20240503 02:14:14.491389 11899 tablet_copy_client.cc:308] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: tablet copy: Beginning tablet copy session from remote peer at address 127.9.151.129:33077
I20240503 02:14:14.492372 12205 tablet_copy_service.cc:140] P 7cd9a8da65604af78132e9045dac5180: Received BeginTabletCopySession request for tablet 375647a877014ca48ca32ff220098cd1 from peer 4c8a3af9b5184a51a3c0fe3221c01c75 ({username='slave'} at 127.9.151.130:58285)
I20240503 02:14:14.492569 12205 tablet_copy_service.cc:161] P 7cd9a8da65604af78132e9045dac5180: Beginning new tablet copy session on tablet 375647a877014ca48ca32ff220098cd1 from peer 4c8a3af9b5184a51a3c0fe3221c01c75 at {username='slave'} at 127.9.151.130:58285: session id = 4c8a3af9b5184a51a3c0fe3221c01c75-375647a877014ca48ca32ff220098cd1
W20240503 02:14:14.492774 12205 tablet_copy_service.cc:478] P 7cd9a8da65604af78132e9045dac5180: Error handling TabletCopyService RPC request from {username='slave'} at 127.9.151.130:58285: Error beginning tablet copy session for tablet 375647a877014ca48ca32ff220098cd1: Illegal state: The tablet is not in a running state: BOOTSTRAPPING
I20240503 02:14:14.516855 11899 ts_tablet_manager.cc:902] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: Initiating tablet copy from peer 7cd9a8da65604af78132e9045dac5180 (127.9.151.129:33077)
I20240503 02:14:14.520992 11899 tablet_copy_client.cc:308] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: tablet copy: Beginning tablet copy session from remote peer at address 127.9.151.129:33077
I20240503 02:14:14.521982 12205 tablet_copy_service.cc:140] P 7cd9a8da65604af78132e9045dac5180: Received BeginTabletCopySession request for tablet 375647a877014ca48ca32ff220098cd1 from peer 4c8a3af9b5184a51a3c0fe3221c01c75 ({username='slave'} at 127.9.151.130:58285)
I20240503 02:14:14.522194 12205 tablet_copy_service.cc:161] P 7cd9a8da65604af78132e9045dac5180: Beginning new tablet copy session on tablet 375647a877014ca48ca32ff220098cd1 from peer 4c8a3af9b5184a51a3c0fe3221c01c75 at {username='slave'} at 127.9.151.130:58285: session id = 4c8a3af9b5184a51a3c0fe3221c01c75-375647a877014ca48ca32ff220098cd1
W20240503 02:14:14.522364 12205 tablet_copy_service.cc:478] P 7cd9a8da65604af78132e9045dac5180: Error handling TabletCopyService RPC request from {username='slave'} at 127.9.151.130:58285: Error beginning tablet copy session for tablet 375647a877014ca48ca32ff220098cd1: Illegal state: The tablet is not in a running state: BOOTSTRAPPING
I20240503 02:14:14.525084 11899 ts_tablet_manager.cc:902] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: Initiating tablet copy from peer 7cd9a8da65604af78132e9045dac5180 (127.9.151.129:33077)
I20240503 02:14:14.526916 11899 tablet_copy_client.cc:308] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: tablet copy: Beginning tablet copy session from remote peer at address 127.9.151.129:33077
I20240503 02:14:14.527755 12205 tablet_copy_service.cc:140] P 7cd9a8da65604af78132e9045dac5180: Received BeginTabletCopySession request for tablet 375647a877014ca48ca32ff220098cd1 from peer 4c8a3af9b5184a51a3c0fe3221c01c75 ({username='slave'} at 127.9.151.130:58285)
I20240503 02:14:14.527915 12205 tablet_copy_service.cc:161] P 7cd9a8da65604af78132e9045dac5180: Beginning new tablet copy session on tablet 375647a877014ca48ca32ff220098cd1 from peer 4c8a3af9b5184a51a3c0fe3221c01c75 at {username='slave'} at 127.9.151.130:58285: session id = 4c8a3af9b5184a51a3c0fe3221c01c75-375647a877014ca48ca32ff220098cd1
W20240503 02:14:14.528118 12205 tablet_copy_service.cc:478] P 7cd9a8da65604af78132e9045dac5180: Error handling TabletCopyService RPC request from {username='slave'} at 127.9.151.130:58285: Error beginning tablet copy session for tablet 375647a877014ca48ca32ff220098cd1: Illegal state: The tablet is not in a running state: BOOTSTRAPPING
I20240503 02:14:14.530230 11899 ts_tablet_manager.cc:902] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: Initiating tablet copy from peer 7cd9a8da65604af78132e9045dac5180 (127.9.151.129:33077)
I20240503 02:14:14.535256 11899 tablet_copy_client.cc:308] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: tablet copy: Beginning tablet copy session from remote peer at address 127.9.151.129:33077
I20240503 02:14:14.536037 12205 tablet_copy_service.cc:140] P 7cd9a8da65604af78132e9045dac5180: Received BeginTabletCopySession request for tablet 375647a877014ca48ca32ff220098cd1 from peer 4c8a3af9b5184a51a3c0fe3221c01c75 ({username='slave'} at 127.9.151.130:58285)
I20240503 02:14:14.536211 12205 tablet_copy_service.cc:161] P 7cd9a8da65604af78132e9045dac5180: Beginning new tablet copy session on tablet 375647a877014ca48ca32ff220098cd1 from peer 4c8a3af9b5184a51a3c0fe3221c01c75 at {username='slave'} at 127.9.151.130:58285: session id = 4c8a3af9b5184a51a3c0fe3221c01c75-375647a877014ca48ca32ff220098cd1
W20240503 02:14:14.536406 12205 tablet_copy_service.cc:478] P 7cd9a8da65604af78132e9045dac5180: Error handling TabletCopyService RPC request from {username='slave'} at 127.9.151.130:58285: Error beginning tablet copy session for tablet 375647a877014ca48ca32ff220098cd1: Illegal state: The tablet is not in a running state: BOOTSTRAPPING
I20240503 02:14:14.547729 11899 ts_tablet_manager.cc:902] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: Initiating tablet copy from peer 7cd9a8da65604af78132e9045dac5180 (127.9.151.129:33077)
I20240503 02:14:14.549333 11899 tablet_copy_client.cc:308] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: tablet copy: Beginning tablet copy session from remote peer at address 127.9.151.129:33077
I20240503 02:14:14.550257 12205 tablet_copy_service.cc:140] P 7cd9a8da65604af78132e9045dac5180: Received BeginTabletCopySession request for tablet 375647a877014ca48ca32ff220098cd1 from peer 4c8a3af9b5184a51a3c0fe3221c01c75 ({username='slave'} at 127.9.151.130:58285)
I20240503 02:14:14.550518 12205 tablet_copy_service.cc:161] P 7cd9a8da65604af78132e9045dac5180: Beginning new tablet copy session on tablet 375647a877014ca48ca32ff220098cd1 from peer 4c8a3af9b5184a51a3c0fe3221c01c75 at {username='slave'} at 127.9.151.130:58285: session id = 4c8a3af9b5184a51a3c0fe3221c01c75-375647a877014ca48ca32ff220098cd1
W20240503 02:14:14.550815 12205 tablet_copy_service.cc:478] P 7cd9a8da65604af78132e9045dac5180: Error handling TabletCopyService RPC request from {username='slave'} at 127.9.151.130:58285: Error beginning tablet copy session for tablet 375647a877014ca48ca32ff220098cd1: Illegal state: The tablet is not in a running state: BOOTSTRAPPING
I20240503 02:14:14.555334 11899 ts_tablet_manager.cc:902] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: Initiating tablet copy from peer 7cd9a8da65604af78132e9045dac5180 (127.9.151.129:33077)
I20240503 02:14:14.556620 11899 tablet_copy_client.cc:308] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: tablet copy: Beginning tablet copy session from remote peer at address 127.9.151.129:33077
I20240503 02:14:14.557377 12205 tablet_copy_service.cc:140] P 7cd9a8da65604af78132e9045dac5180: Received BeginTabletCopySession request for tablet 375647a877014ca48ca32ff220098cd1 from peer 4c8a3af9b5184a51a3c0fe3221c01c75 ({username='slave'} at 127.9.151.130:58285)
I20240503 02:14:14.557598 12205 tablet_copy_service.cc:161] P 7cd9a8da65604af78132e9045dac5180: Beginning new tablet copy session on tablet 375647a877014ca48ca32ff220098cd1 from peer 4c8a3af9b5184a51a3c0fe3221c01c75 at {username='slave'} at 127.9.151.130:58285: session id = 4c8a3af9b5184a51a3c0fe3221c01c75-375647a877014ca48ca32ff220098cd1
W20240503 02:14:14.557813 12205 tablet_copy_service.cc:478] P 7cd9a8da65604af78132e9045dac5180: Error handling TabletCopyService RPC request from {username='slave'} at 127.9.151.130:58285: Error beginning tablet copy session for tablet 375647a877014ca48ca32ff220098cd1: Illegal state: The tablet is not in a running state: BOOTSTRAPPING
I20240503 02:14:14.576102 12095 tablet_bootstrap.cc:492] T 375647a877014ca48ca32ff220098cd1 P 7cd9a8da65604af78132e9045dac5180: Bootstrap replayed 1/1 log segments. Stats: ops{read=214 overwritten=0 applied=214 ignored=0} inserts{seen=10550 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240503 02:14:14.576722 12095 tablet_bootstrap.cc:492] T 375647a877014ca48ca32ff220098cd1 P 7cd9a8da65604af78132e9045dac5180: Bootstrap complete.
I20240503 02:14:14.578114 12095 ts_tablet_manager.cc:1367] T 375647a877014ca48ca32ff220098cd1 P 7cd9a8da65604af78132e9045dac5180: Time spent bootstrapping tablet: real 0.844s user 0.775s sys 0.060s
I20240503 02:14:14.578436 11899 ts_tablet_manager.cc:902] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: Initiating tablet copy from peer 7cd9a8da65604af78132e9045dac5180 (127.9.151.129:33077)
I20240503 02:14:14.583185 11899 tablet_copy_client.cc:308] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: tablet copy: Beginning tablet copy session from remote peer at address 127.9.151.129:33077
I20240503 02:14:14.583323 12095 raft_consensus.cc:348] T 375647a877014ca48ca32ff220098cd1 P 7cd9a8da65604af78132e9045dac5180 [term 3 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7cd9a8da65604af78132e9045dac5180" member_type: VOTER last_known_addr { host: "127.9.151.129" port: 33077 } }
I20240503 02:14:14.584463 12205 tablet_copy_service.cc:140] P 7cd9a8da65604af78132e9045dac5180: Received BeginTabletCopySession request for tablet 375647a877014ca48ca32ff220098cd1 from peer 4c8a3af9b5184a51a3c0fe3221c01c75 ({username='slave'} at 127.9.151.130:58285)
I20240503 02:14:14.584482 12095 raft_consensus.cc:729] T 375647a877014ca48ca32ff220098cd1 P 7cd9a8da65604af78132e9045dac5180 [term 3 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7cd9a8da65604af78132e9045dac5180, State: Initialized, Role: FOLLOWER
I20240503 02:14:14.584699 12205 tablet_copy_service.cc:161] P 7cd9a8da65604af78132e9045dac5180: Beginning new tablet copy session on tablet 375647a877014ca48ca32ff220098cd1 from peer 4c8a3af9b5184a51a3c0fe3221c01c75 at {username='slave'} at 127.9.151.130:58285: session id = 4c8a3af9b5184a51a3c0fe3221c01c75-375647a877014ca48ca32ff220098cd1
W20240503 02:14:14.584967 12205 tablet_copy_service.cc:478] P 7cd9a8da65604af78132e9045dac5180: Error handling TabletCopyService RPC request from {username='slave'} at 127.9.151.130:58285: Error beginning tablet copy session for tablet 375647a877014ca48ca32ff220098cd1: Illegal state: The tablet is not in a running state: BOOTSTRAPPING
I20240503 02:14:14.585527 12095 consensus_queue.cc:260] T 375647a877014ca48ca32ff220098cd1 P 7cd9a8da65604af78132e9045dac5180 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 214, Last appended: 3.214, Last appended by leader: 214, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7cd9a8da65604af78132e9045dac5180" member_type: VOTER last_known_addr { host: "127.9.151.129" port: 33077 } }
I20240503 02:14:14.585842 12095 raft_consensus.cc:388] T 375647a877014ca48ca32ff220098cd1 P 7cd9a8da65604af78132e9045dac5180 [term 3 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20240503 02:14:14.585992 12095 raft_consensus.cc:482] T 375647a877014ca48ca32ff220098cd1 P 7cd9a8da65604af78132e9045dac5180 [term 3 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20240503 02:14:14.586225 12095 raft_consensus.cc:3037] T 375647a877014ca48ca32ff220098cd1 P 7cd9a8da65604af78132e9045dac5180 [term 3 FOLLOWER]: Advancing to term 4
I20240503 02:14:14.588735 11899 ts_tablet_manager.cc:902] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: Initiating tablet copy from peer 7cd9a8da65604af78132e9045dac5180 (127.9.151.129:33077)
I20240503 02:14:14.591024 11899 tablet_copy_client.cc:308] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: tablet copy: Beginning tablet copy session from remote peer at address 127.9.151.129:33077
I20240503 02:14:14.591514 12095 raft_consensus.cc:504] T 375647a877014ca48ca32ff220098cd1 P 7cd9a8da65604af78132e9045dac5180 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7cd9a8da65604af78132e9045dac5180" member_type: VOTER last_known_addr { host: "127.9.151.129" port: 33077 } }
I20240503 02:14:14.592073 12205 tablet_copy_service.cc:140] P 7cd9a8da65604af78132e9045dac5180: Received BeginTabletCopySession request for tablet 375647a877014ca48ca32ff220098cd1 from peer 4c8a3af9b5184a51a3c0fe3221c01c75 ({username='slave'} at 127.9.151.130:58285)
I20240503 02:14:14.592271 12205 tablet_copy_service.cc:161] P 7cd9a8da65604af78132e9045dac5180: Beginning new tablet copy session on tablet 375647a877014ca48ca32ff220098cd1 from peer 4c8a3af9b5184a51a3c0fe3221c01c75 at {username='slave'} at 127.9.151.130:58285: session id = 4c8a3af9b5184a51a3c0fe3221c01c75-375647a877014ca48ca32ff220098cd1
I20240503 02:14:14.592397 12095 leader_election.cc:302] T 375647a877014ca48ca32ff220098cd1 P 7cd9a8da65604af78132e9045dac5180 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7cd9a8da65604af78132e9045dac5180; no voters:
W20240503 02:14:14.592509 12205 tablet_copy_service.cc:478] P 7cd9a8da65604af78132e9045dac5180: Error handling TabletCopyService RPC request from {username='slave'} at 127.9.151.130:58285: Error beginning tablet copy session for tablet 375647a877014ca48ca32ff220098cd1: Illegal state: The tablet is not in a running state: BOOTSTRAPPING
I20240503 02:14:14.593252 12095 leader_election.cc:288] T 375647a877014ca48ca32ff220098cd1 P 7cd9a8da65604af78132e9045dac5180 [CANDIDATE]: Term 4 election: Requested vote from peers
I20240503 02:14:14.593412 12472 raft_consensus.cc:2781] T 375647a877014ca48ca32ff220098cd1 P 7cd9a8da65604af78132e9045dac5180 [term 4 FOLLOWER]: Leader election won for term 4
I20240503 02:14:14.594897 12472 raft_consensus.cc:686] T 375647a877014ca48ca32ff220098cd1 P 7cd9a8da65604af78132e9045dac5180 [term 4 LEADER]: Becoming Leader. State: Replica: 7cd9a8da65604af78132e9045dac5180, State: Running, Role: LEADER
I20240503 02:14:14.595281 12242 heartbeater.cc:503] Master 127.9.151.190:36085 was elected leader, sending a full tablet report...
I20240503 02:14:14.595525 12472 consensus_queue.cc:237] T 375647a877014ca48ca32ff220098cd1 P 7cd9a8da65604af78132e9045dac5180 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 214, Committed index: 214, Last appended: 3.214, Last appended by leader: 214, Current term: 4, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7cd9a8da65604af78132e9045dac5180" member_type: VOTER last_known_addr { host: "127.9.151.129" port: 33077 } }
I20240503 02:14:14.595805 12095 ts_tablet_manager.cc:1398] T 375647a877014ca48ca32ff220098cd1 P 7cd9a8da65604af78132e9045dac5180: Time spent starting tablet: real 0.018s user 0.021s sys 0.000s
I20240503 02:14:14.599587 10221 catalog_manager.cc:5484] T 375647a877014ca48ca32ff220098cd1 P 7cd9a8da65604af78132e9045dac5180 reported cstate change: term changed from 3 to 4. New cstate: current_term: 4 leader_uuid: "7cd9a8da65604af78132e9045dac5180" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7cd9a8da65604af78132e9045dac5180" member_type: VOTER last_known_addr { host: "127.9.151.129" port: 33077 } health_report { overall_health: HEALTHY } } }
I20240503 02:14:14.599917 11899 ts_tablet_manager.cc:902] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: Initiating tablet copy from peer 7cd9a8da65604af78132e9045dac5180 (127.9.151.129:33077)
I20240503 02:14:14.603062 11899 tablet_copy_client.cc:308] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: tablet copy: Beginning tablet copy session from remote peer at address 127.9.151.129:33077
I20240503 02:14:14.604224 12205 tablet_copy_service.cc:140] P 7cd9a8da65604af78132e9045dac5180: Received BeginTabletCopySession request for tablet 375647a877014ca48ca32ff220098cd1 from peer 4c8a3af9b5184a51a3c0fe3221c01c75 ({username='slave'} at 127.9.151.130:58285)
I20240503 02:14:14.604449 12205 tablet_copy_service.cc:161] P 7cd9a8da65604af78132e9045dac5180: Beginning new tablet copy session on tablet 375647a877014ca48ca32ff220098cd1 from peer 4c8a3af9b5184a51a3c0fe3221c01c75 at {username='slave'} at 127.9.151.130:58285: session id = 4c8a3af9b5184a51a3c0fe3221c01c75-375647a877014ca48ca32ff220098cd1
I20240503 02:14:14.607582 12205 tablet_copy_source_session.cc:205] T 375647a877014ca48ca32ff220098cd1 P 7cd9a8da65604af78132e9045dac5180: Tablet Copy: opened 0 blocks and 1 log segments
I20240503 02:14:14.612803 11899 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 375647a877014ca48ca32ff220098cd1. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:14:14.625770 11899 tablet_copy_client.cc:783] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: tablet copy: Starting download of 0 data blocks...
I20240503 02:14:14.626328 11899 tablet_copy_client.cc:647] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: tablet copy: Starting download of 1 WAL segments...
I20240503 02:14:14.632122 11899 tablet_copy_client.cc:523] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240503 02:14:14.636561 11899 tablet_bootstrap.cc:492] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: Bootstrap starting.
I20240503 02:14:14.712977 11899 log.cc:824] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: Log is configured to *not* fsync() on all Append() calls
I20240503 02:14:15.350512 11899 tablet_bootstrap.cc:492] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: Bootstrap replayed 1/1 log segments. Stats: ops{read=215 overwritten=0 applied=215 ignored=0} inserts{seen=10550 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240503 02:14:15.351241 11899 tablet_bootstrap.cc:492] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: Bootstrap complete.
I20240503 02:14:15.351869 11899 ts_tablet_manager.cc:1367] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: Time spent bootstrapping tablet: real 0.716s user 0.653s sys 0.060s
I20240503 02:14:15.358628 11899 raft_consensus.cc:348] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75 [term 4 NON_PARTICIPANT]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7cd9a8da65604af78132e9045dac5180" member_type: VOTER last_known_addr { host: "127.9.151.129" port: 33077 } }
I20240503 02:14:15.359177 11899 raft_consensus.cc:729] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75 [term 4 NON_PARTICIPANT]: Becoming Follower/Learner. State: Replica: 4c8a3af9b5184a51a3c0fe3221c01c75, State: Initialized, Role: NON_PARTICIPANT
I20240503 02:14:15.359865 11899 consensus_queue.cc:260] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 215, Last appended: 4.215, Last appended by leader: 215, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7cd9a8da65604af78132e9045dac5180" member_type: VOTER last_known_addr { host: "127.9.151.129" port: 33077 } }
I20240503 02:14:15.361372 11899 ts_tablet_manager.cc:1398] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: Time spent starting tablet: real 0.009s user 0.005s sys 0.004s
I20240503 02:14:15.362334 12205 tablet_copy_service.cc:342] P 7cd9a8da65604af78132e9045dac5180: Request end of tablet copy session 4c8a3af9b5184a51a3c0fe3221c01c75-375647a877014ca48ca32ff220098cd1 received from {username='slave'} at 127.9.151.130:58285
I20240503 02:14:15.362473 12205 tablet_copy_service.cc:433] P 7cd9a8da65604af78132e9045dac5180: ending tablet copy session 4c8a3af9b5184a51a3c0fe3221c01c75-375647a877014ca48ca32ff220098cd1 on tablet 375647a877014ca48ca32ff220098cd1 with peer 4c8a3af9b5184a51a3c0fe3221c01c75
W20240503 02:14:15.368579 11899 ts_tablet_manager.cc:701] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: Tablet Copy: Invalid argument: Leader has replica of tablet 375647a877014ca48ca32ff220098cd1 with term 0, which is lower than last-logged term 4 on local replica. Rejecting tablet copy request
W20240503 02:14:15.375911 11899 ts_tablet_manager.cc:701] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: Tablet Copy: Invalid argument: Leader has replica of tablet 375647a877014ca48ca32ff220098cd1 with term 0, which is lower than last-logged term 4 on local replica. Rejecting tablet copy request
W20240503 02:14:15.402555 11899 ts_tablet_manager.cc:701] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: Tablet Copy: Invalid argument: Leader has replica of tablet 375647a877014ca48ca32ff220098cd1 with term 0, which is lower than last-logged term 4 on local replica. Rejecting tablet copy request
I20240503 02:14:15.405898 11821 tablet_service.cc:1514] Processing DeleteTablet for tablet 375647a877014ca48ca32ff220098cd1 with delete_type TABLET_DATA_DELETED from {username='slave'} at 127.0.0.1:55326
I20240503 02:14:15.406828 12488 tablet_replica.cc:330] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: stopping tablet replica
I20240503 02:14:15.407375 12488 raft_consensus.cc:2229] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75 [term 4 NON_PARTICIPANT]: Raft consensus shutting down.
I20240503 02:14:15.407729 12488 raft_consensus.cc:2258] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75 [term 4 NON_PARTICIPANT]: Raft consensus is shut down!
I20240503 02:14:15.409003 12488 ts_tablet_manager.cc:1875] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20240503 02:14:15.414332 12488 ts_tablet_manager.cc:1888] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 4.215
I20240503 02:14:15.414532 12488 log.cc:1196] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: Deleting WAL directory at /tmp/dist-test-tasknrxqHk/test-tmp/tablet_copy_client_session-itest.0.TabletCopyClientSessionITest.TestStartTabletCopyWhileSourceBootstrapping.1714702442467385-9822-0/minicluster-data/ts-1/wal/wals/375647a877014ca48ca32ff220098cd1
I20240503 02:14:15.415148 12488 ts_tablet_manager.cc:1909] T 375647a877014ca48ca32ff220098cd1 P 4c8a3af9b5184a51a3c0fe3221c01c75: Deleting consensus metadata
I20240503 02:14:15.417219 9822 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasknrxqHk/build/asan/bin/kudu pid 11759
I20240503 02:14:15.418897 11881 generic_service.cc:196] Checking for leaks (request via RPC)
I20240503 02:14:15.530236 9822 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknrxqHk/build/asan/bin/kudu with pid 11759
I20240503 02:14:15.553522 9822 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknrxqHk/build/asan/bin/kudu
/tmp/dist-test-tasknrxqHk/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknrxqHk/test-tmp/tablet_copy_client_session-itest.0.TabletCopyClientSessionITest.TestStartTabletCopyWhileSourceBootstrapping.1714702442467385-9822-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasknrxqHk/test-tmp/tablet_copy_client_session-itest.0.TabletCopyClientSessionITest.TestStartTabletCopyWhileSourceBootstrapping.1714702442467385-9822-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknrxqHk/test-tmp/tablet_copy_client_session-itest.0.TabletCopyClientSessionITest.TestStartTabletCopyWhileSourceBootstrapping.1714702442467385-9822-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasknrxqHk/test-tmp/tablet_copy_client_session-itest.0.TabletCopyClientSessionITest.TestStartTabletCopyWhileSourceBootstrapping.1714702442467385-9822-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.151.130:35753
--local_ip_for_outbound_sockets=127.9.151.130
--tserver_master_addrs=127.9.151.190:36085
--webserver_port=42537
--webserver_interface=127.9.151.130
--builtin_ntp_servers=127.9.151.148:38865
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin with env {}
W20240503 02:14:15.919853 12490 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240503 02:14:15.920183 12490 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240503 02:14:15.920298 12490 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240503 02:14:15.928406 12490 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240503 02:14:15.928627 12490 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.151.130
I20240503 02:14:15.938187 12490 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.151.148:38865
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknrxqHk/test-tmp/tablet_copy_client_session-itest.0.TabletCopyClientSessionITest.TestStartTabletCopyWhileSourceBootstrapping.1714702442467385-9822-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasknrxqHk/test-tmp/tablet_copy_client_session-itest.0.TabletCopyClientSessionITest.TestStartTabletCopyWhileSourceBootstrapping.1714702442467385-9822-0/minicluster-data/ts-1/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.151.130:35753
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknrxqHk/test-tmp/tablet_copy_client_session-itest.0.TabletCopyClientSessionITest.TestStartTabletCopyWhileSourceBootstrapping.1714702442467385-9822-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.151.130
--webserver_port=42537
--tserver_master_addrs=127.9.151.190:36085
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.151.130
--log_dir=/tmp/dist-test-tasknrxqHk/test-tmp/tablet_copy_client_session-itest.0.TabletCopyClientSessionITest.TestStartTabletCopyWhileSourceBootstrapping.1714702442467385-9822-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c39e72a98e124e5a12080bac42105b996594d806
build type FASTDEBUG
built by None at 03 May 2024 01:43:22 UTC on efd1337a2aa3
build id 99
ASAN enabled
I20240503 02:14:15.939770 12490 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240503 02:14:15.941684 12490 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240503 02:14:15.956468 12495 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240503 02:14:15.957032 12498 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240503 02:14:15.957640 12496 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240503 02:14:15.958400 12490 server_base.cc:1034] running on GCE node
I20240503 02:14:15.959509 12490 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240503 02:14:15.960943 12490 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240503 02:14:15.962229 12490 hybrid_clock.cc:648] HybridClock initialized: now 1714702455962168 us; error 59 us; skew 500 ppm
I20240503 02:14:15.962742 12490 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20240503 02:14:15.979264 12490 webserver.cc:460] Webserver started at http://127.9.151.130:42537/ using document root <none> and password file <none>
I20240503 02:14:15.980396 12490 fs_manager.cc:359] Metadata directory not provided
I20240503 02:14:15.980564 12490 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240503 02:14:15.987394 12490 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.001s
I20240503 02:14:15.991286 12504 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240503 02:14:15.993268 12490 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20240503 02:14:15.993481 12490 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknrxqHk/test-tmp/tablet_copy_client_session-itest.0.TabletCopyClientSessionITest.TestStartTabletCopyWhileSourceBootstrapping.1714702442467385-9822-0/minicluster-data/ts-1/data,/tmp/dist-test-tasknrxqHk/test-tmp/tablet_copy_client_session-itest.0.TabletCopyClientSessionITest.TestStartTabletCopyWhileSourceBootstrapping.1714702442467385-9822-0/minicluster-data/ts-1/wal
uuid: "4c8a3af9b5184a51a3c0fe3221c01c75"
format_stamp: "Formatted at 2024-05-03 02:14:04 on dist-test-slave-twlh"
I20240503 02:14:15.994772 12490 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknrxqHk/test-tmp/tablet_copy_client_session-itest.0.TabletCopyClientSessionITest.TestStartTabletCopyWhileSourceBootstrapping.1714702442467385-9822-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasknrxqHk/test-tmp/tablet_copy_client_session-itest.0.TabletCopyClientSessionITest.TestStartTabletCopyWhileSourceBootstrapping.1714702442467385-9822-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasknrxqHk/test-tmp/tablet_copy_client_session-itest.0.TabletCopyClientSessionITest.TestStartTabletCopyWhileSourceBootstrapping.1714702442467385-9822-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240503 02:14:16.025614 12490 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240503 02:14:16.026542 12490 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240503 02:14:16.026854 12490 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240503 02:14:16.027946 12490 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240503 02:14:16.029729 12490 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240503 02:14:16.029835 12490 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240503 02:14:16.029919 12490 ts_tablet_manager.cc:590] Registered 0 tablets
I20240503 02:14:16.029991 12490 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240503 02:14:16.068013 12490 rpc_server.cc:307] RPC server started. Bound to: 127.9.151.130:35753
I20240503 02:14:16.068113 12616 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.151.130:35753 every 8 connection(s)
I20240503 02:14:16.069651 12490 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknrxqHk/test-tmp/tablet_copy_client_session-itest.0.TabletCopyClientSessionITest.TestStartTabletCopyWhileSourceBootstrapping.1714702442467385-9822-0/minicluster-data/ts-1/data/info.pb
I20240503 02:14:16.076846 9822 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknrxqHk/build/asan/bin/kudu as pid 12490
I20240503 02:14:16.077318 9822 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasknrxqHk/build/asan/bin/kudu pid 11898
I20240503 02:14:16.085973 12617 heartbeater.cc:347] Connected to a master server at 127.9.151.190:36085
I20240503 02:14:16.086364 12617 heartbeater.cc:464] Registering TS with master...
I20240503 02:14:16.087028 12237 generic_service.cc:196] Checking for leaks (request via RPC)
I20240503 02:14:16.087644 12617 heartbeater.cc:511] Master 127.9.151.190:36085 requested a full tablet report, sending...
I20240503 02:14:16.089856 10221 ts_manager.cc:194] Re-registered known tserver with Master: 4c8a3af9b5184a51a3c0fe3221c01c75 (127.9.151.130:35753)
I20240503 02:14:16.091370 10221 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.9.151.130:40727
I20240503 02:14:16.196220 9822 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknrxqHk/build/asan/bin/kudu with pid 11898
I20240503 02:14:16.217610 9822 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasknrxqHk/build/asan/bin/kudu pid 12490
I20240503 02:14:16.228794 12612 generic_service.cc:196] Checking for leaks (request via RPC)
I20240503 02:14:16.316088 9822 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknrxqHk/build/asan/bin/kudu with pid 12490
I20240503 02:14:16.337191 9822 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasknrxqHk/build/asan/bin/kudu pid 10191
I20240503 02:14:16.338922 10253 generic_service.cc:196] Checking for leaks (request via RPC)
I20240503 02:14:16.458354 9822 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknrxqHk/build/asan/bin/kudu with pid 10191
2024-05-03T02:14:16Z chronyd exiting
[ OK ] TabletCopyClientSessionITest.TestStartTabletCopyWhileSourceBootstrapping (13957 ms)
[ RUN ] TabletCopyClientSessionITest.TestStartTabletCopy
2024-05-03T02:14:16Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2024-05-03T02:14:16Z Disabled control of system clock
I20240503 02:14:16.533368 9822 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknrxqHk/build/asan/bin/kudu
/tmp/dist-test-tasknrxqHk/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknrxqHk/test-tmp/tablet_copy_client_session-itest.0.TabletCopyClientSessionITest.TestStartTabletCopy.1714702442467385-9822-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasknrxqHk/test-tmp/tablet_copy_client_session-itest.0.TabletCopyClientSessionITest.TestStartTabletCopy.1714702442467385-9822-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknrxqHk/test-tmp/tablet_copy_client_session-itest.0.TabletCopyClientSessionITest.TestStartTabletCopy.1714702442467385-9822-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasknrxqHk/test-tmp/tablet_copy_client_session-itest.0.TabletCopyClientSessionITest.TestStartTabletCopy.1714702442467385-9822-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.151.190:43557
--webserver_interface=127.9.151.190
--webserver_port=0
--builtin_ntp_servers=127.9.151.148:43997
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.151.190:43557
--master_tombstone_evicted_tablet_replicas=false with env {}
W20240503 02:14:16.896162 12630 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240503 02:14:16.896636 12630 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240503 02:14:16.896816 12630 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240503 02:14:16.906062 12630 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20240503 02:14:16.906241 12630 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240503 02:14:16.906299 12630 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20240503 02:14:16.906404 12630 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20240503 02:14:16.917392 12630 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.151.148:43997
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknrxqHk/test-tmp/tablet_copy_client_session-itest.0.TabletCopyClientSessionITest.TestStartTabletCopy.1714702442467385-9822-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasknrxqHk/test-tmp/tablet_copy_client_session-itest.0.TabletCopyClientSessionITest.TestStartTabletCopy.1714702442467385-9822-0/minicluster-data/master-0/wal
--master_tombstone_evicted_tablet_replicas=false
--ipki_ca_key_size=768
--master_addresses=127.9.151.190:43557
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.151.190:43557
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknrxqHk/test-tmp/tablet_copy_client_session-itest.0.TabletCopyClientSessionITest.TestStartTabletCopy.1714702442467385-9822-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.151.190
--webserver_port=0
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasknrxqHk/test-tmp/tablet_copy_client_session-itest.0.TabletCopyClientSessionITest.TestStartTabletCopy.1714702442467385-9822-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c39e72a98e124e5a12080bac42105b996594d806
build type FASTDEBUG
built by None at 03 May 2024 01:43:22 UTC on efd1337a2aa3
build id 99
ASAN enabled
I20240503 02:14:16.919405 12630 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240503 02:14:16.921589 12630 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240503 02:14:16.935218 12635 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240503 02:14:16.935757 12636 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240503 02:14:16.936236 12638 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240503 02:14:16.937175 12630 server_base.cc:1034] running on GCE node
I20240503 02:14:16.938123 12630 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240503 02:14:16.940011 12630 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240503 02:14:16.941365 12630 hybrid_clock.cc:648] HybridClock initialized: now 1714702456941270 us; error 70 us; skew 500 ppm
I20240503 02:14:16.941792 12630 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20240503 02:14:16.944840 12630 webserver.cc:460] Webserver started at http://127.9.151.190:44151/ using document root <none> and password file <none>
I20240503 02:14:16.945904 12630 fs_manager.cc:359] Metadata directory not provided
I20240503 02:14:16.946066 12630 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240503 02:14:16.946476 12630 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240503 02:14:16.950001 12630 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknrxqHk/test-tmp/tablet_copy_client_session-itest.0.TabletCopyClientSessionITest.TestStartTabletCopy.1714702442467385-9822-0/minicluster-data/master-0/data/instance:
uuid: "2340184c71764af6a09921f4cb722805"
format_stamp: "Formatted at 2024-05-03 02:14:16 on dist-test-slave-twlh"
I20240503 02:14:16.951073 12630 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknrxqHk/test-tmp/tablet_copy_client_session-itest.0.TabletCopyClientSessionITest.TestStartTabletCopy.1714702442467385-9822-0/minicluster-data/master-0/wal/instance:
uuid: "2340184c71764af6a09921f4cb722805"
format_stamp: "Formatted at 2024-05-03 02:14:16 on dist-test-slave-twlh"
I20240503 02:14:16.958483 12630 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.005s sys 0.000s
I20240503 02:14:16.963212 12644 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240503 02:14:16.965163 12630 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.002s
I20240503 02:14:16.965420 12630 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknrxqHk/test-tmp/tablet_copy_client_session-itest.0.TabletCopyClientSessionITest.TestStartTabletCopy.1714702442467385-9822-0/minicluster-data/master-0/data,/tmp/dist-test-tasknrxqHk/test-tmp/tablet_copy_client_session-itest.0.TabletCopyClientSessionITest.TestStartTabletCopy.1714702442467385-9822-0/minicluster-data/master-0/wal
uuid: "2340184c71764af6a09921f4cb722805"
format_stamp: "Formatted at 2024-05-03 02:14:16 on dist-test-slave-twlh"
I20240503 02:14:16.965657 12630 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknrxqHk/test-tmp/tablet_copy_client_session-itest.0.TabletCopyClientSessionITest.TestStartTabletCopy.1714702442467385-9822-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasknrxqHk/test-tmp/tablet_copy_client_session-itest.0.TabletCopyClientSessionITest.TestStartTabletCopy.1714702442467385-9822-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasknrxqHk/test-tmp/tablet_copy_client_session-itest.0.TabletCopyClientSessionITest.TestStartTabletCopy.1714702442467385-9822-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240503 02:14:16.988905 12630 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240503 02:14:16.989854 12630 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240503 02:14:16.990171 12630 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240503 02:14:17.013231 12630 rpc_server.cc:307] RPC server started. Bound to: 127.9.151.190:43557
I20240503 02:14:17.013253 12695 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.151.190:43557 every 8 connection(s)
I20240503 02:14:17.014881 12630 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknrxqHk/test-tmp/tablet_copy_client_session-itest.0.TabletCopyClientSessionITest.TestStartTabletCopy.1714702442467385-9822-0/minicluster-data/master-0/data/info.pb
I20240503 02:14:17.019526 9822 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknrxqHk/build/asan/bin/kudu as pid 12630
I20240503 02:14:17.019658 12696 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:14:17.019830 9822 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknrxqHk/test-tmp/tablet_copy_client_session-itest.0.TabletCopyClientSessionITest.TestStartTabletCopy.1714702442467385-9822-0/minicluster-data/master-0/wal/instance
I20240503 02:14:17.034452 12696 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2340184c71764af6a09921f4cb722805: Bootstrap starting.
I20240503 02:14:17.038028 12696 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2340184c71764af6a09921f4cb722805: Neither blocks nor log segments found. Creating new log.
I20240503 02:14:17.039232 12696 log.cc:824] T 00000000000000000000000000000000 P 2340184c71764af6a09921f4cb722805: Log is configured to *not* fsync() on all Append() calls
I20240503 02:14:17.042779 12696 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2340184c71764af6a09921f4cb722805: No bootstrap required, opened a new log
I20240503 02:14:17.049530 12696 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2340184c71764af6a09921f4cb722805 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2340184c71764af6a09921f4cb722805" member_type: VOTER last_known_addr { host: "127.9.151.190" port: 43557 } }
I20240503 02:14:17.049932 12696 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2340184c71764af6a09921f4cb722805 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:14:17.050053 12696 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2340184c71764af6a09921f4cb722805 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2340184c71764af6a09921f4cb722805, State: Initialized, Role: FOLLOWER
I20240503 02:14:17.050778 12696 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2340184c71764af6a09921f4cb722805 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2340184c71764af6a09921f4cb722805" member_type: VOTER last_known_addr { host: "127.9.151.190" port: 43557 } }
I20240503 02:14:17.050985 12696 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2340184c71764af6a09921f4cb722805 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20240503 02:14:17.051138 12696 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2340184c71764af6a09921f4cb722805 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20240503 02:14:17.051302 12696 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2340184c71764af6a09921f4cb722805 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:14:17.053505 12696 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2340184c71764af6a09921f4cb722805 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2340184c71764af6a09921f4cb722805" member_type: VOTER last_known_addr { host: "127.9.151.190" port: 43557 } }
I20240503 02:14:17.054096 12696 leader_election.cc:302] T 00000000000000000000000000000000 P 2340184c71764af6a09921f4cb722805 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2340184c71764af6a09921f4cb722805; no voters:
I20240503 02:14:17.054836 12696 leader_election.cc:288] T 00000000000000000000000000000000 P 2340184c71764af6a09921f4cb722805 [CANDIDATE]: Term 1 election: Requested vote from peers
I20240503 02:14:17.055490 12701 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2340184c71764af6a09921f4cb722805 [term 1 FOLLOWER]: Leader election won for term 1
I20240503 02:14:17.056753 12701 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2340184c71764af6a09921f4cb722805 [term 1 LEADER]: Becoming Leader. State: Replica: 2340184c71764af6a09921f4cb722805, State: Running, Role: LEADER
I20240503 02:14:17.057502 12701 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2340184c71764af6a09921f4cb722805 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2340184c71764af6a09921f4cb722805" member_type: VOTER last_known_addr { host: "127.9.151.190" port: 43557 } }
I20240503 02:14:17.057729 12696 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2340184c71764af6a09921f4cb722805 [sys.catalog]: configured and running, proceeding with master startup.
I20240503 02:14:17.060784 12702 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2340184c71764af6a09921f4cb722805 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2340184c71764af6a09921f4cb722805" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2340184c71764af6a09921f4cb722805" member_type: VOTER last_known_addr { host: "127.9.151.190" port: 43557 } } }
I20240503 02:14:17.061233 12702 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2340184c71764af6a09921f4cb722805 [sys.catalog]: This master's current role is: LEADER
I20240503 02:14:17.063220 12703 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2340184c71764af6a09921f4cb722805 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2340184c71764af6a09921f4cb722805. Latest consensus state: current_term: 1 leader_uuid: "2340184c71764af6a09921f4cb722805" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2340184c71764af6a09921f4cb722805" member_type: VOTER last_known_addr { host: "127.9.151.190" port: 43557 } } }
I20240503 02:14:17.063627 12707 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20240503 02:14:17.063772 12703 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2340184c71764af6a09921f4cb722805 [sys.catalog]: This master's current role is: LEADER
I20240503 02:14:17.070750 12707 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20240503 02:14:17.083235 12707 catalog_manager.cc:1347] Generated new cluster ID: 0013c8b6660d40fd9bcf5c82b8dcb7bc
I20240503 02:14:17.083439 12707 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20240503 02:14:17.119477 12707 catalog_manager.cc:1370] Generated new certificate authority record
I20240503 02:14:17.121807 12707 catalog_manager.cc:1504] Loading token signing keys...
I20240503 02:14:17.136287 12707 catalog_manager.cc:5857] T 00000000000000000000000000000000 P 2340184c71764af6a09921f4cb722805: Generated new TSK 0
I20240503 02:14:17.137535 12707 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20240503 02:14:17.151156 9822 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknrxqHk/build/asan/bin/kudu
/tmp/dist-test-tasknrxqHk/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknrxqHk/test-tmp/tablet_copy_client_session-itest.0.TabletCopyClientSessionITest.TestStartTabletCopy.1714702442467385-9822-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasknrxqHk/test-tmp/tablet_copy_client_session-itest.0.TabletCopyClientSessionITest.TestStartTabletCopy.1714702442467385-9822-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknrxqHk/test-tmp/tablet_copy_client_session-itest.0.TabletCopyClientSessionITest.TestStartTabletCopy.1714702442467385-9822-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasknrxqHk/test-tmp/tablet_copy_client_session-itest.0.TabletCopyClientSessionITest.TestStartTabletCopy.1714702442467385-9822-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.151.129:0
--local_ip_for_outbound_sockets=127.9.151.129
--webserver_interface=127.9.151.129
--webserver_port=0
--tserver_master_addrs=127.9.151.190:43557
--builtin_ntp_servers=127.9.151.148:43997
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin with env {}
W20240503 02:14:17.546205 12720 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240503 02:14:17.546638 12720 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240503 02:14:17.547024 12720 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240503 02:14:17.555513 12720 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240503 02:14:17.555781 12720 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.151.129
I20240503 02:14:17.565865 12720 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.151.148:43997
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknrxqHk/test-tmp/tablet_copy_client_session-itest.0.TabletCopyClientSessionITest.TestStartTabletCopy.1714702442467385-9822-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasknrxqHk/test-tmp/tablet_copy_client_session-itest.0.TabletCopyClientSessionITest.TestStartTabletCopy.1714702442467385-9822-0/minicluster-data/ts-0/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.151.129:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknrxqHk/test-tmp/tablet_copy_client_session-itest.0.TabletCopyClientSessionITest.TestStartTabletCopy.1714702442467385-9822-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.151.129
--webserver_port=0
--tserver_master_addrs=127.9.151.190:43557
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.151.129
--log_dir=/tmp/dist-test-tasknrxqHk/test-tmp/tablet_copy_client_session-itest.0.TabletCopyClientSessionITest.TestStartTabletCopy.1714702442467385-9822-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c39e72a98e124e5a12080bac42105b996594d806
build type FASTDEBUG
built by None at 03 May 2024 01:43:22 UTC on efd1337a2aa3
build id 99
ASAN enabled
I20240503 02:14:17.567479 12720 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240503 02:14:17.569487 12720 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240503 02:14:17.584038 12726 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240503 02:14:17.584726 12728 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240503 02:14:17.585131 12725 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240503 02:14:17.586026 12720 server_base.cc:1034] running on GCE node
I20240503 02:14:17.587153 12720 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240503 02:14:17.588748 12720 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240503 02:14:17.590066 12720 hybrid_clock.cc:648] HybridClock initialized: now 1714702457589958 us; error 75 us; skew 500 ppm
I20240503 02:14:17.590477 12720 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20240503 02:14:17.593405 12720 webserver.cc:460] Webserver started at http://127.9.151.129:46705/ using document root <none> and password file <none>
I20240503 02:14:17.594269 12720 fs_manager.cc:359] Metadata directory not provided
I20240503 02:14:17.594427 12720 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240503 02:14:17.594818 12720 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240503 02:14:17.597849 12720 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknrxqHk/test-tmp/tablet_copy_client_session-itest.0.TabletCopyClientSessionITest.TestStartTabletCopy.1714702442467385-9822-0/minicluster-data/ts-0/data/instance:
uuid: "e07a4ec83c724757b7f34e9f22517cf2"
format_stamp: "Formatted at 2024-05-03 02:14:17 on dist-test-slave-twlh"
I20240503 02:14:17.598799 12720 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknrxqHk/test-tmp/tablet_copy_client_session-itest.0.TabletCopyClientSessionITest.TestStartTabletCopy.1714702442467385-9822-0/minicluster-data/ts-0/wal/instance:
uuid: "e07a4ec83c724757b7f34e9f22517cf2"
format_stamp: "Formatted at 2024-05-03 02:14:17 on dist-test-slave-twlh"
I20240503 02:14:17.604879 12720 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.004s sys 0.001s
I20240503 02:14:17.610146 12734 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240503 02:14:17.611963 12720 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20240503 02:14:17.612216 12720 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknrxqHk/test-tmp/tablet_copy_client_session-itest.0.TabletCopyClientSessionITest.TestStartTabletCopy.1714702442467385-9822-0/minicluster-data/ts-0/data,/tmp/dist-test-tasknrxqHk/test-tmp/tablet_copy_client_session-itest.0.TabletCopyClientSessionITest.TestStartTabletCopy.1714702442467385-9822-0/minicluster-data/ts-0/wal
uuid: "e07a4ec83c724757b7f34e9f22517cf2"
format_stamp: "Formatted at 2024-05-03 02:14:17 on dist-test-slave-twlh"
I20240503 02:14:17.612448 12720 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknrxqHk/test-tmp/tablet_copy_client_session-itest.0.TabletCopyClientSessionITest.TestStartTabletCopy.1714702442467385-9822-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasknrxqHk/test-tmp/tablet_copy_client_session-itest.0.TabletCopyClientSessionITest.TestStartTabletCopy.1714702442467385-9822-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasknrxqHk/test-tmp/tablet_copy_client_session-itest.0.TabletCopyClientSessionITest.TestStartTabletCopy.1714702442467385-9822-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240503 02:14:17.635929 12720 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240503 02:14:17.636950 12720 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240503 02:14:17.637322 12720 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240503 02:14:17.638383 12720 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240503 02:14:17.640164 12720 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240503 02:14:17.640255 12720 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240503 02:14:17.640393 12720 ts_tablet_manager.cc:590] Registered 0 tablets
I20240503 02:14:17.640448 12720 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240503 02:14:17.680091 12720 rpc_server.cc:307] RPC server started. Bound to: 127.9.151.129:43683
I20240503 02:14:17.680150 12846 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.151.129:43683 every 8 connection(s)
I20240503 02:14:17.681535 12720 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknrxqHk/test-tmp/tablet_copy_client_session-itest.0.TabletCopyClientSessionITest.TestStartTabletCopy.1714702442467385-9822-0/minicluster-data/ts-0/data/info.pb
I20240503 02:14:17.688836 9822 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknrxqHk/build/asan/bin/kudu as pid 12720
I20240503 02:14:17.689033 9822 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknrxqHk/test-tmp/tablet_copy_client_session-itest.0.TabletCopyClientSessionITest.TestStartTabletCopy.1714702442467385-9822-0/minicluster-data/ts-0/wal/instance
I20240503 02:14:17.692842 9822 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknrxqHk/build/asan/bin/kudu
/tmp/dist-test-tasknrxqHk/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknrxqHk/test-tmp/tablet_copy_client_session-itest.0.TabletCopyClientSessionITest.TestStartTabletCopy.1714702442467385-9822-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasknrxqHk/test-tmp/tablet_copy_client_session-itest.0.TabletCopyClientSessionITest.TestStartTabletCopy.1714702442467385-9822-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknrxqHk/test-tmp/tablet_copy_client_session-itest.0.TabletCopyClientSessionITest.TestStartTabletCopy.1714702442467385-9822-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasknrxqHk/test-tmp/tablet_copy_client_session-itest.0.TabletCopyClientSessionITest.TestStartTabletCopy.1714702442467385-9822-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.151.130:0
--local_ip_for_outbound_sockets=127.9.151.130
--webserver_interface=127.9.151.130
--webserver_port=0
--tserver_master_addrs=127.9.151.190:43557
--builtin_ntp_servers=127.9.151.148:43997
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin with env {}
I20240503 02:14:17.697430 12847 heartbeater.cc:347] Connected to a master server at 127.9.151.190:43557
I20240503 02:14:17.697803 12847 heartbeater.cc:464] Registering TS with master...
I20240503 02:14:17.699009 12847 heartbeater.cc:511] Master 127.9.151.190:43557 requested a full tablet report, sending...
I20240503 02:14:17.701567 12661 ts_manager.cc:194] Registered new tserver with Master: e07a4ec83c724757b7f34e9f22517cf2 (127.9.151.129:43683)
I20240503 02:14:17.703339 12661 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.9.151.129:50941
W20240503 02:14:18.062860 12851 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240503 02:14:18.063196 12851 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240503 02:14:18.063352 12851 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240503 02:14:18.072525 12851 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240503 02:14:18.073009 12851 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.151.130
I20240503 02:14:18.084669 12851 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.151.148:43997
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknrxqHk/test-tmp/tablet_copy_client_session-itest.0.TabletCopyClientSessionITest.TestStartTabletCopy.1714702442467385-9822-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasknrxqHk/test-tmp/tablet_copy_client_session-itest.0.TabletCopyClientSessionITest.TestStartTabletCopy.1714702442467385-9822-0/minicluster-data/ts-1/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.151.130:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknrxqHk/test-tmp/tablet_copy_client_session-itest.0.TabletCopyClientSessionITest.TestStartTabletCopy.1714702442467385-9822-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.151.130
--webserver_port=0
--tserver_master_addrs=127.9.151.190:43557
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.151.130
--log_dir=/tmp/dist-test-tasknrxqHk/test-tmp/tablet_copy_client_session-itest.0.TabletCopyClientSessionITest.TestStartTabletCopy.1714702442467385-9822-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c39e72a98e124e5a12080bac42105b996594d806
build type FASTDEBUG
built by None at 03 May 2024 01:43:22 UTC on efd1337a2aa3
build id 99
ASAN enabled
I20240503 02:14:18.086230 12851 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240503 02:14:18.088184 12851 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240503 02:14:18.104440 12857 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240503 02:14:18.104691 12856 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240503 02:14:18.105541 12859 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240503 02:14:18.106280 12851 server_base.cc:1034] running on GCE node
I20240503 02:14:18.107137 12851 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240503 02:14:18.108319 12851 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240503 02:14:18.109601 12851 hybrid_clock.cc:648] HybridClock initialized: now 1714702458109535 us; error 53 us; skew 500 ppm
I20240503 02:14:18.110172 12851 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20240503 02:14:18.113657 12851 webserver.cc:460] Webserver started at http://127.9.151.130:44033/ using document root <none> and password file <none>
I20240503 02:14:18.114501 12851 fs_manager.cc:359] Metadata directory not provided
I20240503 02:14:18.114670 12851 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240503 02:14:18.115073 12851 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240503 02:14:18.118194 12851 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknrxqHk/test-tmp/tablet_copy_client_session-itest.0.TabletCopyClientSessionITest.TestStartTabletCopy.1714702442467385-9822-0/minicluster-data/ts-1/data/instance:
uuid: "5f32fd22d24740cf9e7a68809c17cddc"
format_stamp: "Formatted at 2024-05-03 02:14:18 on dist-test-slave-twlh"
I20240503 02:14:18.119381 12851 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknrxqHk/test-tmp/tablet_copy_client_session-itest.0.TabletCopyClientSessionITest.TestStartTabletCopy.1714702442467385-9822-0/minicluster-data/ts-1/wal/instance:
uuid: "5f32fd22d24740cf9e7a68809c17cddc"
format_stamp: "Formatted at 2024-05-03 02:14:18 on dist-test-slave-twlh"
I20240503 02:14:18.125891 12851 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.000s
I20240503 02:14:18.131496 12865 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240503 02:14:18.133502 12851 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.001s
I20240503 02:14:18.133719 12851 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknrxqHk/test-tmp/tablet_copy_client_session-itest.0.TabletCopyClientSessionITest.TestStartTabletCopy.1714702442467385-9822-0/minicluster-data/ts-1/data,/tmp/dist-test-tasknrxqHk/test-tmp/tablet_copy_client_session-itest.0.TabletCopyClientSessionITest.TestStartTabletCopy.1714702442467385-9822-0/minicluster-data/ts-1/wal
uuid: "5f32fd22d24740cf9e7a68809c17cddc"
format_stamp: "Formatted at 2024-05-03 02:14:18 on dist-test-slave-twlh"
I20240503 02:14:18.133937 12851 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknrxqHk/test-tmp/tablet_copy_client_session-itest.0.TabletCopyClientSessionITest.TestStartTabletCopy.1714702442467385-9822-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasknrxqHk/test-tmp/tablet_copy_client_session-itest.0.TabletCopyClientSessionITest.TestStartTabletCopy.1714702442467385-9822-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasknrxqHk/test-tmp/tablet_copy_client_session-itest.0.TabletCopyClientSessionITest.TestStartTabletCopy.1714702442467385-9822-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240503 02:14:18.162230 12851 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240503 02:14:18.163312 12851 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240503 02:14:18.163655 12851 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240503 02:14:18.164759 12851 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240503 02:14:18.167028 12851 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240503 02:14:18.167153 12851 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240503 02:14:18.167307 12851 ts_tablet_manager.cc:590] Registered 0 tablets
I20240503 02:14:18.167364 12851 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240503 02:14:18.204823 12851 rpc_server.cc:307] RPC server started. Bound to: 127.9.151.130:34791
I20240503 02:14:18.204983 12977 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.151.130:34791 every 8 connection(s)
I20240503 02:14:18.206419 12851 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknrxqHk/test-tmp/tablet_copy_client_session-itest.0.TabletCopyClientSessionITest.TestStartTabletCopy.1714702442467385-9822-0/minicluster-data/ts-1/data/info.pb
I20240503 02:14:18.207882 9822 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknrxqHk/build/asan/bin/kudu as pid 12851
I20240503 02:14:18.208102 9822 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknrxqHk/test-tmp/tablet_copy_client_session-itest.0.TabletCopyClientSessionITest.TestStartTabletCopy.1714702442467385-9822-0/minicluster-data/ts-1/wal/instance
I20240503 02:14:18.221872 12978 heartbeater.cc:347] Connected to a master server at 127.9.151.190:43557
I20240503 02:14:18.222259 12978 heartbeater.cc:464] Registering TS with master...
I20240503 02:14:18.223296 12978 heartbeater.cc:511] Master 127.9.151.190:43557 requested a full tablet report, sending...
I20240503 02:14:18.225559 12660 ts_manager.cc:194] Registered new tserver with Master: 5f32fd22d24740cf9e7a68809c17cddc (127.9.151.130:34791)
I20240503 02:14:18.226891 12660 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.9.151.130:56819
I20240503 02:14:18.239013 9822 external_mini_cluster.cc:930] 2 TS(s) registered with all masters
I20240503 02:14:18.256203 9822 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasknrxqHk/build/asan/bin/kudu pid 12851
I20240503 02:14:18.266120 12973 generic_service.cc:196] Checking for leaks (request via RPC)
I20240503 02:14:18.351997 9822 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknrxqHk/build/asan/bin/kudu with pid 12851
I20240503 02:14:18.369436 9822 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasknrxqHk/build/asan/bin/kudu pid 12630
I20240503 02:14:18.371089 12691 generic_service.cc:196] Checking for leaks (request via RPC)
I20240503 02:14:18.474467 9822 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknrxqHk/build/asan/bin/kudu with pid 12630
I20240503 02:14:18.496347 9822 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknrxqHk/build/asan/bin/kudu
/tmp/dist-test-tasknrxqHk/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknrxqHk/test-tmp/tablet_copy_client_session-itest.0.TabletCopyClientSessionITest.TestStartTabletCopy.1714702442467385-9822-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasknrxqHk/test-tmp/tablet_copy_client_session-itest.0.TabletCopyClientSessionITest.TestStartTabletCopy.1714702442467385-9822-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknrxqHk/test-tmp/tablet_copy_client_session-itest.0.TabletCopyClientSessionITest.TestStartTabletCopy.1714702442467385-9822-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasknrxqHk/test-tmp/tablet_copy_client_session-itest.0.TabletCopyClientSessionITest.TestStartTabletCopy.1714702442467385-9822-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.151.190:43557
--webserver_interface=127.9.151.190
--webserver_port=44151
--builtin_ntp_servers=127.9.151.148:43997
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.151.190:43557
--master_tombstone_evicted_tablet_replicas=false with env {}
I20240503 02:14:18.706650 12847 heartbeater.cc:503] Master 127.9.151.190:43557 was elected leader, sending a full tablet report...
W20240503 02:14:18.708657 12847 heartbeater.cc:644] Failed to heartbeat to 127.9.151.190:43557 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.9.151.190:43557: connect: Connection refused (error 111)
W20240503 02:14:18.860677 12991 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240503 02:14:18.861114 12991 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240503 02:14:18.861282 12991 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240503 02:14:18.870019 12991 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20240503 02:14:18.870182 12991 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240503 02:14:18.870258 12991 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20240503 02:14:18.870358 12991 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20240503 02:14:18.881069 12991 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.151.148:43997
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknrxqHk/test-tmp/tablet_copy_client_session-itest.0.TabletCopyClientSessionITest.TestStartTabletCopy.1714702442467385-9822-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasknrxqHk/test-tmp/tablet_copy_client_session-itest.0.TabletCopyClientSessionITest.TestStartTabletCopy.1714702442467385-9822-0/minicluster-data/master-0/wal
--master_tombstone_evicted_tablet_replicas=false
--ipki_ca_key_size=768
--master_addresses=127.9.151.190:43557
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.151.190:43557
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknrxqHk/test-tmp/tablet_copy_client_session-itest.0.TabletCopyClientSessionITest.TestStartTabletCopy.1714702442467385-9822-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.151.190
--webserver_port=44151
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasknrxqHk/test-tmp/tablet_copy_client_session-itest.0.TabletCopyClientSessionITest.TestStartTabletCopy.1714702442467385-9822-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c39e72a98e124e5a12080bac42105b996594d806
build type FASTDEBUG
built by None at 03 May 2024 01:43:22 UTC on efd1337a2aa3
build id 99
ASAN enabled
I20240503 02:14:18.882936 12991 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240503 02:14:18.884898 12991 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240503 02:14:18.897995 13000 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240503 02:14:18.898597 12997 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240503 02:14:18.899036 12998 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240503 02:14:18.899938 12991 server_base.cc:1034] running on GCE node
I20240503 02:14:18.901104 12991 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240503 02:14:18.903142 12991 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240503 02:14:18.904481 12991 hybrid_clock.cc:648] HybridClock initialized: now 1714702458904390 us; error 56 us; skew 500 ppm
I20240503 02:14:18.904925 12991 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20240503 02:14:18.907753 12991 webserver.cc:460] Webserver started at http://127.9.151.190:44151/ using document root <none> and password file <none>
I20240503 02:14:18.908565 12991 fs_manager.cc:359] Metadata directory not provided
I20240503 02:14:18.908766 12991 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240503 02:14:18.915087 12991 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.000s
I20240503 02:14:18.918831 13006 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240503 02:14:18.920511 12991 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20240503 02:14:18.920707 12991 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknrxqHk/test-tmp/tablet_copy_client_session-itest.0.TabletCopyClientSessionITest.TestStartTabletCopy.1714702442467385-9822-0/minicluster-data/master-0/data,/tmp/dist-test-tasknrxqHk/test-tmp/tablet_copy_client_session-itest.0.TabletCopyClientSessionITest.TestStartTabletCopy.1714702442467385-9822-0/minicluster-data/master-0/wal
uuid: "2340184c71764af6a09921f4cb722805"
format_stamp: "Formatted at 2024-05-03 02:14:16 on dist-test-slave-twlh"
I20240503 02:14:18.921622 12991 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknrxqHk/test-tmp/tablet_copy_client_session-itest.0.TabletCopyClientSessionITest.TestStartTabletCopy.1714702442467385-9822-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasknrxqHk/test-tmp/tablet_copy_client_session-itest.0.TabletCopyClientSessionITest.TestStartTabletCopy.1714702442467385-9822-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasknrxqHk/test-tmp/tablet_copy_client_session-itest.0.TabletCopyClientSessionITest.TestStartTabletCopy.1714702442467385-9822-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240503 02:14:18.948050 12991 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240503 02:14:18.949147 12991 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240503 02:14:18.949426 12991 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240503 02:14:18.972683 12991 rpc_server.cc:307] RPC server started. Bound to: 127.9.151.190:43557
I20240503 02:14:18.972764 13057 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.151.190:43557 every 8 connection(s)
I20240503 02:14:18.974311 12991 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknrxqHk/test-tmp/tablet_copy_client_session-itest.0.TabletCopyClientSessionITest.TestStartTabletCopy.1714702442467385-9822-0/minicluster-data/master-0/data/info.pb
I20240503 02:14:18.980859 9822 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknrxqHk/build/asan/bin/kudu as pid 12991
I20240503 02:14:18.982946 13058 sys_catalog.cc:263] Verifying existing consensus state
I20240503 02:14:18.986908 13058 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2340184c71764af6a09921f4cb722805: Bootstrap starting.
I20240503 02:14:19.003983 13058 log.cc:824] T 00000000000000000000000000000000 P 2340184c71764af6a09921f4cb722805: Log is configured to *not* fsync() on all Append() calls
I20240503 02:14:19.013585 13058 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2340184c71764af6a09921f4cb722805: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240503 02:14:19.014192 13058 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2340184c71764af6a09921f4cb722805: Bootstrap complete.
I20240503 02:14:19.022341 13058 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2340184c71764af6a09921f4cb722805 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2340184c71764af6a09921f4cb722805" member_type: VOTER last_known_addr { host: "127.9.151.190" port: 43557 } }
I20240503 02:14:19.023465 13058 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2340184c71764af6a09921f4cb722805 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2340184c71764af6a09921f4cb722805, State: Initialized, Role: FOLLOWER
I20240503 02:14:19.024302 13058 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2340184c71764af6a09921f4cb722805 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2340184c71764af6a09921f4cb722805" member_type: VOTER last_known_addr { host: "127.9.151.190" port: 43557 } }
I20240503 02:14:19.024576 13058 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2340184c71764af6a09921f4cb722805 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20240503 02:14:19.024698 13058 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2340184c71764af6a09921f4cb722805 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20240503 02:14:19.024878 13058 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2340184c71764af6a09921f4cb722805 [term 1 FOLLOWER]: Advancing to term 2
I20240503 02:14:19.029121 13058 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2340184c71764af6a09921f4cb722805 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2340184c71764af6a09921f4cb722805" member_type: VOTER last_known_addr { host: "127.9.151.190" port: 43557 } }
I20240503 02:14:19.029734 13058 leader_election.cc:302] T 00000000000000000000000000000000 P 2340184c71764af6a09921f4cb722805 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2340184c71764af6a09921f4cb722805; no voters:
I20240503 02:14:19.030367 13058 leader_election.cc:288] T 00000000000000000000000000000000 P 2340184c71764af6a09921f4cb722805 [CANDIDATE]: Term 2 election: Requested vote from peers
I20240503 02:14:19.030573 13063 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2340184c71764af6a09921f4cb722805 [term 2 FOLLOWER]: Leader election won for term 2
I20240503 02:14:19.031632 13063 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2340184c71764af6a09921f4cb722805 [term 2 LEADER]: Becoming Leader. State: Replica: 2340184c71764af6a09921f4cb722805, State: Running, Role: LEADER
I20240503 02:14:19.032385 13063 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2340184c71764af6a09921f4cb722805 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2340184c71764af6a09921f4cb722805" member_type: VOTER last_known_addr { host: "127.9.151.190" port: 43557 } }
I20240503 02:14:19.032758 13058 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2340184c71764af6a09921f4cb722805 [sys.catalog]: configured and running, proceeding with master startup.
I20240503 02:14:19.035849 13065 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2340184c71764af6a09921f4cb722805 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2340184c71764af6a09921f4cb722805. Latest consensus state: current_term: 2 leader_uuid: "2340184c71764af6a09921f4cb722805" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2340184c71764af6a09921f4cb722805" member_type: VOTER last_known_addr { host: "127.9.151.190" port: 43557 } } }
I20240503 02:14:19.036263 13065 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2340184c71764af6a09921f4cb722805 [sys.catalog]: This master's current role is: LEADER
I20240503 02:14:19.036924 13064 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2340184c71764af6a09921f4cb722805 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "2340184c71764af6a09921f4cb722805" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2340184c71764af6a09921f4cb722805" member_type: VOTER last_known_addr { host: "127.9.151.190" port: 43557 } } }
I20240503 02:14:19.037927 13064 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2340184c71764af6a09921f4cb722805 [sys.catalog]: This master's current role is: LEADER
I20240503 02:14:19.039467 13068 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20240503 02:14:19.048597 13068 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20240503 02:14:19.051936 13068 catalog_manager.cc:1259] Loaded cluster ID: 0013c8b6660d40fd9bcf5c82b8dcb7bc
I20240503 02:14:19.052140 13068 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20240503 02:14:19.056865 13068 catalog_manager.cc:1504] Loading token signing keys...
I20240503 02:14:19.058247 13068 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2340184c71764af6a09921f4cb722805: Loaded TSK: 0
I20240503 02:14:19.059684 13068 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20240503 02:14:19.717923 12847 heartbeater.cc:347] Connected to a master server at 127.9.151.190:43557
I20240503 02:14:19.718212 12847 heartbeater.cc:503] Master 127.9.151.190:43557 was elected leader, sending a full tablet report...
I20240503 02:14:19.720625 13023 master_service.cc:432] Got heartbeat from unknown tserver (permanent_uuid: "e07a4ec83c724757b7f34e9f22517cf2" instance_seqno: 1714702457670630) as {username='slave'} at 127.9.151.129:40541; Asking this server to re-register.
I20240503 02:14:19.721714 12847 heartbeater.cc:464] Registering TS with master...
I20240503 02:14:19.722040 12847 heartbeater.cc:511] Master 127.9.151.190:43557 requested a full tablet report, sending...
I20240503 02:14:19.723943 13023 ts_manager.cc:194] Registered new tserver with Master: e07a4ec83c724757b7f34e9f22517cf2 (127.9.151.129:43683)
I20240503 02:14:19.726336 9822 external_mini_cluster.cc:930] 1 TS(s) registered with all masters
I20240503 02:14:19.726523 9822 test_util.cc:268] Using random seed: 1191307948
I20240503 02:14:19.751855 13023 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:40932:
name: "test-workload"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20240503 02:14:19.783416 12782 tablet_service.cc:1467] Processing CreateTablet for tablet 9d6e834a4e964d4f989a0b76a8a80492 (DEFAULT_TABLE table=test-workload [id=4e27302d7b2f4fb78e1b55c0dbbd2fde]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:14:19.785154 12782 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9d6e834a4e964d4f989a0b76a8a80492. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:14:19.796873 13093 tablet_bootstrap.cc:492] T 9d6e834a4e964d4f989a0b76a8a80492 P e07a4ec83c724757b7f34e9f22517cf2: Bootstrap starting.
I20240503 02:14:19.801014 13093 tablet_bootstrap.cc:654] T 9d6e834a4e964d4f989a0b76a8a80492 P e07a4ec83c724757b7f34e9f22517cf2: Neither blocks nor log segments found. Creating new log.
I20240503 02:14:19.802273 13093 log.cc:824] T 9d6e834a4e964d4f989a0b76a8a80492 P e07a4ec83c724757b7f34e9f22517cf2: Log is configured to *not* fsync() on all Append() calls
I20240503 02:14:19.805284 13093 tablet_bootstrap.cc:492] T 9d6e834a4e964d4f989a0b76a8a80492 P e07a4ec83c724757b7f34e9f22517cf2: No bootstrap required, opened a new log
I20240503 02:14:19.805635 13093 ts_tablet_manager.cc:1367] T 9d6e834a4e964d4f989a0b76a8a80492 P e07a4ec83c724757b7f34e9f22517cf2: Time spent bootstrapping tablet: real 0.009s user 0.003s sys 0.003s
I20240503 02:14:19.812680 13093 raft_consensus.cc:348] T 9d6e834a4e964d4f989a0b76a8a80492 P e07a4ec83c724757b7f34e9f22517cf2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e07a4ec83c724757b7f34e9f22517cf2" member_type: VOTER last_known_addr { host: "127.9.151.129" port: 43683 } }
I20240503 02:14:19.813095 13093 raft_consensus.cc:374] T 9d6e834a4e964d4f989a0b76a8a80492 P e07a4ec83c724757b7f34e9f22517cf2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:14:19.813215 13093 raft_consensus.cc:729] T 9d6e834a4e964d4f989a0b76a8a80492 P e07a4ec83c724757b7f34e9f22517cf2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e07a4ec83c724757b7f34e9f22517cf2, State: Initialized, Role: FOLLOWER
I20240503 02:14:19.813987 13093 consensus_queue.cc:260] T 9d6e834a4e964d4f989a0b76a8a80492 P e07a4ec83c724757b7f34e9f22517cf2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e07a4ec83c724757b7f34e9f22517cf2" member_type: VOTER last_known_addr { host: "127.9.151.129" port: 43683 } }
I20240503 02:14:19.814402 13093 raft_consensus.cc:388] T 9d6e834a4e964d4f989a0b76a8a80492 P e07a4ec83c724757b7f34e9f22517cf2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20240503 02:14:19.814567 13093 raft_consensus.cc:482] T 9d6e834a4e964d4f989a0b76a8a80492 P e07a4ec83c724757b7f34e9f22517cf2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20240503 02:14:19.814797 13093 raft_consensus.cc:3037] T 9d6e834a4e964d4f989a0b76a8a80492 P e07a4ec83c724757b7f34e9f22517cf2 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:14:19.817979 13093 raft_consensus.cc:504] T 9d6e834a4e964d4f989a0b76a8a80492 P e07a4ec83c724757b7f34e9f22517cf2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e07a4ec83c724757b7f34e9f22517cf2" member_type: VOTER last_known_addr { host: "127.9.151.129" port: 43683 } }
I20240503 02:14:19.818675 13093 leader_election.cc:302] T 9d6e834a4e964d4f989a0b76a8a80492 P e07a4ec83c724757b7f34e9f22517cf2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e07a4ec83c724757b7f34e9f22517cf2; no voters:
I20240503 02:14:19.819470 13093 leader_election.cc:288] T 9d6e834a4e964d4f989a0b76a8a80492 P e07a4ec83c724757b7f34e9f22517cf2 [CANDIDATE]: Term 1 election: Requested vote from peers
I20240503 02:14:19.819723 13095 raft_consensus.cc:2781] T 9d6e834a4e964d4f989a0b76a8a80492 P e07a4ec83c724757b7f34e9f22517cf2 [term 1 FOLLOWER]: Leader election won for term 1
I20240503 02:14:19.820541 13095 raft_consensus.cc:686] T 9d6e834a4e964d4f989a0b76a8a80492 P e07a4ec83c724757b7f34e9f22517cf2 [term 1 LEADER]: Becoming Leader. State: Replica: e07a4ec83c724757b7f34e9f22517cf2, State: Running, Role: LEADER
I20240503 02:14:19.821274 13095 consensus_queue.cc:237] T 9d6e834a4e964d4f989a0b76a8a80492 P e07a4ec83c724757b7f34e9f22517cf2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e07a4ec83c724757b7f34e9f22517cf2" member_type: VOTER last_known_addr { host: "127.9.151.129" port: 43683 } }
I20240503 02:14:19.821945 13093 ts_tablet_manager.cc:1398] T 9d6e834a4e964d4f989a0b76a8a80492 P e07a4ec83c724757b7f34e9f22517cf2: Time spent starting tablet: real 0.016s user 0.016s sys 0.000s
I20240503 02:14:19.827440 13023 catalog_manager.cc:5484] T 9d6e834a4e964d4f989a0b76a8a80492 P e07a4ec83c724757b7f34e9f22517cf2 reported cstate change: term changed from 0 to 1, leader changed from <none> to e07a4ec83c724757b7f34e9f22517cf2 (127.9.151.129). New cstate: current_term: 1 leader_uuid: "e07a4ec83c724757b7f34e9f22517cf2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e07a4ec83c724757b7f34e9f22517cf2" member_type: VOTER last_known_addr { host: "127.9.151.129" port: 43683 } health_report { overall_health: HEALTHY } } }
I20240503 02:14:20.524533 9822 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknrxqHk/build/asan/bin/kudu
/tmp/dist-test-tasknrxqHk/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknrxqHk/test-tmp/tablet_copy_client_session-itest.0.TabletCopyClientSessionITest.TestStartTabletCopy.1714702442467385-9822-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasknrxqHk/test-tmp/tablet_copy_client_session-itest.0.TabletCopyClientSessionITest.TestStartTabletCopy.1714702442467385-9822-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknrxqHk/test-tmp/tablet_copy_client_session-itest.0.TabletCopyClientSessionITest.TestStartTabletCopy.1714702442467385-9822-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasknrxqHk/test-tmp/tablet_copy_client_session-itest.0.TabletCopyClientSessionITest.TestStartTabletCopy.1714702442467385-9822-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.151.130:34791
--local_ip_for_outbound_sockets=127.9.151.130
--tserver_master_addrs=127.9.151.190:43557
--webserver_port=44033
--webserver_interface=127.9.151.130
--builtin_ntp_servers=127.9.151.148:43997
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin with env {}
W20240503 02:14:20.896132 13111 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240503 02:14:20.896540 13111 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240503 02:14:20.896713 13111 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240503 02:14:20.904734 13111 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240503 02:14:20.905001 13111 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.151.130
I20240503 02:14:20.914992 13111 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.151.148:43997
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknrxqHk/test-tmp/tablet_copy_client_session-itest.0.TabletCopyClientSessionITest.TestStartTabletCopy.1714702442467385-9822-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasknrxqHk/test-tmp/tablet_copy_client_session-itest.0.TabletCopyClientSessionITest.TestStartTabletCopy.1714702442467385-9822-0/minicluster-data/ts-1/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.151.130:34791
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknrxqHk/test-tmp/tablet_copy_client_session-itest.0.TabletCopyClientSessionITest.TestStartTabletCopy.1714702442467385-9822-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.151.130
--webserver_port=44033
--tserver_master_addrs=127.9.151.190:43557
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.151.130
--log_dir=/tmp/dist-test-tasknrxqHk/test-tmp/tablet_copy_client_session-itest.0.TabletCopyClientSessionITest.TestStartTabletCopy.1714702442467385-9822-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c39e72a98e124e5a12080bac42105b996594d806
build type FASTDEBUG
built by None at 03 May 2024 01:43:22 UTC on efd1337a2aa3
build id 99
ASAN enabled
I20240503 02:14:20.916512 13111 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240503 02:14:20.918591 13111 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240503 02:14:20.933270 13119 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240503 02:14:20.933655 13117 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240503 02:14:20.934082 13116 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240503 02:14:20.934875 13111 server_base.cc:1034] running on GCE node
I20240503 02:14:20.935590 13111 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240503 02:14:20.936650 13111 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240503 02:14:20.937932 13111 hybrid_clock.cc:648] HybridClock initialized: now 1714702460937826 us; error 60 us; skew 500 ppm
I20240503 02:14:20.938416 13111 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20240503 02:14:20.941841 13111 webserver.cc:460] Webserver started at http://127.9.151.130:44033/ using document root <none> and password file <none>
I20240503 02:14:20.942750 13111 fs_manager.cc:359] Metadata directory not provided
I20240503 02:14:20.942917 13111 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240503 02:14:20.949581 13111 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.002s sys 0.004s
I20240503 02:14:20.953408 13125 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240503 02:14:20.955238 13111 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20240503 02:14:20.955471 13111 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknrxqHk/test-tmp/tablet_copy_client_session-itest.0.TabletCopyClientSessionITest.TestStartTabletCopy.1714702442467385-9822-0/minicluster-data/ts-1/data,/tmp/dist-test-tasknrxqHk/test-tmp/tablet_copy_client_session-itest.0.TabletCopyClientSessionITest.TestStartTabletCopy.1714702442467385-9822-0/minicluster-data/ts-1/wal
uuid: "5f32fd22d24740cf9e7a68809c17cddc"
format_stamp: "Formatted at 2024-05-03 02:14:18 on dist-test-slave-twlh"
I20240503 02:14:20.956418 13111 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknrxqHk/test-tmp/tablet_copy_client_session-itest.0.TabletCopyClientSessionITest.TestStartTabletCopy.1714702442467385-9822-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasknrxqHk/test-tmp/tablet_copy_client_session-itest.0.TabletCopyClientSessionITest.TestStartTabletCopy.1714702442467385-9822-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasknrxqHk/test-tmp/tablet_copy_client_session-itest.0.TabletCopyClientSessionITest.TestStartTabletCopy.1714702442467385-9822-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240503 02:14:20.987205 13111 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240503 02:14:20.988421 13111 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240503 02:14:20.988785 13111 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240503 02:14:20.989951 13111 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240503 02:14:20.992136 13111 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240503 02:14:20.992272 13111 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240503 02:14:20.992405 13111 ts_tablet_manager.cc:590] Registered 0 tablets
I20240503 02:14:20.992486 13111 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240503 02:14:21.037194 13111 rpc_server.cc:307] RPC server started. Bound to: 127.9.151.130:34791
I20240503 02:14:21.037285 13237 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.151.130:34791 every 8 connection(s)
I20240503 02:14:21.038815 13111 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknrxqHk/test-tmp/tablet_copy_client_session-itest.0.TabletCopyClientSessionITest.TestStartTabletCopy.1714702442467385-9822-0/minicluster-data/ts-1/data/info.pb
I20240503 02:14:21.039930 9822 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknrxqHk/build/asan/bin/kudu as pid 13111
I20240503 02:14:21.060359 13238 heartbeater.cc:347] Connected to a master server at 127.9.151.190:43557
I20240503 02:14:21.060758 13238 heartbeater.cc:464] Registering TS with master...
I20240503 02:14:21.062067 13238 heartbeater.cc:511] Master 127.9.151.190:43557 requested a full tablet report, sending...
I20240503 02:14:21.062098 13244 ts_tablet_manager.cc:902] T 9d6e834a4e964d4f989a0b76a8a80492 P 5f32fd22d24740cf9e7a68809c17cddc: Initiating tablet copy from peer e07a4ec83c724757b7f34e9f22517cf2 (127.9.151.129:43683)
I20240503 02:14:21.064124 13023 ts_manager.cc:194] Registered new tserver with Master: 5f32fd22d24740cf9e7a68809c17cddc (127.9.151.130:34791)
I20240503 02:14:21.065270 13023 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.9.151.130:57685
I20240503 02:14:21.067028 13244 tablet_copy_client.cc:308] T 9d6e834a4e964d4f989a0b76a8a80492 P 5f32fd22d24740cf9e7a68809c17cddc: tablet copy: Beginning tablet copy session from remote peer at address 127.9.151.129:43683
I20240503 02:14:21.074010 12822 tablet_copy_service.cc:140] P e07a4ec83c724757b7f34e9f22517cf2: Received BeginTabletCopySession request for tablet 9d6e834a4e964d4f989a0b76a8a80492 from peer 5f32fd22d24740cf9e7a68809c17cddc ({username='slave'} at 127.9.151.130:52239)
I20240503 02:14:21.074298 12822 tablet_copy_service.cc:161] P e07a4ec83c724757b7f34e9f22517cf2: Beginning new tablet copy session on tablet 9d6e834a4e964d4f989a0b76a8a80492 from peer 5f32fd22d24740cf9e7a68809c17cddc at {username='slave'} at 127.9.151.130:52239: session id = 5f32fd22d24740cf9e7a68809c17cddc-9d6e834a4e964d4f989a0b76a8a80492
I20240503 02:14:21.077019 12822 tablet_copy_source_session.cc:205] T 9d6e834a4e964d4f989a0b76a8a80492 P e07a4ec83c724757b7f34e9f22517cf2: Tablet Copy: opened 0 blocks and 1 log segments
I20240503 02:14:21.082074 13244 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9d6e834a4e964d4f989a0b76a8a80492. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:14:21.093509 13244 tablet_copy_client.cc:783] T 9d6e834a4e964d4f989a0b76a8a80492 P 5f32fd22d24740cf9e7a68809c17cddc: tablet copy: Starting download of 0 data blocks...
I20240503 02:14:21.093859 13244 tablet_copy_client.cc:647] T 9d6e834a4e964d4f989a0b76a8a80492 P 5f32fd22d24740cf9e7a68809c17cddc: tablet copy: Starting download of 1 WAL segments...
I20240503 02:14:21.099301 13244 tablet_copy_client.cc:523] T 9d6e834a4e964d4f989a0b76a8a80492 P 5f32fd22d24740cf9e7a68809c17cddc: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240503 02:14:21.102772 13244 tablet_bootstrap.cc:492] T 9d6e834a4e964d4f989a0b76a8a80492 P 5f32fd22d24740cf9e7a68809c17cddc: Bootstrap starting.
I20240503 02:14:21.168210 13244 log.cc:824] T 9d6e834a4e964d4f989a0b76a8a80492 P 5f32fd22d24740cf9e7a68809c17cddc: Log is configured to *not* fsync() on all Append() calls
I20240503 02:14:21.747594 13244 tablet_bootstrap.cc:492] T 9d6e834a4e964d4f989a0b76a8a80492 P 5f32fd22d24740cf9e7a68809c17cddc: Bootstrap replayed 1/1 log segments. Stats: ops{read=213 overwritten=0 applied=213 ignored=0} inserts{seen=10600 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240503 02:14:21.748162 13244 tablet_bootstrap.cc:492] T 9d6e834a4e964d4f989a0b76a8a80492 P 5f32fd22d24740cf9e7a68809c17cddc: Bootstrap complete.
I20240503 02:14:21.748724 13244 ts_tablet_manager.cc:1367] T 9d6e834a4e964d4f989a0b76a8a80492 P 5f32fd22d24740cf9e7a68809c17cddc: Time spent bootstrapping tablet: real 0.646s user 0.607s sys 0.040s
I20240503 02:14:21.754670 13244 raft_consensus.cc:348] T 9d6e834a4e964d4f989a0b76a8a80492 P 5f32fd22d24740cf9e7a68809c17cddc [term 1 NON_PARTICIPANT]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e07a4ec83c724757b7f34e9f22517cf2" member_type: VOTER last_known_addr { host: "127.9.151.129" port: 43683 } }
I20240503 02:14:21.755156 13244 raft_consensus.cc:729] T 9d6e834a4e964d4f989a0b76a8a80492 P 5f32fd22d24740cf9e7a68809c17cddc [term 1 NON_PARTICIPANT]: Becoming Follower/Learner. State: Replica: 5f32fd22d24740cf9e7a68809c17cddc, State: Initialized, Role: NON_PARTICIPANT
I20240503 02:14:21.755681 13244 consensus_queue.cc:260] T 9d6e834a4e964d4f989a0b76a8a80492 P 5f32fd22d24740cf9e7a68809c17cddc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 213, Last appended: 1.213, Last appended by leader: 213, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e07a4ec83c724757b7f34e9f22517cf2" member_type: VOTER last_known_addr { host: "127.9.151.129" port: 43683 } }
I20240503 02:14:21.756918 13238 heartbeater.cc:503] Master 127.9.151.190:43557 was elected leader, sending a full tablet report...
I20240503 02:14:21.757598 13244 ts_tablet_manager.cc:1398] T 9d6e834a4e964d4f989a0b76a8a80492 P 5f32fd22d24740cf9e7a68809c17cddc: Time spent starting tablet: real 0.009s user 0.009s sys 0.000s
I20240503 02:14:21.758646 12822 tablet_copy_service.cc:342] P e07a4ec83c724757b7f34e9f22517cf2: Request end of tablet copy session 5f32fd22d24740cf9e7a68809c17cddc-9d6e834a4e964d4f989a0b76a8a80492 received from {username='slave'} at 127.9.151.130:52239
I20240503 02:14:21.758882 12822 tablet_copy_service.cc:433] P e07a4ec83c724757b7f34e9f22517cf2: ending tablet copy session 5f32fd22d24740cf9e7a68809c17cddc-9d6e834a4e964d4f989a0b76a8a80492 on tablet 9d6e834a4e964d4f989a0b76a8a80492 with peer 5f32fd22d24740cf9e7a68809c17cddc
I20240503 02:14:21.758882 13173 tablet_service.cc:1514] Processing DeleteTablet for tablet 9d6e834a4e964d4f989a0b76a8a80492 with delete_type TABLET_DATA_TOMBSTONED from {username='slave'} at 127.0.0.1:60634
I20240503 02:14:21.759766 13252 tablet_replica.cc:330] T 9d6e834a4e964d4f989a0b76a8a80492 P 5f32fd22d24740cf9e7a68809c17cddc: stopping tablet replica
I20240503 02:14:21.760231 13252 raft_consensus.cc:2229] T 9d6e834a4e964d4f989a0b76a8a80492 P 5f32fd22d24740cf9e7a68809c17cddc [term 1 NON_PARTICIPANT]: Raft consensus shutting down.
I20240503 02:14:21.760895 13252 raft_consensus.cc:2258] T 9d6e834a4e964d4f989a0b76a8a80492 P 5f32fd22d24740cf9e7a68809c17cddc [term 1 NON_PARTICIPANT]: Raft consensus is shut down!
I20240503 02:14:21.762347 13252 ts_tablet_manager.cc:1875] T 9d6e834a4e964d4f989a0b76a8a80492 P 5f32fd22d24740cf9e7a68809c17cddc: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240503 02:14:21.767519 13252 ts_tablet_manager.cc:1888] T 9d6e834a4e964d4f989a0b76a8a80492 P 5f32fd22d24740cf9e7a68809c17cddc: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.213
I20240503 02:14:21.767710 13252 log.cc:1196] T 9d6e834a4e964d4f989a0b76a8a80492 P 5f32fd22d24740cf9e7a68809c17cddc: Deleting WAL directory at /tmp/dist-test-tasknrxqHk/test-tmp/tablet_copy_client_session-itest.0.TabletCopyClientSessionITest.TestStartTabletCopy.1714702442467385-9822-0/minicluster-data/ts-1/wal/wals/9d6e834a4e964d4f989a0b76a8a80492
I20240503 02:14:21.769879 13244 ts_tablet_manager.cc:902] T 9d6e834a4e964d4f989a0b76a8a80492 P 5f32fd22d24740cf9e7a68809c17cddc: Initiating tablet copy from peer e07a4ec83c724757b7f34e9f22517cf2 (127.9.151.129:43683)
I20240503 02:14:21.771212 13244 tablet_copy_client.cc:308] T 9d6e834a4e964d4f989a0b76a8a80492 P 5f32fd22d24740cf9e7a68809c17cddc: tablet copy: Beginning tablet copy session from remote peer at address 127.9.151.129:43683
I20240503 02:14:21.772068 12822 tablet_copy_service.cc:140] P e07a4ec83c724757b7f34e9f22517cf2: Received BeginTabletCopySession request for tablet 9d6e834a4e964d4f989a0b76a8a80492 from peer 5f32fd22d24740cf9e7a68809c17cddc ({username='slave'} at 127.9.151.130:52239)
I20240503 02:14:21.772249 12822 tablet_copy_service.cc:161] P e07a4ec83c724757b7f34e9f22517cf2: Beginning new tablet copy session on tablet 9d6e834a4e964d4f989a0b76a8a80492 from peer 5f32fd22d24740cf9e7a68809c17cddc at {username='slave'} at 127.9.151.130:52239: session id = 5f32fd22d24740cf9e7a68809c17cddc-9d6e834a4e964d4f989a0b76a8a80492
I20240503 02:14:21.774533 12822 tablet_copy_source_session.cc:205] T 9d6e834a4e964d4f989a0b76a8a80492 P e07a4ec83c724757b7f34e9f22517cf2: Tablet Copy: opened 0 blocks and 1 log segments
I20240503 02:14:21.775893 13244 ts_tablet_manager.cc:1875] T 9d6e834a4e964d4f989a0b76a8a80492 P 5f32fd22d24740cf9e7a68809c17cddc: Deleting tablet data with delete state TABLET_DATA_COPYING
I20240503 02:14:21.780799 13244 ts_tablet_manager.cc:1888] T 9d6e834a4e964d4f989a0b76a8a80492 P 5f32fd22d24740cf9e7a68809c17cddc: tablet deleted with delete type TABLET_DATA_COPYING: last-logged OpId 1.213
I20240503 02:14:21.781144 13244 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9d6e834a4e964d4f989a0b76a8a80492. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:14:21.783632 13244 tablet_copy_client.cc:783] T 9d6e834a4e964d4f989a0b76a8a80492 P 5f32fd22d24740cf9e7a68809c17cddc: tablet copy: Starting download of 0 data blocks...
I20240503 02:14:21.783902 13244 tablet_copy_client.cc:647] T 9d6e834a4e964d4f989a0b76a8a80492 P 5f32fd22d24740cf9e7a68809c17cddc: tablet copy: Starting download of 1 WAL segments...
I20240503 02:14:21.787792 13244 tablet_copy_client.cc:523] T 9d6e834a4e964d4f989a0b76a8a80492 P 5f32fd22d24740cf9e7a68809c17cddc: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240503 02:14:21.790769 13244 tablet_bootstrap.cc:492] T 9d6e834a4e964d4f989a0b76a8a80492 P 5f32fd22d24740cf9e7a68809c17cddc: Bootstrap starting.
I20240503 02:14:22.415910 13244 tablet_bootstrap.cc:492] T 9d6e834a4e964d4f989a0b76a8a80492 P 5f32fd22d24740cf9e7a68809c17cddc: Bootstrap replayed 1/1 log segments. Stats: ops{read=213 overwritten=0 applied=213 ignored=0} inserts{seen=10600 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240503 02:14:22.416358 13244 tablet_bootstrap.cc:492] T 9d6e834a4e964d4f989a0b76a8a80492 P 5f32fd22d24740cf9e7a68809c17cddc: Bootstrap complete.
I20240503 02:14:22.416751 13244 ts_tablet_manager.cc:1367] T 9d6e834a4e964d4f989a0b76a8a80492 P 5f32fd22d24740cf9e7a68809c17cddc: Time spent bootstrapping tablet: real 0.626s user 0.562s sys 0.064s
I20240503 02:14:22.417590 13244 raft_consensus.cc:348] T 9d6e834a4e964d4f989a0b76a8a80492 P 5f32fd22d24740cf9e7a68809c17cddc [term 1 NON_PARTICIPANT]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e07a4ec83c724757b7f34e9f22517cf2" member_type: VOTER last_known_addr { host: "127.9.151.129" port: 43683 } }
I20240503 02:14:22.417784 13244 raft_consensus.cc:729] T 9d6e834a4e964d4f989a0b76a8a80492 P 5f32fd22d24740cf9e7a68809c17cddc [term 1 NON_PARTICIPANT]: Becoming Follower/Learner. State: Replica: 5f32fd22d24740cf9e7a68809c17cddc, State: Initialized, Role: NON_PARTICIPANT
I20240503 02:14:22.418012 13244 consensus_queue.cc:260] T 9d6e834a4e964d4f989a0b76a8a80492 P 5f32fd22d24740cf9e7a68809c17cddc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 213, Last appended: 1.213, Last appended by leader: 213, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e07a4ec83c724757b7f34e9f22517cf2" member_type: VOTER last_known_addr { host: "127.9.151.129" port: 43683 } }
I20240503 02:14:22.418709 13244 ts_tablet_manager.cc:1398] T 9d6e834a4e964d4f989a0b76a8a80492 P 5f32fd22d24740cf9e7a68809c17cddc: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20240503 02:14:22.419593 12822 tablet_copy_service.cc:342] P e07a4ec83c724757b7f34e9f22517cf2: Request end of tablet copy session 5f32fd22d24740cf9e7a68809c17cddc-9d6e834a4e964d4f989a0b76a8a80492 received from {username='slave'} at 127.9.151.130:52239
I20240503 02:14:22.419718 12822 tablet_copy_service.cc:433] P e07a4ec83c724757b7f34e9f22517cf2: ending tablet copy session 5f32fd22d24740cf9e7a68809c17cddc-9d6e834a4e964d4f989a0b76a8a80492 on tablet 9d6e834a4e964d4f989a0b76a8a80492 with peer 5f32fd22d24740cf9e7a68809c17cddc
I20240503 02:14:22.427578 9822 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasknrxqHk/build/asan/bin/kudu pid 12720
I20240503 02:14:22.428833 12842 generic_service.cc:196] Checking for leaks (request via RPC)
I20240503 02:14:22.534168 9822 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknrxqHk/build/asan/bin/kudu with pid 12720
I20240503 02:14:22.555261 9822 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasknrxqHk/build/asan/bin/kudu pid 13111
I20240503 02:14:22.556407 13233 generic_service.cc:196] Checking for leaks (request via RPC)
I20240503 02:14:22.677131 9822 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknrxqHk/build/asan/bin/kudu with pid 13111
I20240503 02:14:22.698896 9822 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasknrxqHk/build/asan/bin/kudu pid 12991
I20240503 02:14:22.700109 13053 generic_service.cc:196] Checking for leaks (request via RPC)
I20240503 02:14:22.813899 9822 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknrxqHk/build/asan/bin/kudu with pid 12991
2024-05-03T02:14:22Z chronyd exiting
[ OK ] TabletCopyClientSessionITest.TestStartTabletCopy (6353 ms)
[ RUN ] TabletCopyClientSessionITest.TestCopyFromCrashedSource
2024-05-03T02:14:22Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2024-05-03T02:14:22Z Disabled control of system clock
I20240503 02:14:22.889655 9822 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknrxqHk/build/asan/bin/kudu
/tmp/dist-test-tasknrxqHk/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknrxqHk/test-tmp/tablet_copy_client_session-itest.0.TabletCopyClientSessionITest.TestCopyFromCrashedSource.1714702442467385-9822-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasknrxqHk/test-tmp/tablet_copy_client_session-itest.0.TabletCopyClientSessionITest.TestCopyFromCrashedSource.1714702442467385-9822-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknrxqHk/test-tmp/tablet_copy_client_session-itest.0.TabletCopyClientSessionITest.TestCopyFromCrashedSource.1714702442467385-9822-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasknrxqHk/test-tmp/tablet_copy_client_session-itest.0.TabletCopyClientSessionITest.TestCopyFromCrashedSource.1714702442467385-9822-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.151.190:34365
--webserver_interface=127.9.151.190
--webserver_port=0
--builtin_ntp_servers=127.9.151.148:33391
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.151.190:34365
--master_tombstone_evicted_tablet_replicas=false with env {}
W20240503 02:14:23.260241 13265 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240503 02:14:23.260579 13265 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240503 02:14:23.260692 13265 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240503 02:14:23.268720 13265 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20240503 02:14:23.268862 13265 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240503 02:14:23.268908 13265 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20240503 02:14:23.268965 13265 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20240503 02:14:23.278657 13265 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.151.148:33391
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknrxqHk/test-tmp/tablet_copy_client_session-itest.0.TabletCopyClientSessionITest.TestCopyFromCrashedSource.1714702442467385-9822-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasknrxqHk/test-tmp/tablet_copy_client_session-itest.0.TabletCopyClientSessionITest.TestCopyFromCrashedSource.1714702442467385-9822-0/minicluster-data/master-0/wal
--master_tombstone_evicted_tablet_replicas=false
--ipki_ca_key_size=768
--master_addresses=127.9.151.190:34365
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.151.190:34365
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknrxqHk/test-tmp/tablet_copy_client_session-itest.0.TabletCopyClientSessionITest.TestCopyFromCrashedSource.1714702442467385-9822-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.151.190
--webserver_port=0
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasknrxqHk/test-tmp/tablet_copy_client_session-itest.0.TabletCopyClientSessionITest.TestCopyFromCrashedSource.1714702442467385-9822-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c39e72a98e124e5a12080bac42105b996594d806
build type FASTDEBUG
built by None at 03 May 2024 01:43:22 UTC on efd1337a2aa3
build id 99
ASAN enabled
I20240503 02:14:23.280148 13265 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240503 02:14:23.281975 13265 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240503 02:14:23.295017 13270 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240503 02:14:23.295661 13271 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240503 02:14:23.296231 13273 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240503 02:14:23.297109 13265 server_base.cc:1034] running on GCE node
I20240503 02:14:23.298302 13265 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240503 02:14:23.300406 13265 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240503 02:14:23.301738 13265 hybrid_clock.cc:648] HybridClock initialized: now 1714702463301638 us; error 61 us; skew 500 ppm
I20240503 02:14:23.302245 13265 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20240503 02:14:23.305473 13265 webserver.cc:460] Webserver started at http://127.9.151.190:43663/ using document root <none> and password file <none>
I20240503 02:14:23.306300 13265 fs_manager.cc:359] Metadata directory not provided
I20240503 02:14:23.306452 13265 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240503 02:14:23.306866 13265 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240503 02:14:23.309728 13265 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknrxqHk/test-tmp/tablet_copy_client_session-itest.0.TabletCopyClientSessionITest.TestCopyFromCrashedSource.1714702442467385-9822-0/minicluster-data/master-0/data/instance:
uuid: "7039e750de844d2b8b3f0414147e02a2"
format_stamp: "Formatted at 2024-05-03 02:14:23 on dist-test-slave-twlh"
I20240503 02:14:23.310567 13265 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknrxqHk/test-tmp/tablet_copy_client_session-itest.0.TabletCopyClientSessionITest.TestCopyFromCrashedSource.1714702442467385-9822-0/minicluster-data/master-0/wal/instance:
uuid: "7039e750de844d2b8b3f0414147e02a2"
format_stamp: "Formatted at 2024-05-03 02:14:23 on dist-test-slave-twlh"
I20240503 02:14:23.316457 13265 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.004s
I20240503 02:14:23.320837 13279 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240503 02:14:23.323069 13265 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.002s
I20240503 02:14:23.323343 13265 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknrxqHk/test-tmp/tablet_copy_client_session-itest.0.TabletCopyClientSessionITest.TestCopyFromCrashedSource.1714702442467385-9822-0/minicluster-data/master-0/data,/tmp/dist-test-tasknrxqHk/test-tmp/tablet_copy_client_session-itest.0.TabletCopyClientSessionITest.TestCopyFromCrashedSource.1714702442467385-9822-0/minicluster-data/master-0/wal
uuid: "7039e750de844d2b8b3f0414147e02a2"
format_stamp: "Formatted at 2024-05-03 02:14:23 on dist-test-slave-twlh"
I20240503 02:14:23.323594 13265 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknrxqHk/test-tmp/tablet_copy_client_session-itest.0.TabletCopyClientSessionITest.TestCopyFromCrashedSource.1714702442467385-9822-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasknrxqHk/test-tmp/tablet_copy_client_session-itest.0.TabletCopyClientSessionITest.TestCopyFromCrashedSource.1714702442467385-9822-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasknrxqHk/test-tmp/tablet_copy_client_session-itest.0.TabletCopyClientSessionITest.TestCopyFromCrashedSource.1714702442467385-9822-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240503 02:14:23.359805 13265 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240503 02:14:23.360764 13265 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240503 02:14:23.361080 13265 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240503 02:14:23.381785 13265 rpc_server.cc:307] RPC server started. Bound to: 127.9.151.190:34365
I20240503 02:14:23.381799 13330 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.151.190:34365 every 8 connection(s)
I20240503 02:14:23.383354 13265 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknrxqHk/test-tmp/tablet_copy_client_session-itest.0.TabletCopyClientSessionITest.TestCopyFromCrashedSource.1714702442467385-9822-0/minicluster-data/master-0/data/info.pb
I20240503 02:14:23.387800 13331 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:14:23.387866 9822 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknrxqHk/build/asan/bin/kudu as pid 13265
I20240503 02:14:23.388156 9822 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknrxqHk/test-tmp/tablet_copy_client_session-itest.0.TabletCopyClientSessionITest.TestCopyFromCrashedSource.1714702442467385-9822-0/minicluster-data/master-0/wal/instance
I20240503 02:14:23.403909 13331 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7039e750de844d2b8b3f0414147e02a2: Bootstrap starting.
I20240503 02:14:23.407351 13331 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7039e750de844d2b8b3f0414147e02a2: Neither blocks nor log segments found. Creating new log.
I20240503 02:14:23.408483 13331 log.cc:824] T 00000000000000000000000000000000 P 7039e750de844d2b8b3f0414147e02a2: Log is configured to *not* fsync() on all Append() calls
I20240503 02:14:23.412066 13331 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7039e750de844d2b8b3f0414147e02a2: No bootstrap required, opened a new log
I20240503 02:14:23.419137 13331 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7039e750de844d2b8b3f0414147e02a2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7039e750de844d2b8b3f0414147e02a2" member_type: VOTER last_known_addr { host: "127.9.151.190" port: 34365 } }
I20240503 02:14:23.419633 13331 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7039e750de844d2b8b3f0414147e02a2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:14:23.419747 13331 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7039e750de844d2b8b3f0414147e02a2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7039e750de844d2b8b3f0414147e02a2, State: Initialized, Role: FOLLOWER
I20240503 02:14:23.420569 13331 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7039e750de844d2b8b3f0414147e02a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7039e750de844d2b8b3f0414147e02a2" member_type: VOTER last_known_addr { host: "127.9.151.190" port: 34365 } }
I20240503 02:14:23.420825 13331 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7039e750de844d2b8b3f0414147e02a2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20240503 02:14:23.420982 13331 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7039e750de844d2b8b3f0414147e02a2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20240503 02:14:23.421160 13331 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7039e750de844d2b8b3f0414147e02a2 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:14:23.423794 13331 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7039e750de844d2b8b3f0414147e02a2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7039e750de844d2b8b3f0414147e02a2" member_type: VOTER last_known_addr { host: "127.9.151.190" port: 34365 } }
I20240503 02:14:23.424414 13331 leader_election.cc:302] T 00000000000000000000000000000000 P 7039e750de844d2b8b3f0414147e02a2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7039e750de844d2b8b3f0414147e02a2; no voters:
I20240503 02:14:23.425391 13331 leader_election.cc:288] T 00000000000000000000000000000000 P 7039e750de844d2b8b3f0414147e02a2 [CANDIDATE]: Term 1 election: Requested vote from peers
I20240503 02:14:23.425541 13336 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7039e750de844d2b8b3f0414147e02a2 [term 1 FOLLOWER]: Leader election won for term 1
I20240503 02:14:23.426807 13336 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7039e750de844d2b8b3f0414147e02a2 [term 1 LEADER]: Becoming Leader. State: Replica: 7039e750de844d2b8b3f0414147e02a2, State: Running, Role: LEADER
I20240503 02:14:23.427702 13336 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7039e750de844d2b8b3f0414147e02a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7039e750de844d2b8b3f0414147e02a2" member_type: VOTER last_known_addr { host: "127.9.151.190" port: 34365 } }
I20240503 02:14:23.428881 13331 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7039e750de844d2b8b3f0414147e02a2 [sys.catalog]: configured and running, proceeding with master startup.
I20240503 02:14:23.431882 13337 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7039e750de844d2b8b3f0414147e02a2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7039e750de844d2b8b3f0414147e02a2. Latest consensus state: current_term: 1 leader_uuid: "7039e750de844d2b8b3f0414147e02a2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7039e750de844d2b8b3f0414147e02a2" member_type: VOTER last_known_addr { host: "127.9.151.190" port: 34365 } } }
I20240503 02:14:23.431864 13338 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7039e750de844d2b8b3f0414147e02a2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7039e750de844d2b8b3f0414147e02a2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7039e750de844d2b8b3f0414147e02a2" member_type: VOTER last_known_addr { host: "127.9.151.190" port: 34365 } } }
I20240503 02:14:23.432374 13337 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7039e750de844d2b8b3f0414147e02a2 [sys.catalog]: This master's current role is: LEADER
I20240503 02:14:23.432399 13338 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7039e750de844d2b8b3f0414147e02a2 [sys.catalog]: This master's current role is: LEADER
I20240503 02:14:23.438802 13344 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20240503 02:14:23.444670 13344 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20240503 02:14:23.455873 13344 catalog_manager.cc:1347] Generated new cluster ID: 0c3deaa48eae4c97ab80c02219ac1dfc
I20240503 02:14:23.456028 13344 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20240503 02:14:23.472165 13344 catalog_manager.cc:1370] Generated new certificate authority record
I20240503 02:14:23.473721 13344 catalog_manager.cc:1504] Loading token signing keys...
I20240503 02:14:23.485038 13344 catalog_manager.cc:5857] T 00000000000000000000000000000000 P 7039e750de844d2b8b3f0414147e02a2: Generated new TSK 0
I20240503 02:14:23.486207 13344 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20240503 02:14:23.508090 9822 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknrxqHk/build/asan/bin/kudu
/tmp/dist-test-tasknrxqHk/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknrxqHk/test-tmp/tablet_copy_client_session-itest.0.TabletCopyClientSessionITest.TestCopyFromCrashedSource.1714702442467385-9822-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasknrxqHk/test-tmp/tablet_copy_client_session-itest.0.TabletCopyClientSessionITest.TestCopyFromCrashedSource.1714702442467385-9822-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknrxqHk/test-tmp/tablet_copy_client_session-itest.0.TabletCopyClientSessionITest.TestCopyFromCrashedSource.1714702442467385-9822-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasknrxqHk/test-tmp/tablet_copy_client_session-itest.0.TabletCopyClientSessionITest.TestCopyFromCrashedSource.1714702442467385-9822-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.151.129:0
--local_ip_for_outbound_sockets=127.9.151.129
--webserver_interface=127.9.151.129
--webserver_port=0
--tserver_master_addrs=127.9.151.190:34365
--builtin_ntp_servers=127.9.151.148:33391
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin with env {}
W20240503 02:14:23.883402 13355 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240503 02:14:23.883733 13355 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240503 02:14:23.883824 13355 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240503 02:14:23.892498 13355 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240503 02:14:23.892750 13355 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.151.129
I20240503 02:14:23.902549 13355 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.151.148:33391
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknrxqHk/test-tmp/tablet_copy_client_session-itest.0.TabletCopyClientSessionITest.TestCopyFromCrashedSource.1714702442467385-9822-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasknrxqHk/test-tmp/tablet_copy_client_session-itest.0.TabletCopyClientSessionITest.TestCopyFromCrashedSource.1714702442467385-9822-0/minicluster-data/ts-0/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.151.129:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknrxqHk/test-tmp/tablet_copy_client_session-itest.0.TabletCopyClientSessionITest.TestCopyFromCrashedSource.1714702442467385-9822-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.151.129
--webserver_port=0
--tserver_master_addrs=127.9.151.190:34365
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.151.129
--log_dir=/tmp/dist-test-tasknrxqHk/test-tmp/tablet_copy_client_session-itest.0.TabletCopyClientSessionITest.TestCopyFromCrashedSource.1714702442467385-9822-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c39e72a98e124e5a12080bac42105b996594d806
build type FASTDEBUG
built by None at 03 May 2024 01:43:22 UTC on efd1337a2aa3
build id 99
ASAN enabled
I20240503 02:14:23.904006 13355 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240503 02:14:23.905964 13355 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240503 02:14:23.920421 13360 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240503 02:14:23.921016 13363 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240503 02:14:23.921590 13361 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240503 02:14:23.922539 13355 server_base.cc:1034] running on GCE node
I20240503 02:14:23.923513 13355 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240503 02:14:23.924620 13355 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240503 02:14:23.925890 13355 hybrid_clock.cc:648] HybridClock initialized: now 1714702463925820 us; error 60 us; skew 500 ppm
I20240503 02:14:23.926218 13355 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20240503 02:14:23.929222 13355 webserver.cc:460] Webserver started at http://127.9.151.129:35553/ using document root <none> and password file <none>
I20240503 02:14:23.930094 13355 fs_manager.cc:359] Metadata directory not provided
I20240503 02:14:23.930224 13355 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240503 02:14:23.930522 13355 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240503 02:14:23.933367 13355 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknrxqHk/test-tmp/tablet_copy_client_session-itest.0.TabletCopyClientSessionITest.TestCopyFromCrashedSource.1714702442467385-9822-0/minicluster-data/ts-0/data/instance:
uuid: "38821a3d9d854640be3badeb93c01371"
format_stamp: "Formatted at 2024-05-03 02:14:23 on dist-test-slave-twlh"
I20240503 02:14:23.934170 13355 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknrxqHk/test-tmp/tablet_copy_client_session-itest.0.TabletCopyClientSessionITest.TestCopyFromCrashedSource.1714702442467385-9822-0/minicluster-data/ts-0/wal/instance:
uuid: "38821a3d9d854640be3badeb93c01371"
format_stamp: "Formatted at 2024-05-03 02:14:23 on dist-test-slave-twlh"
I20240503 02:14:23.940810 13355 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.008s sys 0.000s
I20240503 02:14:23.944885 13369 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240503 02:14:23.946607 13355 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.000s
I20240503 02:14:23.946863 13355 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknrxqHk/test-tmp/tablet_copy_client_session-itest.0.TabletCopyClientSessionITest.TestCopyFromCrashedSource.1714702442467385-9822-0/minicluster-data/ts-0/data,/tmp/dist-test-tasknrxqHk/test-tmp/tablet_copy_client_session-itest.0.TabletCopyClientSessionITest.TestCopyFromCrashedSource.1714702442467385-9822-0/minicluster-data/ts-0/wal
uuid: "38821a3d9d854640be3badeb93c01371"
format_stamp: "Formatted at 2024-05-03 02:14:23 on dist-test-slave-twlh"
I20240503 02:14:23.947073 13355 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknrxqHk/test-tmp/tablet_copy_client_session-itest.0.TabletCopyClientSessionITest.TestCopyFromCrashedSource.1714702442467385-9822-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasknrxqHk/test-tmp/tablet_copy_client_session-itest.0.TabletCopyClientSessionITest.TestCopyFromCrashedSource.1714702442467385-9822-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasknrxqHk/test-tmp/tablet_copy_client_session-itest.0.TabletCopyClientSessionITest.TestCopyFromCrashedSource.1714702442467385-9822-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240503 02:14:23.966408 13355 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240503 02:14:23.967345 13355 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240503 02:14:23.967628 13355 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240503 02:14:23.968619 13355 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240503 02:14:23.970206 13355 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240503 02:14:23.970283 13355 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240503 02:14:23.970371 13355 ts_tablet_manager.cc:590] Registered 0 tablets
I20240503 02:14:23.970415 13355 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240503 02:14:24.006645 13355 rpc_server.cc:307] RPC server started. Bound to: 127.9.151.129:35867
I20240503 02:14:24.006717 13481 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.151.129:35867 every 8 connection(s)
I20240503 02:14:24.008102 13355 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknrxqHk/test-tmp/tablet_copy_client_session-itest.0.TabletCopyClientSessionITest.TestCopyFromCrashedSource.1714702442467385-9822-0/minicluster-data/ts-0/data/info.pb
I20240503 02:14:24.012828 9822 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknrxqHk/build/asan/bin/kudu as pid 13355
I20240503 02:14:24.013046 9822 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknrxqHk/test-tmp/tablet_copy_client_session-itest.0.TabletCopyClientSessionITest.TestCopyFromCrashedSource.1714702442467385-9822-0/minicluster-data/ts-0/wal/instance
I20240503 02:14:24.017472 9822 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknrxqHk/build/asan/bin/kudu
/tmp/dist-test-tasknrxqHk/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknrxqHk/test-tmp/tablet_copy_client_session-itest.0.TabletCopyClientSessionITest.TestCopyFromCrashedSource.1714702442467385-9822-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasknrxqHk/test-tmp/tablet_copy_client_session-itest.0.TabletCopyClientSessionITest.TestCopyFromCrashedSource.1714702442467385-9822-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknrxqHk/test-tmp/tablet_copy_client_session-itest.0.TabletCopyClientSessionITest.TestCopyFromCrashedSource.1714702442467385-9822-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasknrxqHk/test-tmp/tablet_copy_client_session-itest.0.TabletCopyClientSessionITest.TestCopyFromCrashedSource.1714702442467385-9822-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.151.130:0
--local_ip_for_outbound_sockets=127.9.151.130
--webserver_interface=127.9.151.130
--webserver_port=0
--tserver_master_addrs=127.9.151.190:34365
--builtin_ntp_servers=127.9.151.148:33391
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin with env {}
I20240503 02:14:24.024988 13482 heartbeater.cc:347] Connected to a master server at 127.9.151.190:34365
I20240503 02:14:24.025334 13482 heartbeater.cc:464] Registering TS with master...
I20240503 02:14:24.026242 13482 heartbeater.cc:511] Master 127.9.151.190:34365 requested a full tablet report, sending...
I20240503 02:14:24.028612 13296 ts_manager.cc:194] Registered new tserver with Master: 38821a3d9d854640be3badeb93c01371 (127.9.151.129:35867)
I20240503 02:14:24.030370 13296 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.9.151.129:40489
W20240503 02:14:24.387028 13486 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240503 02:14:24.387388 13486 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240503 02:14:24.387557 13486 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240503 02:14:24.395838 13486 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240503 02:14:24.396104 13486 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.151.130
I20240503 02:14:24.406736 13486 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.151.148:33391
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknrxqHk/test-tmp/tablet_copy_client_session-itest.0.TabletCopyClientSessionITest.TestCopyFromCrashedSource.1714702442467385-9822-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasknrxqHk/test-tmp/tablet_copy_client_session-itest.0.TabletCopyClientSessionITest.TestCopyFromCrashedSource.1714702442467385-9822-0/minicluster-data/ts-1/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.151.130:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknrxqHk/test-tmp/tablet_copy_client_session-itest.0.TabletCopyClientSessionITest.TestCopyFromCrashedSource.1714702442467385-9822-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.151.130
--webserver_port=0
--tserver_master_addrs=127.9.151.190:34365
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.151.130
--log_dir=/tmp/dist-test-tasknrxqHk/test-tmp/tablet_copy_client_session-itest.0.TabletCopyClientSessionITest.TestCopyFromCrashedSource.1714702442467385-9822-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c39e72a98e124e5a12080bac42105b996594d806
build type FASTDEBUG
built by None at 03 May 2024 01:43:22 UTC on efd1337a2aa3
build id 99
ASAN enabled
I20240503 02:14:24.408237 13486 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240503 02:14:24.410127 13486 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240503 02:14:24.425606 13492 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240503 02:14:24.425968 13491 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240503 02:14:24.426512 13494 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240503 02:14:24.427387 13486 server_base.cc:1034] running on GCE node
I20240503 02:14:24.428088 13486 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240503 02:14:24.429340 13486 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240503 02:14:24.430868 13486 hybrid_clock.cc:648] HybridClock initialized: now 1714702464430752 us; error 84 us; skew 500 ppm
I20240503 02:14:24.431249 13486 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20240503 02:14:24.434077 13486 webserver.cc:460] Webserver started at http://127.9.151.130:44487/ using document root <none> and password file <none>
I20240503 02:14:24.435150 13486 fs_manager.cc:359] Metadata directory not provided
I20240503 02:14:24.435350 13486 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240503 02:14:24.435705 13486 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240503 02:14:24.438659 13486 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknrxqHk/test-tmp/tablet_copy_client_session-itest.0.TabletCopyClientSessionITest.TestCopyFromCrashedSource.1714702442467385-9822-0/minicluster-data/ts-1/data/instance:
uuid: "033a76d3974c473b98a3e73990319e3e"
format_stamp: "Formatted at 2024-05-03 02:14:24 on dist-test-slave-twlh"
I20240503 02:14:24.439599 13486 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknrxqHk/test-tmp/tablet_copy_client_session-itest.0.TabletCopyClientSessionITest.TestCopyFromCrashedSource.1714702442467385-9822-0/minicluster-data/ts-1/wal/instance:
uuid: "033a76d3974c473b98a3e73990319e3e"
format_stamp: "Formatted at 2024-05-03 02:14:24 on dist-test-slave-twlh"
I20240503 02:14:24.445987 13486 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.000s
I20240503 02:14:24.450234 13500 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240503 02:14:24.452057 13486 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.000s sys 0.003s
I20240503 02:14:24.452277 13486 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknrxqHk/test-tmp/tablet_copy_client_session-itest.0.TabletCopyClientSessionITest.TestCopyFromCrashedSource.1714702442467385-9822-0/minicluster-data/ts-1/data,/tmp/dist-test-tasknrxqHk/test-tmp/tablet_copy_client_session-itest.0.TabletCopyClientSessionITest.TestCopyFromCrashedSource.1714702442467385-9822-0/minicluster-data/ts-1/wal
uuid: "033a76d3974c473b98a3e73990319e3e"
format_stamp: "Formatted at 2024-05-03 02:14:24 on dist-test-slave-twlh"
I20240503 02:14:24.452523 13486 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknrxqHk/test-tmp/tablet_copy_client_session-itest.0.TabletCopyClientSessionITest.TestCopyFromCrashedSource.1714702442467385-9822-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasknrxqHk/test-tmp/tablet_copy_client_session-itest.0.TabletCopyClientSessionITest.TestCopyFromCrashedSource.1714702442467385-9822-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasknrxqHk/test-tmp/tablet_copy_client_session-itest.0.TabletCopyClientSessionITest.TestCopyFromCrashedSource.1714702442467385-9822-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240503 02:14:24.479445 13486 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240503 02:14:24.480396 13486 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240503 02:14:24.480732 13486 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240503 02:14:24.481894 13486 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240503 02:14:24.483741 13486 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240503 02:14:24.483835 13486 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240503 02:14:24.483989 13486 ts_tablet_manager.cc:590] Registered 0 tablets
I20240503 02:14:24.484052 13486 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240503 02:14:24.521281 13486 rpc_server.cc:307] RPC server started. Bound to: 127.9.151.130:39493
I20240503 02:14:24.521323 13612 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.151.130:39493 every 8 connection(s)
I20240503 02:14:24.522658 13486 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknrxqHk/test-tmp/tablet_copy_client_session-itest.0.TabletCopyClientSessionITest.TestCopyFromCrashedSource.1714702442467385-9822-0/minicluster-data/ts-1/data/info.pb
I20240503 02:14:24.525823 9822 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknrxqHk/build/asan/bin/kudu as pid 13486
I20240503 02:14:24.526125 9822 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknrxqHk/test-tmp/tablet_copy_client_session-itest.0.TabletCopyClientSessionITest.TestCopyFromCrashedSource.1714702442467385-9822-0/minicluster-data/ts-1/wal/instance
I20240503 02:14:24.538062 13613 heartbeater.cc:347] Connected to a master server at 127.9.151.190:34365
I20240503 02:14:24.538417 13613 heartbeater.cc:464] Registering TS with master...
I20240503 02:14:24.539664 13613 heartbeater.cc:511] Master 127.9.151.190:34365 requested a full tablet report, sending...
I20240503 02:14:24.542340 13296 ts_manager.cc:194] Registered new tserver with Master: 033a76d3974c473b98a3e73990319e3e (127.9.151.130:39493)
I20240503 02:14:24.543488 13296 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.9.151.130:36859
I20240503 02:14:24.544288 9822 external_mini_cluster.cc:930] 2 TS(s) registered with all masters
I20240503 02:14:24.560559 9822 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasknrxqHk/build/asan/bin/kudu pid 13486
I20240503 02:14:24.571357 13608 generic_service.cc:196] Checking for leaks (request via RPC)
I20240503 02:14:24.654757 9822 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknrxqHk/build/asan/bin/kudu with pid 13486
I20240503 02:14:24.671443 9822 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasknrxqHk/build/asan/bin/kudu pid 13265
I20240503 02:14:24.673058 13326 generic_service.cc:196] Checking for leaks (request via RPC)
I20240503 02:14:24.778856 9822 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknrxqHk/build/asan/bin/kudu with pid 13265
I20240503 02:14:24.800442 9822 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknrxqHk/build/asan/bin/kudu
/tmp/dist-test-tasknrxqHk/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknrxqHk/test-tmp/tablet_copy_client_session-itest.0.TabletCopyClientSessionITest.TestCopyFromCrashedSource.1714702442467385-9822-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasknrxqHk/test-tmp/tablet_copy_client_session-itest.0.TabletCopyClientSessionITest.TestCopyFromCrashedSource.1714702442467385-9822-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknrxqHk/test-tmp/tablet_copy_client_session-itest.0.TabletCopyClientSessionITest.TestCopyFromCrashedSource.1714702442467385-9822-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasknrxqHk/test-tmp/tablet_copy_client_session-itest.0.TabletCopyClientSessionITest.TestCopyFromCrashedSource.1714702442467385-9822-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.151.190:34365
--webserver_interface=127.9.151.190
--webserver_port=43663
--builtin_ntp_servers=127.9.151.148:33391
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.151.190:34365
--master_tombstone_evicted_tablet_replicas=false with env {}
I20240503 02:14:25.033851 13482 heartbeater.cc:503] Master 127.9.151.190:34365 was elected leader, sending a full tablet report...
W20240503 02:14:25.035593 13482 heartbeater.cc:644] Failed to heartbeat to 127.9.151.190:34365 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.9.151.190:34365: connect: Connection refused (error 111)
W20240503 02:14:25.155704 13626 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240503 02:14:25.156045 13626 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240503 02:14:25.156164 13626 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240503 02:14:25.164423 13626 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20240503 02:14:25.164572 13626 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240503 02:14:25.164618 13626 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20240503 02:14:25.164671 13626 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20240503 02:14:25.174427 13626 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.151.148:33391
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknrxqHk/test-tmp/tablet_copy_client_session-itest.0.TabletCopyClientSessionITest.TestCopyFromCrashedSource.1714702442467385-9822-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasknrxqHk/test-tmp/tablet_copy_client_session-itest.0.TabletCopyClientSessionITest.TestCopyFromCrashedSource.1714702442467385-9822-0/minicluster-data/master-0/wal
--master_tombstone_evicted_tablet_replicas=false
--ipki_ca_key_size=768
--master_addresses=127.9.151.190:34365
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.151.190:34365
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknrxqHk/test-tmp/tablet_copy_client_session-itest.0.TabletCopyClientSessionITest.TestCopyFromCrashedSource.1714702442467385-9822-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.151.190
--webserver_port=43663
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasknrxqHk/test-tmp/tablet_copy_client_session-itest.0.TabletCopyClientSessionITest.TestCopyFromCrashedSource.1714702442467385-9822-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c39e72a98e124e5a12080bac42105b996594d806
build type FASTDEBUG
built by None at 03 May 2024 01:43:22 UTC on efd1337a2aa3
build id 99
ASAN enabled
I20240503 02:14:25.175899 13626 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240503 02:14:25.177731 13626 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240503 02:14:25.190106 13632 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240503 02:14:25.190738 13633 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240503 02:14:25.191337 13635 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240503 02:14:25.192188 13626 server_base.cc:1034] running on GCE node
I20240503 02:14:25.193063 13626 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240503 02:14:25.194693 13626 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240503 02:14:25.195981 13626 hybrid_clock.cc:648] HybridClock initialized: now 1714702465195908 us; error 44 us; skew 500 ppm
I20240503 02:14:25.196369 13626 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20240503 02:14:25.199033 13626 webserver.cc:460] Webserver started at http://127.9.151.190:43663/ using document root <none> and password file <none>
I20240503 02:14:25.199842 13626 fs_manager.cc:359] Metadata directory not provided
I20240503 02:14:25.200011 13626 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240503 02:14:25.206709 13626 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.004s sys 0.002s
I20240503 02:14:25.210484 13641 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240503 02:14:25.212622 13626 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20240503 02:14:25.212863 13626 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknrxqHk/test-tmp/tablet_copy_client_session-itest.0.TabletCopyClientSessionITest.TestCopyFromCrashedSource.1714702442467385-9822-0/minicluster-data/master-0/data,/tmp/dist-test-tasknrxqHk/test-tmp/tablet_copy_client_session-itest.0.TabletCopyClientSessionITest.TestCopyFromCrashedSource.1714702442467385-9822-0/minicluster-data/master-0/wal
uuid: "7039e750de844d2b8b3f0414147e02a2"
format_stamp: "Formatted at 2024-05-03 02:14:23 on dist-test-slave-twlh"
I20240503 02:14:25.213829 13626 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknrxqHk/test-tmp/tablet_copy_client_session-itest.0.TabletCopyClientSessionITest.TestCopyFromCrashedSource.1714702442467385-9822-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasknrxqHk/test-tmp/tablet_copy_client_session-itest.0.TabletCopyClientSessionITest.TestCopyFromCrashedSource.1714702442467385-9822-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasknrxqHk/test-tmp/tablet_copy_client_session-itest.0.TabletCopyClientSessionITest.TestCopyFromCrashedSource.1714702442467385-9822-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240503 02:14:25.259280 13626 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240503 02:14:25.260280 13626 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240503 02:14:25.260588 13626 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240503 02:14:25.281335 13626 rpc_server.cc:307] RPC server started. Bound to: 127.9.151.190:34365
I20240503 02:14:25.281353 13692 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.151.190:34365 every 8 connection(s)
I20240503 02:14:25.283020 13626 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknrxqHk/test-tmp/tablet_copy_client_session-itest.0.TabletCopyClientSessionITest.TestCopyFromCrashedSource.1714702442467385-9822-0/minicluster-data/master-0/data/info.pb
I20240503 02:14:25.283861 9822 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknrxqHk/build/asan/bin/kudu as pid 13626
I20240503 02:14:25.291254 13693 sys_catalog.cc:263] Verifying existing consensus state
I20240503 02:14:25.294765 13693 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7039e750de844d2b8b3f0414147e02a2: Bootstrap starting.
I20240503 02:14:25.311766 13693 log.cc:824] T 00000000000000000000000000000000 P 7039e750de844d2b8b3f0414147e02a2: Log is configured to *not* fsync() on all Append() calls
I20240503 02:14:25.321724 13693 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7039e750de844d2b8b3f0414147e02a2: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240503 02:14:25.322523 13693 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7039e750de844d2b8b3f0414147e02a2: Bootstrap complete.
I20240503 02:14:25.331959 13693 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7039e750de844d2b8b3f0414147e02a2 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7039e750de844d2b8b3f0414147e02a2" member_type: VOTER last_known_addr { host: "127.9.151.190" port: 34365 } }
I20240503 02:14:25.332816 13693 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7039e750de844d2b8b3f0414147e02a2 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7039e750de844d2b8b3f0414147e02a2, State: Initialized, Role: FOLLOWER
I20240503 02:14:25.333541 13693 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7039e750de844d2b8b3f0414147e02a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7039e750de844d2b8b3f0414147e02a2" member_type: VOTER last_known_addr { host: "127.9.151.190" port: 34365 } }
I20240503 02:14:25.333776 13693 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7039e750de844d2b8b3f0414147e02a2 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20240503 02:14:25.333899 13693 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7039e750de844d2b8b3f0414147e02a2 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20240503 02:14:25.334103 13693 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7039e750de844d2b8b3f0414147e02a2 [term 1 FOLLOWER]: Advancing to term 2
I20240503 02:14:25.338493 13693 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7039e750de844d2b8b3f0414147e02a2 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7039e750de844d2b8b3f0414147e02a2" member_type: VOTER last_known_addr { host: "127.9.151.190" port: 34365 } }
I20240503 02:14:25.339186 13693 leader_election.cc:302] T 00000000000000000000000000000000 P 7039e750de844d2b8b3f0414147e02a2 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7039e750de844d2b8b3f0414147e02a2; no voters:
I20240503 02:14:25.339893 13693 leader_election.cc:288] T 00000000000000000000000000000000 P 7039e750de844d2b8b3f0414147e02a2 [CANDIDATE]: Term 2 election: Requested vote from peers
I20240503 02:14:25.340083 13698 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7039e750de844d2b8b3f0414147e02a2 [term 2 FOLLOWER]: Leader election won for term 2
I20240503 02:14:25.341768 13698 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7039e750de844d2b8b3f0414147e02a2 [term 2 LEADER]: Becoming Leader. State: Replica: 7039e750de844d2b8b3f0414147e02a2, State: Running, Role: LEADER
I20240503 02:14:25.342403 13698 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7039e750de844d2b8b3f0414147e02a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7039e750de844d2b8b3f0414147e02a2" member_type: VOTER last_known_addr { host: "127.9.151.190" port: 34365 } }
I20240503 02:14:25.342558 13693 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7039e750de844d2b8b3f0414147e02a2 [sys.catalog]: configured and running, proceeding with master startup.
I20240503 02:14:25.345042 13700 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7039e750de844d2b8b3f0414147e02a2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7039e750de844d2b8b3f0414147e02a2. Latest consensus state: current_term: 2 leader_uuid: "7039e750de844d2b8b3f0414147e02a2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7039e750de844d2b8b3f0414147e02a2" member_type: VOTER last_known_addr { host: "127.9.151.190" port: 34365 } } }
I20240503 02:14:25.345449 13700 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7039e750de844d2b8b3f0414147e02a2 [sys.catalog]: This master's current role is: LEADER
I20240503 02:14:25.346063 13699 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7039e750de844d2b8b3f0414147e02a2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "7039e750de844d2b8b3f0414147e02a2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7039e750de844d2b8b3f0414147e02a2" member_type: VOTER last_known_addr { host: "127.9.151.190" port: 34365 } } }
I20240503 02:14:25.346534 13699 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7039e750de844d2b8b3f0414147e02a2 [sys.catalog]: This master's current role is: LEADER
I20240503 02:14:25.348201 13705 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20240503 02:14:25.355422 13705 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20240503 02:14:25.357118 13705 catalog_manager.cc:1259] Loaded cluster ID: 0c3deaa48eae4c97ab80c02219ac1dfc
I20240503 02:14:25.357220 13705 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20240503 02:14:25.360348 13705 catalog_manager.cc:1504] Loading token signing keys...
I20240503 02:14:25.361673 13705 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7039e750de844d2b8b3f0414147e02a2: Loaded TSK: 0
I20240503 02:14:25.363101 13705 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20240503 02:14:26.045413 13482 heartbeater.cc:347] Connected to a master server at 127.9.151.190:34365
I20240503 02:14:26.045679 13482 heartbeater.cc:503] Master 127.9.151.190:34365 was elected leader, sending a full tablet report...
I20240503 02:14:26.047798 13658 master_service.cc:432] Got heartbeat from unknown tserver (permanent_uuid: "38821a3d9d854640be3badeb93c01371" instance_seqno: 1714702463997937) as {username='slave'} at 127.9.151.129:57965; Asking this server to re-register.
I20240503 02:14:26.048861 13482 heartbeater.cc:464] Registering TS with master...
I20240503 02:14:26.049289 13482 heartbeater.cc:511] Master 127.9.151.190:34365 requested a full tablet report, sending...
I20240503 02:14:26.051048 13658 ts_manager.cc:194] Registered new tserver with Master: 38821a3d9d854640be3badeb93c01371 (127.9.151.129:35867)
I20240503 02:14:26.057041 9822 external_mini_cluster.cc:930] 1 TS(s) registered with all masters
I20240503 02:14:26.057233 9822 test_util.cc:268] Using random seed: 1197638659
I20240503 02:14:26.085341 13658 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:37658:
name: "test-workload"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20240503 02:14:26.114274 13417 tablet_service.cc:1467] Processing CreateTablet for tablet 6ee6909dc83f497c8e687f2cc5fb7d08 (DEFAULT_TABLE table=test-workload [id=690a0c5aaaad4103b441e71c6ea1c636]), partition=RANGE (key) PARTITION UNBOUNDED
I20240503 02:14:26.115695 13417 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6ee6909dc83f497c8e687f2cc5fb7d08. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:14:26.128305 13728 tablet_bootstrap.cc:492] T 6ee6909dc83f497c8e687f2cc5fb7d08 P 38821a3d9d854640be3badeb93c01371: Bootstrap starting.
I20240503 02:14:26.131865 13728 tablet_bootstrap.cc:654] T 6ee6909dc83f497c8e687f2cc5fb7d08 P 38821a3d9d854640be3badeb93c01371: Neither blocks nor log segments found. Creating new log.
I20240503 02:14:26.133008 13728 log.cc:824] T 6ee6909dc83f497c8e687f2cc5fb7d08 P 38821a3d9d854640be3badeb93c01371: Log is configured to *not* fsync() on all Append() calls
I20240503 02:14:26.136477 13728 tablet_bootstrap.cc:492] T 6ee6909dc83f497c8e687f2cc5fb7d08 P 38821a3d9d854640be3badeb93c01371: No bootstrap required, opened a new log
I20240503 02:14:26.136914 13728 ts_tablet_manager.cc:1367] T 6ee6909dc83f497c8e687f2cc5fb7d08 P 38821a3d9d854640be3badeb93c01371: Time spent bootstrapping tablet: real 0.009s user 0.007s sys 0.000s
I20240503 02:14:26.143958 13728 raft_consensus.cc:348] T 6ee6909dc83f497c8e687f2cc5fb7d08 P 38821a3d9d854640be3badeb93c01371 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "38821a3d9d854640be3badeb93c01371" member_type: VOTER last_known_addr { host: "127.9.151.129" port: 35867 } }
I20240503 02:14:26.144378 13728 raft_consensus.cc:374] T 6ee6909dc83f497c8e687f2cc5fb7d08 P 38821a3d9d854640be3badeb93c01371 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:14:26.144483 13728 raft_consensus.cc:729] T 6ee6909dc83f497c8e687f2cc5fb7d08 P 38821a3d9d854640be3badeb93c01371 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 38821a3d9d854640be3badeb93c01371, State: Initialized, Role: FOLLOWER
I20240503 02:14:26.145202 13728 consensus_queue.cc:260] T 6ee6909dc83f497c8e687f2cc5fb7d08 P 38821a3d9d854640be3badeb93c01371 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "38821a3d9d854640be3badeb93c01371" member_type: VOTER last_known_addr { host: "127.9.151.129" port: 35867 } }
I20240503 02:14:26.145419 13728 raft_consensus.cc:388] T 6ee6909dc83f497c8e687f2cc5fb7d08 P 38821a3d9d854640be3badeb93c01371 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20240503 02:14:26.145558 13728 raft_consensus.cc:482] T 6ee6909dc83f497c8e687f2cc5fb7d08 P 38821a3d9d854640be3badeb93c01371 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20240503 02:14:26.145723 13728 raft_consensus.cc:3037] T 6ee6909dc83f497c8e687f2cc5fb7d08 P 38821a3d9d854640be3badeb93c01371 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:14:26.148286 13728 raft_consensus.cc:504] T 6ee6909dc83f497c8e687f2cc5fb7d08 P 38821a3d9d854640be3badeb93c01371 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "38821a3d9d854640be3badeb93c01371" member_type: VOTER last_known_addr { host: "127.9.151.129" port: 35867 } }
I20240503 02:14:26.148929 13728 leader_election.cc:302] T 6ee6909dc83f497c8e687f2cc5fb7d08 P 38821a3d9d854640be3badeb93c01371 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 38821a3d9d854640be3badeb93c01371; no voters:
I20240503 02:14:26.149637 13728 leader_election.cc:288] T 6ee6909dc83f497c8e687f2cc5fb7d08 P 38821a3d9d854640be3badeb93c01371 [CANDIDATE]: Term 1 election: Requested vote from peers
I20240503 02:14:26.149830 13730 raft_consensus.cc:2781] T 6ee6909dc83f497c8e687f2cc5fb7d08 P 38821a3d9d854640be3badeb93c01371 [term 1 FOLLOWER]: Leader election won for term 1
I20240503 02:14:26.150445 13730 raft_consensus.cc:686] T 6ee6909dc83f497c8e687f2cc5fb7d08 P 38821a3d9d854640be3badeb93c01371 [term 1 LEADER]: Becoming Leader. State: Replica: 38821a3d9d854640be3badeb93c01371, State: Running, Role: LEADER
I20240503 02:14:26.151147 13730 consensus_queue.cc:237] T 6ee6909dc83f497c8e687f2cc5fb7d08 P 38821a3d9d854640be3badeb93c01371 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "38821a3d9d854640be3badeb93c01371" member_type: VOTER last_known_addr { host: "127.9.151.129" port: 35867 } }
I20240503 02:14:26.151578 13728 ts_tablet_manager.cc:1398] T 6ee6909dc83f497c8e687f2cc5fb7d08 P 38821a3d9d854640be3badeb93c01371: Time spent starting tablet: real 0.015s user 0.011s sys 0.005s
I20240503 02:14:26.156862 13658 catalog_manager.cc:5484] T 6ee6909dc83f497c8e687f2cc5fb7d08 P 38821a3d9d854640be3badeb93c01371 reported cstate change: term changed from 0 to 1, leader changed from <none> to 38821a3d9d854640be3badeb93c01371 (127.9.151.129). New cstate: current_term: 1 leader_uuid: "38821a3d9d854640be3badeb93c01371" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "38821a3d9d854640be3badeb93c01371" member_type: VOTER last_known_addr { host: "127.9.151.129" port: 35867 } health_report { overall_health: HEALTHY } } }
I20240503 02:14:26.876979 9822 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknrxqHk/build/asan/bin/kudu
/tmp/dist-test-tasknrxqHk/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknrxqHk/test-tmp/tablet_copy_client_session-itest.0.TabletCopyClientSessionITest.TestCopyFromCrashedSource.1714702442467385-9822-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasknrxqHk/test-tmp/tablet_copy_client_session-itest.0.TabletCopyClientSessionITest.TestCopyFromCrashedSource.1714702442467385-9822-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknrxqHk/test-tmp/tablet_copy_client_session-itest.0.TabletCopyClientSessionITest.TestCopyFromCrashedSource.1714702442467385-9822-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasknrxqHk/test-tmp/tablet_copy_client_session-itest.0.TabletCopyClientSessionITest.TestCopyFromCrashedSource.1714702442467385-9822-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.151.130:39493
--local_ip_for_outbound_sockets=127.9.151.130
--tserver_master_addrs=127.9.151.190:34365
--webserver_port=44487
--webserver_interface=127.9.151.130
--builtin_ntp_servers=127.9.151.148:33391
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin with env {}
W20240503 02:14:27.238545 13747 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240503 02:14:27.238927 13747 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240503 02:14:27.239068 13747 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240503 02:14:27.247087 13747 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240503 02:14:27.247299 13747 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.151.130
I20240503 02:14:27.257421 13747 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.151.148:33391
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknrxqHk/test-tmp/tablet_copy_client_session-itest.0.TabletCopyClientSessionITest.TestCopyFromCrashedSource.1714702442467385-9822-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasknrxqHk/test-tmp/tablet_copy_client_session-itest.0.TabletCopyClientSessionITest.TestCopyFromCrashedSource.1714702442467385-9822-0/minicluster-data/ts-1/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.151.130:39493
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknrxqHk/test-tmp/tablet_copy_client_session-itest.0.TabletCopyClientSessionITest.TestCopyFromCrashedSource.1714702442467385-9822-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.151.130
--webserver_port=44487
--tserver_master_addrs=127.9.151.190:34365
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.151.130
--log_dir=/tmp/dist-test-tasknrxqHk/test-tmp/tablet_copy_client_session-itest.0.TabletCopyClientSessionITest.TestCopyFromCrashedSource.1714702442467385-9822-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c39e72a98e124e5a12080bac42105b996594d806
build type FASTDEBUG
built by None at 03 May 2024 01:43:22 UTC on efd1337a2aa3
build id 99
ASAN enabled
I20240503 02:14:27.258956 13747 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240503 02:14:27.260771 13747 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240503 02:14:27.275130 13753 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240503 02:14:27.275725 13755 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240503 02:14:27.276238 13752 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240503 02:14:27.277022 13747 server_base.cc:1034] running on GCE node
I20240503 02:14:27.277694 13747 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240503 02:14:27.278839 13747 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240503 02:14:27.280122 13747 hybrid_clock.cc:648] HybridClock initialized: now 1714702467280032 us; error 59 us; skew 500 ppm
I20240503 02:14:27.280892 13747 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20240503 02:14:27.284252 13747 webserver.cc:460] Webserver started at http://127.9.151.130:44487/ using document root <none> and password file <none>
I20240503 02:14:27.285259 13747 fs_manager.cc:359] Metadata directory not provided
I20240503 02:14:27.285430 13747 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240503 02:14:27.292747 13747 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.004s sys 0.000s
I20240503 02:14:27.296463 13761 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240503 02:14:27.298334 13747 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.001s
I20240503 02:14:27.298547 13747 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknrxqHk/test-tmp/tablet_copy_client_session-itest.0.TabletCopyClientSessionITest.TestCopyFromCrashedSource.1714702442467385-9822-0/minicluster-data/ts-1/data,/tmp/dist-test-tasknrxqHk/test-tmp/tablet_copy_client_session-itest.0.TabletCopyClientSessionITest.TestCopyFromCrashedSource.1714702442467385-9822-0/minicluster-data/ts-1/wal
uuid: "033a76d3974c473b98a3e73990319e3e"
format_stamp: "Formatted at 2024-05-03 02:14:24 on dist-test-slave-twlh"
I20240503 02:14:27.299743 13747 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknrxqHk/test-tmp/tablet_copy_client_session-itest.0.TabletCopyClientSessionITest.TestCopyFromCrashedSource.1714702442467385-9822-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasknrxqHk/test-tmp/tablet_copy_client_session-itest.0.TabletCopyClientSessionITest.TestCopyFromCrashedSource.1714702442467385-9822-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasknrxqHk/test-tmp/tablet_copy_client_session-itest.0.TabletCopyClientSessionITest.TestCopyFromCrashedSource.1714702442467385-9822-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240503 02:14:27.331056 13747 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240503 02:14:27.332008 13747 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240503 02:14:27.332299 13747 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240503 02:14:27.333400 13747 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240503 02:14:27.335227 13747 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240503 02:14:27.335330 13747 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240503 02:14:27.335412 13747 ts_tablet_manager.cc:590] Registered 0 tablets
I20240503 02:14:27.335471 13747 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240503 02:14:27.378054 13747 rpc_server.cc:307] RPC server started. Bound to: 127.9.151.130:39493
I20240503 02:14:27.378208 13873 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.151.130:39493 every 8 connection(s)
I20240503 02:14:27.379830 13747 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknrxqHk/test-tmp/tablet_copy_client_session-itest.0.TabletCopyClientSessionITest.TestCopyFromCrashedSource.1714702442467385-9822-0/minicluster-data/ts-1/data/info.pb
I20240503 02:14:27.382414 9822 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknrxqHk/build/asan/bin/kudu as pid 13747
W20240503 02:14:27.388981 13477 generic_service.cc:139] {username='slave'} at 127.0.0.1:54136 forcing change of non-runtime-safe flag fault_crash_on_handle_tc_fetch_data
I20240503 02:14:27.389235 13477 generic_service.cc:175] {username='slave'} at 127.0.0.1:54136 changed flags via RPC: fault_crash_on_handle_tc_fetch_data from '0' to '1.0'
I20240503 02:14:27.399538 13874 heartbeater.cc:347] Connected to a master server at 127.9.151.190:34365
I20240503 02:14:27.399981 13874 heartbeater.cc:464] Registering TS with master...
I20240503 02:14:27.401508 13874 heartbeater.cc:511] Master 127.9.151.190:34365 requested a full tablet report, sending...
I20240503 02:14:27.403592 13880 ts_tablet_manager.cc:902] T 6ee6909dc83f497c8e687f2cc5fb7d08 P 033a76d3974c473b98a3e73990319e3e: Initiating tablet copy from peer 38821a3d9d854640be3badeb93c01371 (127.9.151.129:35867)
I20240503 02:14:27.403789 13658 ts_manager.cc:194] Registered new tserver with Master: 033a76d3974c473b98a3e73990319e3e (127.9.151.130:39493)
I20240503 02:14:27.405427 13658 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.9.151.130:54113
I20240503 02:14:27.405841 13880 tablet_copy_client.cc:308] T 6ee6909dc83f497c8e687f2cc5fb7d08 P 033a76d3974c473b98a3e73990319e3e: tablet copy: Beginning tablet copy session from remote peer at address 127.9.151.129:35867
I20240503 02:14:27.414167 13457 tablet_copy_service.cc:140] P 38821a3d9d854640be3badeb93c01371: Received BeginTabletCopySession request for tablet 6ee6909dc83f497c8e687f2cc5fb7d08 from peer 033a76d3974c473b98a3e73990319e3e ({username='slave'} at 127.9.151.130:36707)
I20240503 02:14:27.414487 13457 tablet_copy_service.cc:161] P 38821a3d9d854640be3badeb93c01371: Beginning new tablet copy session on tablet 6ee6909dc83f497c8e687f2cc5fb7d08 from peer 033a76d3974c473b98a3e73990319e3e at {username='slave'} at 127.9.151.130:36707: session id = 033a76d3974c473b98a3e73990319e3e-6ee6909dc83f497c8e687f2cc5fb7d08
I20240503 02:14:27.417932 13457 tablet_copy_source_session.cc:205] T 6ee6909dc83f497c8e687f2cc5fb7d08 P 38821a3d9d854640be3badeb93c01371: Tablet Copy: opened 0 blocks and 1 log segments
I20240503 02:14:27.424232 13880 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6ee6909dc83f497c8e687f2cc5fb7d08. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:14:27.435849 13880 tablet_copy_client.cc:783] T 6ee6909dc83f497c8e687f2cc5fb7d08 P 033a76d3974c473b98a3e73990319e3e: tablet copy: Starting download of 0 data blocks...
I20240503 02:14:27.436190 13880 tablet_copy_client.cc:647] T 6ee6909dc83f497c8e687f2cc5fb7d08 P 033a76d3974c473b98a3e73990319e3e: tablet copy: Starting download of 1 WAL segments...
W20240503 02:14:27.437945 13457 fault_injection.cc:43] FAULT INJECTION ENABLED!
W20240503 02:14:27.438068 13457 fault_injection.cc:44] THIS SERVER MAY CRASH!
E20240503 02:14:27.438163 13457 fault_injection.cc:59] injecting fault for kudu (pid 13355): FLAGS_fault_crash_on_handle_tc_fetch_data (process will exit)
W20240503 02:14:27.458441 13880 ts_tablet_manager.cc:980] T 6ee6909dc83f497c8e687f2cc5fb7d08 P 033a76d3974c473b98a3e73990319e3e: Tablet Copy: Unable to fetch data from remote peer 38821a3d9d854640be3badeb93c01371 (127.9.151.129:35867): Network error: Unable to download WAL segment with seq. number 1: unable to fetch data from remote: recv got EOF from 127.9.151.129:35867 (error 108)
I20240503 02:14:27.458636 13880 tablet_replica.cc:330] T 6ee6909dc83f497c8e687f2cc5fb7d08 P 033a76d3974c473b98a3e73990319e3e: stopping tablet replica
I20240503 02:14:27.458901 13880 raft_consensus.cc:2229] T 6ee6909dc83f497c8e687f2cc5fb7d08 P 033a76d3974c473b98a3e73990319e3e [term 1 NON_PARTICIPANT]: Raft consensus shutting down.
I20240503 02:14:27.459106 13880 raft_consensus.cc:2258] T 6ee6909dc83f497c8e687f2cc5fb7d08 P 033a76d3974c473b98a3e73990319e3e [term 1 NON_PARTICIPANT]: Raft consensus is shut down!
W20240503 02:14:27.460409 13880 tablet_copy_client.cc:1108] T 6ee6909dc83f497c8e687f2cc5fb7d08 P 033a76d3974c473b98a3e73990319e3e: tablet copy: Unable to close tablet copy session: Network error: failure ending tablet copy session: Client connection negotiation failed: client connection to 127.9.151.129:35867: connect: Connection refused (error 111)
W20240503 02:14:27.461172 13880 tablet_metadata.cc:476] failed to load DataDirGroup from superblock: Already present: tried to load directory group for tablet 6ee6909dc83f497c8e687f2cc5fb7d08 but one is already registered
I20240503 02:14:27.461426 13880 ts_tablet_manager.cc:1875] T 6ee6909dc83f497c8e687f2cc5fb7d08 P 033a76d3974c473b98a3e73990319e3e: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240503 02:14:27.466344 13880 ts_tablet_manager.cc:1888] T 6ee6909dc83f497c8e687f2cc5fb7d08 P 033a76d3974c473b98a3e73990319e3e: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.0
I20240503 02:14:27.466531 13880 log.cc:1196] T 6ee6909dc83f497c8e687f2cc5fb7d08 P 033a76d3974c473b98a3e73990319e3e: Deleting WAL directory at /tmp/dist-test-tasknrxqHk/test-tmp/tablet_copy_client_session-itest.0.TabletCopyClientSessionITest.TestCopyFromCrashedSource.1714702442467385-9822-0/minicluster-data/ts-1/wal/wals/6ee6909dc83f497c8e687f2cc5fb7d08
I20240503 02:14:27.471130 9822 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknrxqHk/build/asan/bin/kudu
/tmp/dist-test-tasknrxqHk/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknrxqHk/test-tmp/tablet_copy_client_session-itest.0.TabletCopyClientSessionITest.TestCopyFromCrashedSource.1714702442467385-9822-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasknrxqHk/test-tmp/tablet_copy_client_session-itest.0.TabletCopyClientSessionITest.TestCopyFromCrashedSource.1714702442467385-9822-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknrxqHk/test-tmp/tablet_copy_client_session-itest.0.TabletCopyClientSessionITest.TestCopyFromCrashedSource.1714702442467385-9822-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasknrxqHk/test-tmp/tablet_copy_client_session-itest.0.TabletCopyClientSessionITest.TestCopyFromCrashedSource.1714702442467385-9822-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.151.129:35867
--local_ip_for_outbound_sockets=127.9.151.129
--tserver_master_addrs=127.9.151.190:34365
--webserver_port=35553
--webserver_interface=127.9.151.129
--builtin_ntp_servers=127.9.151.148:33391
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin with env {}
W20240503 02:14:27.829496 13886 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240503 02:14:27.829838 13886 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240503 02:14:27.829931 13886 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240503 02:14:27.837936 13886 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240503 02:14:27.838148 13886 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.151.129
I20240503 02:14:27.847836 13886 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.151.148:33391
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknrxqHk/test-tmp/tablet_copy_client_session-itest.0.TabletCopyClientSessionITest.TestCopyFromCrashedSource.1714702442467385-9822-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasknrxqHk/test-tmp/tablet_copy_client_session-itest.0.TabletCopyClientSessionITest.TestCopyFromCrashedSource.1714702442467385-9822-0/minicluster-data/ts-0/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.151.129:35867
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknrxqHk/test-tmp/tablet_copy_client_session-itest.0.TabletCopyClientSessionITest.TestCopyFromCrashedSource.1714702442467385-9822-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.151.129
--webserver_port=35553
--tserver_master_addrs=127.9.151.190:34365
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.151.129
--log_dir=/tmp/dist-test-tasknrxqHk/test-tmp/tablet_copy_client_session-itest.0.TabletCopyClientSessionITest.TestCopyFromCrashedSource.1714702442467385-9822-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c39e72a98e124e5a12080bac42105b996594d806
build type FASTDEBUG
built by None at 03 May 2024 01:43:22 UTC on efd1337a2aa3
build id 99
ASAN enabled
I20240503 02:14:27.849476 13886 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240503 02:14:27.851420 13886 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240503 02:14:27.866348 13891 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240503 02:14:27.866765 13894 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240503 02:14:27.867311 13892 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240503 02:14:27.868049 13886 server_base.cc:1034] running on GCE node
I20240503 02:14:27.869062 13886 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240503 02:14:27.870411 13886 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240503 02:14:27.871719 13886 hybrid_clock.cc:648] HybridClock initialized: now 1714702467871652 us; error 91 us; skew 500 ppm
I20240503 02:14:27.872143 13886 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20240503 02:14:27.875610 13886 webserver.cc:460] Webserver started at http://127.9.151.129:35553/ using document root <none> and password file <none>
I20240503 02:14:27.876710 13886 fs_manager.cc:359] Metadata directory not provided
I20240503 02:14:27.876904 13886 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240503 02:14:27.883965 13886 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.004s sys 0.000s
I20240503 02:14:27.887655 13900 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240503 02:14:27.889634 13886 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20240503 02:14:27.889828 13886 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknrxqHk/test-tmp/tablet_copy_client_session-itest.0.TabletCopyClientSessionITest.TestCopyFromCrashedSource.1714702442467385-9822-0/minicluster-data/ts-0/data,/tmp/dist-test-tasknrxqHk/test-tmp/tablet_copy_client_session-itest.0.TabletCopyClientSessionITest.TestCopyFromCrashedSource.1714702442467385-9822-0/minicluster-data/ts-0/wal
uuid: "38821a3d9d854640be3badeb93c01371"
format_stamp: "Formatted at 2024-05-03 02:14:23 on dist-test-slave-twlh"
I20240503 02:14:27.890861 13886 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknrxqHk/test-tmp/tablet_copy_client_session-itest.0.TabletCopyClientSessionITest.TestCopyFromCrashedSource.1714702442467385-9822-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasknrxqHk/test-tmp/tablet_copy_client_session-itest.0.TabletCopyClientSessionITest.TestCopyFromCrashedSource.1714702442467385-9822-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasknrxqHk/test-tmp/tablet_copy_client_session-itest.0.TabletCopyClientSessionITest.TestCopyFromCrashedSource.1714702442467385-9822-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240503 02:14:27.909528 13886 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240503 02:14:27.910573 13886 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240503 02:14:27.910925 13886 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240503 02:14:27.912120 13886 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240503 02:14:27.914590 13907 ts_tablet_manager.cc:522] Loading tablet metadata (0/1 complete)
I20240503 02:14:27.920775 13886 ts_tablet_manager.cc:559] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20240503 02:14:27.920938 13886 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.007s user 0.001s sys 0.000s
I20240503 02:14:27.921072 13886 ts_tablet_manager.cc:574] Registering tablets (0/1 complete)
I20240503 02:14:27.924261 13886 ts_tablet_manager.cc:590] Registered 1 tablets
I20240503 02:14:27.924362 13886 ts_tablet_manager.cc:569] Time spent register tablets: real 0.003s user 0.002s sys 0.000s
I20240503 02:14:27.925235 13907 tablet_bootstrap.cc:492] T 6ee6909dc83f497c8e687f2cc5fb7d08 P 38821a3d9d854640be3badeb93c01371: Bootstrap starting.
I20240503 02:14:27.971540 13886 rpc_server.cc:307] RPC server started. Bound to: 127.9.151.129:35867
I20240503 02:14:27.971572 14013 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.151.129:35867 every 8 connection(s)
I20240503 02:14:27.973465 13886 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknrxqHk/test-tmp/tablet_copy_client_session-itest.0.TabletCopyClientSessionITest.TestCopyFromCrashedSource.1714702442467385-9822-0/minicluster-data/ts-0/data/info.pb
I20240503 02:14:27.976898 9822 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknrxqHk/build/asan/bin/kudu as pid 13886
I20240503 02:14:27.991108 14014 heartbeater.cc:347] Connected to a master server at 127.9.151.190:34365
I20240503 02:14:27.991456 14014 heartbeater.cc:464] Registering TS with master...
I20240503 02:14:27.992625 14014 heartbeater.cc:511] Master 127.9.151.190:34365 requested a full tablet report, sending...
I20240503 02:14:27.996122 13658 ts_manager.cc:194] Re-registered known tserver with Master: 38821a3d9d854640be3badeb93c01371 (127.9.151.129:35867)
I20240503 02:14:27.997817 13658 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.9.151.129:46229
I20240503 02:14:28.025456 13907 log.cc:824] T 6ee6909dc83f497c8e687f2cc5fb7d08 P 38821a3d9d854640be3badeb93c01371: Log is configured to *not* fsync() on all Append() calls
I20240503 02:14:28.409305 13874 heartbeater.cc:503] Master 127.9.151.190:34365 was elected leader, sending a full tablet report...
W20240503 02:14:28.411725 13658 catalog_manager.cc:5397] Tablet 6ee6909dc83f497c8e687f2cc5fb7d08 (table test-workload [id=690a0c5aaaad4103b441e71c6ea1c636]) has failed on TS 033a76d3974c473b98a3e73990319e3e (127.9.151.130:39493): Network error: Unable to download WAL segment with seq. number 1: unable to fetch data from remote: recv got EOF from 127.9.151.129:35867 (error 108)
I20240503 02:14:28.616071 13907 tablet_bootstrap.cc:492] T 6ee6909dc83f497c8e687f2cc5fb7d08 P 38821a3d9d854640be3badeb93c01371: Bootstrap replayed 1/1 log segments. Stats: ops{read=212 overwritten=0 applied=212 ignored=0} inserts{seen=10550 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240503 02:14:28.616798 13907 tablet_bootstrap.cc:492] T 6ee6909dc83f497c8e687f2cc5fb7d08 P 38821a3d9d854640be3badeb93c01371: Bootstrap complete.
I20240503 02:14:28.618110 13907 ts_tablet_manager.cc:1367] T 6ee6909dc83f497c8e687f2cc5fb7d08 P 38821a3d9d854640be3badeb93c01371: Time spent bootstrapping tablet: real 0.693s user 0.623s sys 0.064s
I20240503 02:14:28.622807 13907 raft_consensus.cc:348] T 6ee6909dc83f497c8e687f2cc5fb7d08 P 38821a3d9d854640be3badeb93c01371 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "38821a3d9d854640be3badeb93c01371" member_type: VOTER last_known_addr { host: "127.9.151.129" port: 35867 } }
I20240503 02:14:28.623625 13907 raft_consensus.cc:729] T 6ee6909dc83f497c8e687f2cc5fb7d08 P 38821a3d9d854640be3badeb93c01371 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 38821a3d9d854640be3badeb93c01371, State: Initialized, Role: FOLLOWER
I20240503 02:14:28.624265 13907 consensus_queue.cc:260] T 6ee6909dc83f497c8e687f2cc5fb7d08 P 38821a3d9d854640be3badeb93c01371 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 212, Last appended: 1.212, Last appended by leader: 212, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "38821a3d9d854640be3badeb93c01371" member_type: VOTER last_known_addr { host: "127.9.151.129" port: 35867 } }
I20240503 02:14:28.624471 13907 raft_consensus.cc:388] T 6ee6909dc83f497c8e687f2cc5fb7d08 P 38821a3d9d854640be3badeb93c01371 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20240503 02:14:28.624562 13907 raft_consensus.cc:482] T 6ee6909dc83f497c8e687f2cc5fb7d08 P 38821a3d9d854640be3badeb93c01371 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20240503 02:14:28.624707 13907 raft_consensus.cc:3037] T 6ee6909dc83f497c8e687f2cc5fb7d08 P 38821a3d9d854640be3badeb93c01371 [term 1 FOLLOWER]: Advancing to term 2
I20240503 02:14:28.628618 13907 raft_consensus.cc:504] T 6ee6909dc83f497c8e687f2cc5fb7d08 P 38821a3d9d854640be3badeb93c01371 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "38821a3d9d854640be3badeb93c01371" member_type: VOTER last_known_addr { host: "127.9.151.129" port: 35867 } }
I20240503 02:14:28.629205 13907 leader_election.cc:302] T 6ee6909dc83f497c8e687f2cc5fb7d08 P 38821a3d9d854640be3badeb93c01371 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 38821a3d9d854640be3badeb93c01371; no voters:
I20240503 02:14:28.629912 13907 leader_election.cc:288] T 6ee6909dc83f497c8e687f2cc5fb7d08 P 38821a3d9d854640be3badeb93c01371 [CANDIDATE]: Term 2 election: Requested vote from peers
I20240503 02:14:28.630136 14021 raft_consensus.cc:2781] T 6ee6909dc83f497c8e687f2cc5fb7d08 P 38821a3d9d854640be3badeb93c01371 [term 2 FOLLOWER]: Leader election won for term 2
I20240503 02:14:28.631119 14014 heartbeater.cc:503] Master 127.9.151.190:34365 was elected leader, sending a full tablet report...
I20240503 02:14:28.631234 14021 raft_consensus.cc:686] T 6ee6909dc83f497c8e687f2cc5fb7d08 P 38821a3d9d854640be3badeb93c01371 [term 2 LEADER]: Becoming Leader. State: Replica: 38821a3d9d854640be3badeb93c01371, State: Running, Role: LEADER
I20240503 02:14:28.631678 13907 ts_tablet_manager.cc:1398] T 6ee6909dc83f497c8e687f2cc5fb7d08 P 38821a3d9d854640be3badeb93c01371: Time spent starting tablet: real 0.013s user 0.015s sys 0.001s
I20240503 02:14:28.631773 14021 consensus_queue.cc:237] T 6ee6909dc83f497c8e687f2cc5fb7d08 P 38821a3d9d854640be3badeb93c01371 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 212, Committed index: 212, Last appended: 1.212, Last appended by leader: 212, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "38821a3d9d854640be3badeb93c01371" member_type: VOTER last_known_addr { host: "127.9.151.129" port: 35867 } }
I20240503 02:14:28.635668 13658 catalog_manager.cc:5484] T 6ee6909dc83f497c8e687f2cc5fb7d08 P 38821a3d9d854640be3badeb93c01371 reported cstate change: term changed from 1 to 2. New cstate: current_term: 2 leader_uuid: "38821a3d9d854640be3badeb93c01371" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "38821a3d9d854640be3badeb93c01371" member_type: VOTER last_known_addr { host: "127.9.151.129" port: 35867 } health_report { overall_health: HEALTHY } } }
I20240503 02:14:28.643091 14030 ts_tablet_manager.cc:902] T 6ee6909dc83f497c8e687f2cc5fb7d08 P 033a76d3974c473b98a3e73990319e3e: Initiating tablet copy from peer 38821a3d9d854640be3badeb93c01371 (127.9.151.129:35867)
I20240503 02:14:28.644731 14030 tablet_copy_client.cc:308] T 6ee6909dc83f497c8e687f2cc5fb7d08 P 033a76d3974c473b98a3e73990319e3e: tablet copy: Beginning tablet copy session from remote peer at address 127.9.151.129:35867
I20240503 02:14:28.651960 13989 tablet_copy_service.cc:140] P 38821a3d9d854640be3badeb93c01371: Received BeginTabletCopySession request for tablet 6ee6909dc83f497c8e687f2cc5fb7d08 from peer 033a76d3974c473b98a3e73990319e3e ({username='slave'} at 127.9.151.130:45591)
I20240503 02:14:28.652243 13989 tablet_copy_service.cc:161] P 38821a3d9d854640be3badeb93c01371: Beginning new tablet copy session on tablet 6ee6909dc83f497c8e687f2cc5fb7d08 from peer 033a76d3974c473b98a3e73990319e3e at {username='slave'} at 127.9.151.130:45591: session id = 033a76d3974c473b98a3e73990319e3e-6ee6909dc83f497c8e687f2cc5fb7d08
I20240503 02:14:28.654796 13989 tablet_copy_source_session.cc:205] T 6ee6909dc83f497c8e687f2cc5fb7d08 P 38821a3d9d854640be3badeb93c01371: Tablet Copy: opened 0 blocks and 1 log segments
I20240503 02:14:28.656737 14030 ts_tablet_manager.cc:1875] T 6ee6909dc83f497c8e687f2cc5fb7d08 P 033a76d3974c473b98a3e73990319e3e: Deleting tablet data with delete state TABLET_DATA_COPYING
I20240503 02:14:28.662343 14030 ts_tablet_manager.cc:1888] T 6ee6909dc83f497c8e687f2cc5fb7d08 P 033a76d3974c473b98a3e73990319e3e: tablet deleted with delete type TABLET_DATA_COPYING: last-logged OpId 1.0
I20240503 02:14:28.662724 14030 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6ee6909dc83f497c8e687f2cc5fb7d08. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:14:28.665220 14030 tablet_copy_client.cc:783] T 6ee6909dc83f497c8e687f2cc5fb7d08 P 033a76d3974c473b98a3e73990319e3e: tablet copy: Starting download of 0 data blocks...
I20240503 02:14:28.665472 14030 tablet_copy_client.cc:647] T 6ee6909dc83f497c8e687f2cc5fb7d08 P 033a76d3974c473b98a3e73990319e3e: tablet copy: Starting download of 1 WAL segments...
I20240503 02:14:28.670776 14030 tablet_copy_client.cc:523] T 6ee6909dc83f497c8e687f2cc5fb7d08 P 033a76d3974c473b98a3e73990319e3e: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240503 02:14:28.673969 14030 tablet_bootstrap.cc:492] T 6ee6909dc83f497c8e687f2cc5fb7d08 P 033a76d3974c473b98a3e73990319e3e: Bootstrap starting.
I20240503 02:14:28.739753 14030 log.cc:824] T 6ee6909dc83f497c8e687f2cc5fb7d08 P 033a76d3974c473b98a3e73990319e3e: Log is configured to *not* fsync() on all Append() calls
I20240503 02:14:29.316972 14030 tablet_bootstrap.cc:492] T 6ee6909dc83f497c8e687f2cc5fb7d08 P 033a76d3974c473b98a3e73990319e3e: Bootstrap replayed 1/1 log segments. Stats: ops{read=213 overwritten=0 applied=213 ignored=0} inserts{seen=10550 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240503 02:14:29.317487 14030 tablet_bootstrap.cc:492] T 6ee6909dc83f497c8e687f2cc5fb7d08 P 033a76d3974c473b98a3e73990319e3e: Bootstrap complete.
I20240503 02:14:29.317997 14030 ts_tablet_manager.cc:1367] T 6ee6909dc83f497c8e687f2cc5fb7d08 P 033a76d3974c473b98a3e73990319e3e: Time spent bootstrapping tablet: real 0.644s user 0.578s sys 0.066s
I20240503 02:14:29.322515 14030 raft_consensus.cc:348] T 6ee6909dc83f497c8e687f2cc5fb7d08 P 033a76d3974c473b98a3e73990319e3e [term 2 NON_PARTICIPANT]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "38821a3d9d854640be3badeb93c01371" member_type: VOTER last_known_addr { host: "127.9.151.129" port: 35867 } }
I20240503 02:14:29.322937 14030 raft_consensus.cc:729] T 6ee6909dc83f497c8e687f2cc5fb7d08 P 033a76d3974c473b98a3e73990319e3e [term 2 NON_PARTICIPANT]: Becoming Follower/Learner. State: Replica: 033a76d3974c473b98a3e73990319e3e, State: Initialized, Role: NON_PARTICIPANT
I20240503 02:14:29.323915 14030 consensus_queue.cc:260] T 6ee6909dc83f497c8e687f2cc5fb7d08 P 033a76d3974c473b98a3e73990319e3e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 213, Last appended: 2.213, Last appended by leader: 213, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "38821a3d9d854640be3badeb93c01371" member_type: VOTER last_known_addr { host: "127.9.151.129" port: 35867 } }
I20240503 02:14:29.326046 14030 ts_tablet_manager.cc:1398] T 6ee6909dc83f497c8e687f2cc5fb7d08 P 033a76d3974c473b98a3e73990319e3e: Time spent starting tablet: real 0.008s user 0.010s sys 0.000s
I20240503 02:14:29.326216 9822 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasknrxqHk/build/asan/bin/kudu pid 13886
I20240503 02:14:29.327087 13989 tablet_copy_service.cc:342] P 38821a3d9d854640be3badeb93c01371: Request end of tablet copy session 033a76d3974c473b98a3e73990319e3e-6ee6909dc83f497c8e687f2cc5fb7d08 received from {username='slave'} at 127.9.151.130:45591
I20240503 02:14:29.327283 13989 tablet_copy_service.cc:433] P 38821a3d9d854640be3badeb93c01371: ending tablet copy session 033a76d3974c473b98a3e73990319e3e-6ee6909dc83f497c8e687f2cc5fb7d08 on tablet 6ee6909dc83f497c8e687f2cc5fb7d08 with peer 033a76d3974c473b98a3e73990319e3e
I20240503 02:14:29.327365 14009 generic_service.cc:196] Checking for leaks (request via RPC)
I20240503 02:14:29.436558 9822 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknrxqHk/build/asan/bin/kudu with pid 13886
I20240503 02:14:29.457723 9822 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasknrxqHk/build/asan/bin/kudu pid 13747
I20240503 02:14:29.459048 13869 generic_service.cc:196] Checking for leaks (request via RPC)
I20240503 02:14:29.565485 9822 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknrxqHk/build/asan/bin/kudu with pid 13747
I20240503 02:14:29.585515 9822 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasknrxqHk/build/asan/bin/kudu pid 13626
I20240503 02:14:29.586763 13688 generic_service.cc:196] Checking for leaks (request via RPC)
I20240503 02:14:29.693145 9822 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknrxqHk/build/asan/bin/kudu with pid 13626
2024-05-03T02:14:29Z chronyd exiting
[ OK ] TabletCopyClientSessionITest.TestCopyFromCrashedSource (6870 ms)
[ RUN ] TabletCopyClientSessionITest.TestTabletCopyWithBusySource
2024-05-03T02:14:29Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2024-05-03T02:14:29Z Disabled control of system clock
I20240503 02:14:29.760443 9822 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknrxqHk/build/asan/bin/kudu
/tmp/dist-test-tasknrxqHk/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknrxqHk/test-tmp/tablet_copy_client_session-itest.0.TabletCopyClientSessionITest.TestTabletCopyWithBusySource.1714702442467385-9822-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasknrxqHk/test-tmp/tablet_copy_client_session-itest.0.TabletCopyClientSessionITest.TestTabletCopyWithBusySource.1714702442467385-9822-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknrxqHk/test-tmp/tablet_copy_client_session-itest.0.TabletCopyClientSessionITest.TestTabletCopyWithBusySource.1714702442467385-9822-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasknrxqHk/test-tmp/tablet_copy_client_session-itest.0.TabletCopyClientSessionITest.TestTabletCopyWithBusySource.1714702442467385-9822-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.151.190:38767
--webserver_interface=127.9.151.190
--webserver_port=0
--builtin_ntp_servers=127.9.151.148:43735
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.151.190:38767
--master_tombstone_evicted_tablet_replicas=false with env {}
W20240503 02:14:30.122311 14045 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240503 02:14:30.122659 14045 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240503 02:14:30.122814 14045 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240503 02:14:30.131229 14045 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20240503 02:14:30.131372 14045 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240503 02:14:30.131420 14045 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20240503 02:14:30.131480 14045 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20240503 02:14:30.141606 14045 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.151.148:43735
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknrxqHk/test-tmp/tablet_copy_client_session-itest.0.TabletCopyClientSessionITest.TestTabletCopyWithBusySource.1714702442467385-9822-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasknrxqHk/test-tmp/tablet_copy_client_session-itest.0.TabletCopyClientSessionITest.TestTabletCopyWithBusySource.1714702442467385-9822-0/minicluster-data/master-0/wal
--master_tombstone_evicted_tablet_replicas=false
--ipki_ca_key_size=768
--master_addresses=127.9.151.190:38767
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.151.190:38767
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknrxqHk/test-tmp/tablet_copy_client_session-itest.0.TabletCopyClientSessionITest.TestTabletCopyWithBusySource.1714702442467385-9822-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.151.190
--webserver_port=0
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasknrxqHk/test-tmp/tablet_copy_client_session-itest.0.TabletCopyClientSessionITest.TestTabletCopyWithBusySource.1714702442467385-9822-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c39e72a98e124e5a12080bac42105b996594d806
build type FASTDEBUG
built by None at 03 May 2024 01:43:22 UTC on efd1337a2aa3
build id 99
ASAN enabled
I20240503 02:14:30.143023 14045 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240503 02:14:30.144857 14045 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240503 02:14:30.156862 14050 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240503 02:14:30.157399 14051 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240503 02:14:30.157953 14053 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240503 02:14:30.158779 14045 server_base.cc:1034] running on GCE node
I20240503 02:14:30.159657 14045 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240503 02:14:30.161304 14045 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240503 02:14:30.162590 14045 hybrid_clock.cc:648] HybridClock initialized: now 1714702470162510 us; error 61 us; skew 500 ppm
I20240503 02:14:30.163022 14045 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20240503 02:14:30.165874 14045 webserver.cc:460] Webserver started at http://127.9.151.190:33137/ using document root <none> and password file <none>
I20240503 02:14:30.166723 14045 fs_manager.cc:359] Metadata directory not provided
I20240503 02:14:30.166893 14045 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240503 02:14:30.167225 14045 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240503 02:14:30.170109 14045 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknrxqHk/test-tmp/tablet_copy_client_session-itest.0.TabletCopyClientSessionITest.TestTabletCopyWithBusySource.1714702442467385-9822-0/minicluster-data/master-0/data/instance:
uuid: "e0c1752ef8e54d56a0d646d3bccdaf63"
format_stamp: "Formatted at 2024-05-03 02:14:30 on dist-test-slave-twlh"
I20240503 02:14:30.171046 14045 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknrxqHk/test-tmp/tablet_copy_client_session-itest.0.TabletCopyClientSessionITest.TestTabletCopyWithBusySource.1714702442467385-9822-0/minicluster-data/master-0/wal/instance:
uuid: "e0c1752ef8e54d56a0d646d3bccdaf63"
format_stamp: "Formatted at 2024-05-03 02:14:30 on dist-test-slave-twlh"
I20240503 02:14:30.176957 14045 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20240503 02:14:30.180871 14059 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240503 02:14:30.182559 14045 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20240503 02:14:30.182823 14045 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknrxqHk/test-tmp/tablet_copy_client_session-itest.0.TabletCopyClientSessionITest.TestTabletCopyWithBusySource.1714702442467385-9822-0/minicluster-data/master-0/data,/tmp/dist-test-tasknrxqHk/test-tmp/tablet_copy_client_session-itest.0.TabletCopyClientSessionITest.TestTabletCopyWithBusySource.1714702442467385-9822-0/minicluster-data/master-0/wal
uuid: "e0c1752ef8e54d56a0d646d3bccdaf63"
format_stamp: "Formatted at 2024-05-03 02:14:30 on dist-test-slave-twlh"
I20240503 02:14:30.183030 14045 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknrxqHk/test-tmp/tablet_copy_client_session-itest.0.TabletCopyClientSessionITest.TestTabletCopyWithBusySource.1714702442467385-9822-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasknrxqHk/test-tmp/tablet_copy_client_session-itest.0.TabletCopyClientSessionITest.TestTabletCopyWithBusySource.1714702442467385-9822-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasknrxqHk/test-tmp/tablet_copy_client_session-itest.0.TabletCopyClientSessionITest.TestTabletCopyWithBusySource.1714702442467385-9822-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240503 02:14:30.214221 14045 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240503 02:14:30.215220 14045 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240503 02:14:30.215538 14045 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240503 02:14:30.237901 14110 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.151.190:38767 every 8 connection(s)
I20240503 02:14:30.237901 14045 rpc_server.cc:307] RPC server started. Bound to: 127.9.151.190:38767
I20240503 02:14:30.239729 14045 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknrxqHk/test-tmp/tablet_copy_client_session-itest.0.TabletCopyClientSessionITest.TestTabletCopyWithBusySource.1714702442467385-9822-0/minicluster-data/master-0/data/info.pb
I20240503 02:14:30.244565 14111 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:14:30.246805 9822 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknrxqHk/build/asan/bin/kudu as pid 14045
I20240503 02:14:30.247067 9822 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknrxqHk/test-tmp/tablet_copy_client_session-itest.0.TabletCopyClientSessionITest.TestTabletCopyWithBusySource.1714702442467385-9822-0/minicluster-data/master-0/wal/instance
I20240503 02:14:30.257339 14111 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e0c1752ef8e54d56a0d646d3bccdaf63: Bootstrap starting.
I20240503 02:14:30.261060 14111 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e0c1752ef8e54d56a0d646d3bccdaf63: Neither blocks nor log segments found. Creating new log.
I20240503 02:14:30.262264 14111 log.cc:824] T 00000000000000000000000000000000 P e0c1752ef8e54d56a0d646d3bccdaf63: Log is configured to *not* fsync() on all Append() calls
I20240503 02:14:30.265822 14111 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e0c1752ef8e54d56a0d646d3bccdaf63: No bootstrap required, opened a new log
I20240503 02:14:30.272935 14111 raft_consensus.cc:348] T 00000000000000000000000000000000 P e0c1752ef8e54d56a0d646d3bccdaf63 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e0c1752ef8e54d56a0d646d3bccdaf63" member_type: VOTER last_known_addr { host: "127.9.151.190" port: 38767 } }
I20240503 02:14:30.273380 14111 raft_consensus.cc:374] T 00000000000000000000000000000000 P e0c1752ef8e54d56a0d646d3bccdaf63 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:14:30.273488 14111 raft_consensus.cc:729] T 00000000000000000000000000000000 P e0c1752ef8e54d56a0d646d3bccdaf63 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e0c1752ef8e54d56a0d646d3bccdaf63, State: Initialized, Role: FOLLOWER
I20240503 02:14:30.274207 14111 consensus_queue.cc:260] T 00000000000000000000000000000000 P e0c1752ef8e54d56a0d646d3bccdaf63 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e0c1752ef8e54d56a0d646d3bccdaf63" member_type: VOTER last_known_addr { host: "127.9.151.190" port: 38767 } }
I20240503 02:14:30.274416 14111 raft_consensus.cc:388] T 00000000000000000000000000000000 P e0c1752ef8e54d56a0d646d3bccdaf63 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20240503 02:14:30.274533 14111 raft_consensus.cc:482] T 00000000000000000000000000000000 P e0c1752ef8e54d56a0d646d3bccdaf63 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20240503 02:14:30.274739 14111 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e0c1752ef8e54d56a0d646d3bccdaf63 [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:14:30.277195 14111 raft_consensus.cc:504] T 00000000000000000000000000000000 P e0c1752ef8e54d56a0d646d3bccdaf63 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e0c1752ef8e54d56a0d646d3bccdaf63" member_type: VOTER last_known_addr { host: "127.9.151.190" port: 38767 } }
I20240503 02:14:30.277869 14111 leader_election.cc:302] T 00000000000000000000000000000000 P e0c1752ef8e54d56a0d646d3bccdaf63 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e0c1752ef8e54d56a0d646d3bccdaf63; no voters:
I20240503 02:14:30.278540 14111 leader_election.cc:288] T 00000000000000000000000000000000 P e0c1752ef8e54d56a0d646d3bccdaf63 [CANDIDATE]: Term 1 election: Requested vote from peers
I20240503 02:14:30.278769 14116 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e0c1752ef8e54d56a0d646d3bccdaf63 [term 1 FOLLOWER]: Leader election won for term 1
I20240503 02:14:30.279462 14116 raft_consensus.cc:686] T 00000000000000000000000000000000 P e0c1752ef8e54d56a0d646d3bccdaf63 [term 1 LEADER]: Becoming Leader. State: Replica: e0c1752ef8e54d56a0d646d3bccdaf63, State: Running, Role: LEADER
I20240503 02:14:30.280105 14116 consensus_queue.cc:237] T 00000000000000000000000000000000 P e0c1752ef8e54d56a0d646d3bccdaf63 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e0c1752ef8e54d56a0d646d3bccdaf63" member_type: VOTER last_known_addr { host: "127.9.151.190" port: 38767 } }
I20240503 02:14:30.280823 14111 sys_catalog.cc:564] T 00000000000000000000000000000000 P e0c1752ef8e54d56a0d646d3bccdaf63 [sys.catalog]: configured and running, proceeding with master startup.
I20240503 02:14:30.283936 14118 sys_catalog.cc:455] T 00000000000000000000000000000000 P e0c1752ef8e54d56a0d646d3bccdaf63 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e0c1752ef8e54d56a0d646d3bccdaf63. Latest consensus state: current_term: 1 leader_uuid: "e0c1752ef8e54d56a0d646d3bccdaf63" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e0c1752ef8e54d56a0d646d3bccdaf63" member_type: VOTER last_known_addr { host: "127.9.151.190" port: 38767 } } }
I20240503 02:14:30.284348 14118 sys_catalog.cc:458] T 00000000000000000000000000000000 P e0c1752ef8e54d56a0d646d3bccdaf63 [sys.catalog]: This master's current role is: LEADER
I20240503 02:14:30.286851 14122 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20240503 02:14:30.286522 14117 sys_catalog.cc:455] T 00000000000000000000000000000000 P e0c1752ef8e54d56a0d646d3bccdaf63 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e0c1752ef8e54d56a0d646d3bccdaf63" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e0c1752ef8e54d56a0d646d3bccdaf63" member_type: VOTER last_known_addr { host: "127.9.151.190" port: 38767 } } }
I20240503 02:14:30.287091 14117 sys_catalog.cc:458] T 00000000000000000000000000000000 P e0c1752ef8e54d56a0d646d3bccdaf63 [sys.catalog]: This master's current role is: LEADER
I20240503 02:14:30.293272 14122 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20240503 02:14:30.305840 14122 catalog_manager.cc:1347] Generated new cluster ID: 0e481206868d4979948cc7ff4ad193ec
I20240503 02:14:30.306002 14122 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20240503 02:14:30.335409 14122 catalog_manager.cc:1370] Generated new certificate authority record
I20240503 02:14:30.337569 14122 catalog_manager.cc:1504] Loading token signing keys...
I20240503 02:14:30.348029 14122 catalog_manager.cc:5857] T 00000000000000000000000000000000 P e0c1752ef8e54d56a0d646d3bccdaf63: Generated new TSK 0
I20240503 02:14:30.349138 14122 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20240503 02:14:30.369378 9822 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknrxqHk/build/asan/bin/kudu
/tmp/dist-test-tasknrxqHk/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknrxqHk/test-tmp/tablet_copy_client_session-itest.0.TabletCopyClientSessionITest.TestTabletCopyWithBusySource.1714702442467385-9822-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasknrxqHk/test-tmp/tablet_copy_client_session-itest.0.TabletCopyClientSessionITest.TestTabletCopyWithBusySource.1714702442467385-9822-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknrxqHk/test-tmp/tablet_copy_client_session-itest.0.TabletCopyClientSessionITest.TestTabletCopyWithBusySource.1714702442467385-9822-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasknrxqHk/test-tmp/tablet_copy_client_session-itest.0.TabletCopyClientSessionITest.TestTabletCopyWithBusySource.1714702442467385-9822-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.151.129:0
--local_ip_for_outbound_sockets=127.9.151.129
--webserver_interface=127.9.151.129
--webserver_port=0
--tserver_master_addrs=127.9.151.190:38767
--builtin_ntp_servers=127.9.151.148:43735
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--num_tablets_to_copy_simultaneously=20 with env {}
W20240503 02:14:30.748028 14135 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240503 02:14:30.748396 14135 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240503 02:14:30.748528 14135 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240503 02:14:30.756588 14135 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240503 02:14:30.756866 14135 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.151.129
I20240503 02:14:30.768061 14135 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.151.148:43735
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknrxqHk/test-tmp/tablet_copy_client_session-itest.0.TabletCopyClientSessionITest.TestTabletCopyWithBusySource.1714702442467385-9822-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasknrxqHk/test-tmp/tablet_copy_client_session-itest.0.TabletCopyClientSessionITest.TestTabletCopyWithBusySource.1714702442467385-9822-0/minicluster-data/ts-0/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.151.129:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknrxqHk/test-tmp/tablet_copy_client_session-itest.0.TabletCopyClientSessionITest.TestTabletCopyWithBusySource.1714702442467385-9822-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.151.129
--webserver_port=0
--tserver_master_addrs=127.9.151.190:38767
--num_tablets_to_copy_simultaneously=20
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.151.129
--log_dir=/tmp/dist-test-tasknrxqHk/test-tmp/tablet_copy_client_session-itest.0.TabletCopyClientSessionITest.TestTabletCopyWithBusySource.1714702442467385-9822-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c39e72a98e124e5a12080bac42105b996594d806
build type FASTDEBUG
built by None at 03 May 2024 01:43:22 UTC on efd1337a2aa3
build id 99
ASAN enabled
I20240503 02:14:30.769608 14135 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240503 02:14:30.771605 14135 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240503 02:14:30.788245 14143 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240503 02:14:30.788661 14141 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240503 02:14:30.789342 14140 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240503 02:14:30.790004 14135 server_base.cc:1034] running on GCE node
I20240503 02:14:30.790819 14135 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240503 02:14:30.792006 14135 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240503 02:14:30.793316 14135 hybrid_clock.cc:648] HybridClock initialized: now 1714702470793281 us; error 98 us; skew 500 ppm
I20240503 02:14:30.793709 14135 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20240503 02:14:30.796720 14135 webserver.cc:460] Webserver started at http://127.9.151.129:43777/ using document root <none> and password file <none>
I20240503 02:14:30.797710 14135 fs_manager.cc:359] Metadata directory not provided
I20240503 02:14:30.797900 14135 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240503 02:14:30.798261 14135 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240503 02:14:30.801295 14135 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknrxqHk/test-tmp/tablet_copy_client_session-itest.0.TabletCopyClientSessionITest.TestTabletCopyWithBusySource.1714702442467385-9822-0/minicluster-data/ts-0/data/instance:
uuid: "27f29b79cf9b42f4a5b9838965bbb72d"
format_stamp: "Formatted at 2024-05-03 02:14:30 on dist-test-slave-twlh"
I20240503 02:14:30.802193 14135 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknrxqHk/test-tmp/tablet_copy_client_session-itest.0.TabletCopyClientSessionITest.TestTabletCopyWithBusySource.1714702442467385-9822-0/minicluster-data/ts-0/wal/instance:
uuid: "27f29b79cf9b42f4a5b9838965bbb72d"
format_stamp: "Formatted at 2024-05-03 02:14:30 on dist-test-slave-twlh"
I20240503 02:14:30.808274 14135 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.001s
I20240503 02:14:30.812855 14149 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240503 02:14:30.814992 14135 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.006s sys 0.000s
I20240503 02:14:30.815222 14135 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknrxqHk/test-tmp/tablet_copy_client_session-itest.0.TabletCopyClientSessionITest.TestTabletCopyWithBusySource.1714702442467385-9822-0/minicluster-data/ts-0/data,/tmp/dist-test-tasknrxqHk/test-tmp/tablet_copy_client_session-itest.0.TabletCopyClientSessionITest.TestTabletCopyWithBusySource.1714702442467385-9822-0/minicluster-data/ts-0/wal
uuid: "27f29b79cf9b42f4a5b9838965bbb72d"
format_stamp: "Formatted at 2024-05-03 02:14:30 on dist-test-slave-twlh"
I20240503 02:14:30.815443 14135 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknrxqHk/test-tmp/tablet_copy_client_session-itest.0.TabletCopyClientSessionITest.TestTabletCopyWithBusySource.1714702442467385-9822-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasknrxqHk/test-tmp/tablet_copy_client_session-itest.0.TabletCopyClientSessionITest.TestTabletCopyWithBusySource.1714702442467385-9822-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasknrxqHk/test-tmp/tablet_copy_client_session-itest.0.TabletCopyClientSessionITest.TestTabletCopyWithBusySource.1714702442467385-9822-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240503 02:14:30.839779 14135 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240503 02:14:30.841133 14135 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240503 02:14:30.841531 14135 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240503 02:14:30.842912 14135 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240503 02:14:30.844775 14135 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240503 02:14:30.844877 14135 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240503 02:14:30.845007 14135 ts_tablet_manager.cc:590] Registered 0 tablets
I20240503 02:14:30.845067 14135 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240503 02:14:30.885679 14135 rpc_server.cc:307] RPC server started. Bound to: 127.9.151.129:33313
I20240503 02:14:30.885697 14261 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.151.129:33313 every 8 connection(s)
I20240503 02:14:30.887482 14135 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknrxqHk/test-tmp/tablet_copy_client_session-itest.0.TabletCopyClientSessionITest.TestTabletCopyWithBusySource.1714702442467385-9822-0/minicluster-data/ts-0/data/info.pb
I20240503 02:14:30.887887 9822 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknrxqHk/build/asan/bin/kudu as pid 14135
I20240503 02:14:30.888092 9822 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknrxqHk/test-tmp/tablet_copy_client_session-itest.0.TabletCopyClientSessionITest.TestTabletCopyWithBusySource.1714702442467385-9822-0/minicluster-data/ts-0/wal/instance
I20240503 02:14:30.892587 9822 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknrxqHk/build/asan/bin/kudu
/tmp/dist-test-tasknrxqHk/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknrxqHk/test-tmp/tablet_copy_client_session-itest.0.TabletCopyClientSessionITest.TestTabletCopyWithBusySource.1714702442467385-9822-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasknrxqHk/test-tmp/tablet_copy_client_session-itest.0.TabletCopyClientSessionITest.TestTabletCopyWithBusySource.1714702442467385-9822-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknrxqHk/test-tmp/tablet_copy_client_session-itest.0.TabletCopyClientSessionITest.TestTabletCopyWithBusySource.1714702442467385-9822-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasknrxqHk/test-tmp/tablet_copy_client_session-itest.0.TabletCopyClientSessionITest.TestTabletCopyWithBusySource.1714702442467385-9822-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.151.130:0
--local_ip_for_outbound_sockets=127.9.151.130
--webserver_interface=127.9.151.130
--webserver_port=0
--tserver_master_addrs=127.9.151.190:38767
--builtin_ntp_servers=127.9.151.148:43735
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--num_tablets_to_copy_simultaneously=20 with env {}
I20240503 02:14:30.915026 14262 heartbeater.cc:347] Connected to a master server at 127.9.151.190:38767
I20240503 02:14:30.915766 14262 heartbeater.cc:464] Registering TS with master...
I20240503 02:14:30.917351 14262 heartbeater.cc:511] Master 127.9.151.190:38767 requested a full tablet report, sending...
I20240503 02:14:30.920441 14076 ts_manager.cc:194] Registered new tserver with Master: 27f29b79cf9b42f4a5b9838965bbb72d (127.9.151.129:33313)
I20240503 02:14:30.922169 14076 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.9.151.129:43341
W20240503 02:14:31.288076 14266 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240503 02:14:31.288667 14266 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240503 02:14:31.288877 14266 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240503 02:14:31.297437 14266 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240503 02:14:31.297677 14266 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.151.130
I20240503 02:14:31.307921 14266 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.151.148:43735
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknrxqHk/test-tmp/tablet_copy_client_session-itest.0.TabletCopyClientSessionITest.TestTabletCopyWithBusySource.1714702442467385-9822-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasknrxqHk/test-tmp/tablet_copy_client_session-itest.0.TabletCopyClientSessionITest.TestTabletCopyWithBusySource.1714702442467385-9822-0/minicluster-data/ts-1/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.151.130:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknrxqHk/test-tmp/tablet_copy_client_session-itest.0.TabletCopyClientSessionITest.TestTabletCopyWithBusySource.1714702442467385-9822-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.151.130
--webserver_port=0
--tserver_master_addrs=127.9.151.190:38767
--num_tablets_to_copy_simultaneously=20
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.151.130
--log_dir=/tmp/dist-test-tasknrxqHk/test-tmp/tablet_copy_client_session-itest.0.TabletCopyClientSessionITest.TestTabletCopyWithBusySource.1714702442467385-9822-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c39e72a98e124e5a12080bac42105b996594d806
build type FASTDEBUG
built by None at 03 May 2024 01:43:22 UTC on efd1337a2aa3
build id 99
ASAN enabled
I20240503 02:14:31.309422 14266 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240503 02:14:31.311376 14266 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240503 02:14:31.327383 14271 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240503 02:14:31.328112 14272 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240503 02:14:31.328522 14274 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240503 02:14:31.329454 14266 server_base.cc:1034] running on GCE node
I20240503 02:14:31.330443 14266 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240503 02:14:31.331763 14266 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240503 02:14:31.333124 14266 hybrid_clock.cc:648] HybridClock initialized: now 1714702471333017 us; error 55 us; skew 500 ppm
I20240503 02:14:31.333633 14266 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20240503 02:14:31.341609 14266 webserver.cc:460] Webserver started at http://127.9.151.130:40243/ using document root <none> and password file <none>
I20240503 02:14:31.342720 14266 fs_manager.cc:359] Metadata directory not provided
I20240503 02:14:31.342954 14266 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240503 02:14:31.343274 14266 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240503 02:14:31.346161 14266 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknrxqHk/test-tmp/tablet_copy_client_session-itest.0.TabletCopyClientSessionITest.TestTabletCopyWithBusySource.1714702442467385-9822-0/minicluster-data/ts-1/data/instance:
uuid: "5f665335495c43dcbc53e072f6b1f44f"
format_stamp: "Formatted at 2024-05-03 02:14:31 on dist-test-slave-twlh"
I20240503 02:14:31.347430 14266 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknrxqHk/test-tmp/tablet_copy_client_session-itest.0.TabletCopyClientSessionITest.TestTabletCopyWithBusySource.1714702442467385-9822-0/minicluster-data/ts-1/wal/instance:
uuid: "5f665335495c43dcbc53e072f6b1f44f"
format_stamp: "Formatted at 2024-05-03 02:14:31 on dist-test-slave-twlh"
I20240503 02:14:31.354841 14266 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.007s sys 0.001s
I20240503 02:14:31.359220 14280 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240503 02:14:31.361253 14266 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20240503 02:14:31.361487 14266 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknrxqHk/test-tmp/tablet_copy_client_session-itest.0.TabletCopyClientSessionITest.TestTabletCopyWithBusySource.1714702442467385-9822-0/minicluster-data/ts-1/data,/tmp/dist-test-tasknrxqHk/test-tmp/tablet_copy_client_session-itest.0.TabletCopyClientSessionITest.TestTabletCopyWithBusySource.1714702442467385-9822-0/minicluster-data/ts-1/wal
uuid: "5f665335495c43dcbc53e072f6b1f44f"
format_stamp: "Formatted at 2024-05-03 02:14:31 on dist-test-slave-twlh"
I20240503 02:14:31.361717 14266 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknrxqHk/test-tmp/tablet_copy_client_session-itest.0.TabletCopyClientSessionITest.TestTabletCopyWithBusySource.1714702442467385-9822-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasknrxqHk/test-tmp/tablet_copy_client_session-itest.0.TabletCopyClientSessionITest.TestTabletCopyWithBusySource.1714702442467385-9822-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasknrxqHk/test-tmp/tablet_copy_client_session-itest.0.TabletCopyClientSessionITest.TestTabletCopyWithBusySource.1714702442467385-9822-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240503 02:14:31.381233 14266 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240503 02:14:31.382251 14266 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240503 02:14:31.382575 14266 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240503 02:14:31.383829 14266 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240503 02:14:31.386018 14266 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240503 02:14:31.386126 14266 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240503 02:14:31.386273 14266 ts_tablet_manager.cc:590] Registered 0 tablets
I20240503 02:14:31.386370 14266 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.001s sys 0.000s
I20240503 02:14:31.425242 14266 rpc_server.cc:307] RPC server started. Bound to: 127.9.151.130:41429
I20240503 02:14:31.425369 14392 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.151.130:41429 every 8 connection(s)
I20240503 02:14:31.426791 14266 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknrxqHk/test-tmp/tablet_copy_client_session-itest.0.TabletCopyClientSessionITest.TestTabletCopyWithBusySource.1714702442467385-9822-0/minicluster-data/ts-1/data/info.pb
I20240503 02:14:31.433429 9822 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknrxqHk/build/asan/bin/kudu as pid 14266
I20240503 02:14:31.433663 9822 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknrxqHk/test-tmp/tablet_copy_client_session-itest.0.TabletCopyClientSessionITest.TestTabletCopyWithBusySource.1714702442467385-9822-0/minicluster-data/ts-1/wal/instance
I20240503 02:14:31.443346 14393 heartbeater.cc:347] Connected to a master server at 127.9.151.190:38767
I20240503 02:14:31.443698 14393 heartbeater.cc:464] Registering TS with master...
I20240503 02:14:31.444619 14393 heartbeater.cc:511] Master 127.9.151.190:38767 requested a full tablet report, sending...
I20240503 02:14:31.446512 14076 ts_manager.cc:194] Registered new tserver with Master: 5f665335495c43dcbc53e072f6b1f44f (127.9.151.130:41429)
I20240503 02:14:31.447424 14076 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.9.151.130:44323
I20240503 02:14:31.450951 9822 external_mini_cluster.cc:930] 2 TS(s) registered with all masters
I20240503 02:14:31.469939 9822 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasknrxqHk/build/asan/bin/kudu pid 14266
I20240503 02:14:31.481309 14388 generic_service.cc:196] Checking for leaks (request via RPC)
I20240503 02:14:31.568902 9822 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknrxqHk/build/asan/bin/kudu with pid 14266
I20240503 02:14:31.587060 9822 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasknrxqHk/build/asan/bin/kudu pid 14045
I20240503 02:14:31.588841 14106 generic_service.cc:196] Checking for leaks (request via RPC)
I20240503 02:14:31.696071 9822 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknrxqHk/build/asan/bin/kudu with pid 14045
I20240503 02:14:31.718194 9822 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknrxqHk/build/asan/bin/kudu
/tmp/dist-test-tasknrxqHk/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknrxqHk/test-tmp/tablet_copy_client_session-itest.0.TabletCopyClientSessionITest.TestTabletCopyWithBusySource.1714702442467385-9822-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasknrxqHk/test-tmp/tablet_copy_client_session-itest.0.TabletCopyClientSessionITest.TestTabletCopyWithBusySource.1714702442467385-9822-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknrxqHk/test-tmp/tablet_copy_client_session-itest.0.TabletCopyClientSessionITest.TestTabletCopyWithBusySource.1714702442467385-9822-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasknrxqHk/test-tmp/tablet_copy_client_session-itest.0.TabletCopyClientSessionITest.TestTabletCopyWithBusySource.1714702442467385-9822-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.151.190:38767
--webserver_interface=127.9.151.190
--webserver_port=33137
--builtin_ntp_servers=127.9.151.148:43735
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.151.190:38767
--master_tombstone_evicted_tablet_replicas=false with env {}
I20240503 02:14:31.925664 14262 heartbeater.cc:503] Master 127.9.151.190:38767 was elected leader, sending a full tablet report...
W20240503 02:14:31.927460 14262 heartbeater.cc:644] Failed to heartbeat to 127.9.151.190:38767 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.9.151.190:38767: connect: Connection refused (error 111)
W20240503 02:14:32.094434 14406 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240503 02:14:32.094817 14406 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240503 02:14:32.094971 14406 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240503 02:14:32.102974 14406 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20240503 02:14:32.103140 14406 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240503 02:14:32.103216 14406 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20240503 02:14:32.103313 14406 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20240503 02:14:32.113219 14406 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.151.148:43735
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknrxqHk/test-tmp/tablet_copy_client_session-itest.0.TabletCopyClientSessionITest.TestTabletCopyWithBusySource.1714702442467385-9822-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasknrxqHk/test-tmp/tablet_copy_client_session-itest.0.TabletCopyClientSessionITest.TestTabletCopyWithBusySource.1714702442467385-9822-0/minicluster-data/master-0/wal
--master_tombstone_evicted_tablet_replicas=false
--ipki_ca_key_size=768
--master_addresses=127.9.151.190:38767
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.151.190:38767
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknrxqHk/test-tmp/tablet_copy_client_session-itest.0.TabletCopyClientSessionITest.TestTabletCopyWithBusySource.1714702442467385-9822-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.151.190
--webserver_port=33137
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasknrxqHk/test-tmp/tablet_copy_client_session-itest.0.TabletCopyClientSessionITest.TestTabletCopyWithBusySource.1714702442467385-9822-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c39e72a98e124e5a12080bac42105b996594d806
build type FASTDEBUG
built by None at 03 May 2024 01:43:22 UTC on efd1337a2aa3
build id 99
ASAN enabled
I20240503 02:14:32.114769 14406 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240503 02:14:32.116640 14406 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240503 02:14:32.130733 14412 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240503 02:14:32.131139 14413 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240503 02:14:32.131748 14415 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240503 02:14:32.132516 14406 server_base.cc:1034] running on GCE node
I20240503 02:14:32.133466 14406 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240503 02:14:32.135282 14406 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240503 02:14:32.136613 14406 hybrid_clock.cc:648] HybridClock initialized: now 1714702472136549 us; error 36 us; skew 500 ppm
I20240503 02:14:32.136977 14406 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20240503 02:14:32.139681 14406 webserver.cc:460] Webserver started at http://127.9.151.190:33137/ using document root <none> and password file <none>
I20240503 02:14:32.140487 14406 fs_manager.cc:359] Metadata directory not provided
I20240503 02:14:32.140645 14406 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240503 02:14:32.146935 14406 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.006s sys 0.000s
I20240503 02:14:32.150533 14421 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240503 02:14:32.152325 14406 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20240503 02:14:32.152550 14406 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknrxqHk/test-tmp/tablet_copy_client_session-itest.0.TabletCopyClientSessionITest.TestTabletCopyWithBusySource.1714702442467385-9822-0/minicluster-data/master-0/data,/tmp/dist-test-tasknrxqHk/test-tmp/tablet_copy_client_session-itest.0.TabletCopyClientSessionITest.TestTabletCopyWithBusySource.1714702442467385-9822-0/minicluster-data/master-0/wal
uuid: "e0c1752ef8e54d56a0d646d3bccdaf63"
format_stamp: "Formatted at 2024-05-03 02:14:30 on dist-test-slave-twlh"
I20240503 02:14:32.153496 14406 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknrxqHk/test-tmp/tablet_copy_client_session-itest.0.TabletCopyClientSessionITest.TestTabletCopyWithBusySource.1714702442467385-9822-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasknrxqHk/test-tmp/tablet_copy_client_session-itest.0.TabletCopyClientSessionITest.TestTabletCopyWithBusySource.1714702442467385-9822-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasknrxqHk/test-tmp/tablet_copy_client_session-itest.0.TabletCopyClientSessionITest.TestTabletCopyWithBusySource.1714702442467385-9822-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240503 02:14:32.196931 14406 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240503 02:14:32.198026 14406 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240503 02:14:32.198405 14406 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240503 02:14:32.221185 14472 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.151.190:38767 every 8 connection(s)
I20240503 02:14:32.221266 14406 rpc_server.cc:307] RPC server started. Bound to: 127.9.151.190:38767
I20240503 02:14:32.223620 14406 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknrxqHk/test-tmp/tablet_copy_client_session-itest.0.TabletCopyClientSessionITest.TestTabletCopyWithBusySource.1714702442467385-9822-0/minicluster-data/master-0/data/info.pb
I20240503 02:14:32.227710 9822 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknrxqHk/build/asan/bin/kudu as pid 14406
I20240503 02:14:32.232398 14473 sys_catalog.cc:263] Verifying existing consensus state
I20240503 02:14:32.238332 14473 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e0c1752ef8e54d56a0d646d3bccdaf63: Bootstrap starting.
I20240503 02:14:32.258141 14473 log.cc:824] T 00000000000000000000000000000000 P e0c1752ef8e54d56a0d646d3bccdaf63: Log is configured to *not* fsync() on all Append() calls
I20240503 02:14:32.269634 14473 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e0c1752ef8e54d56a0d646d3bccdaf63: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240503 02:14:32.270438 14473 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e0c1752ef8e54d56a0d646d3bccdaf63: Bootstrap complete.
I20240503 02:14:32.279071 14473 raft_consensus.cc:348] T 00000000000000000000000000000000 P e0c1752ef8e54d56a0d646d3bccdaf63 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e0c1752ef8e54d56a0d646d3bccdaf63" member_type: VOTER last_known_addr { host: "127.9.151.190" port: 38767 } }
I20240503 02:14:32.279950 14473 raft_consensus.cc:729] T 00000000000000000000000000000000 P e0c1752ef8e54d56a0d646d3bccdaf63 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: e0c1752ef8e54d56a0d646d3bccdaf63, State: Initialized, Role: FOLLOWER
I20240503 02:14:32.280652 14473 consensus_queue.cc:260] T 00000000000000000000000000000000 P e0c1752ef8e54d56a0d646d3bccdaf63 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e0c1752ef8e54d56a0d646d3bccdaf63" member_type: VOTER last_known_addr { host: "127.9.151.190" port: 38767 } }
I20240503 02:14:32.281030 14473 raft_consensus.cc:388] T 00000000000000000000000000000000 P e0c1752ef8e54d56a0d646d3bccdaf63 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20240503 02:14:32.281168 14473 raft_consensus.cc:482] T 00000000000000000000000000000000 P e0c1752ef8e54d56a0d646d3bccdaf63 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20240503 02:14:32.281363 14473 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e0c1752ef8e54d56a0d646d3bccdaf63 [term 1 FOLLOWER]: Advancing to term 2
I20240503 02:14:32.285826 14473 raft_consensus.cc:504] T 00000000000000000000000000000000 P e0c1752ef8e54d56a0d646d3bccdaf63 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e0c1752ef8e54d56a0d646d3bccdaf63" member_type: VOTER last_known_addr { host: "127.9.151.190" port: 38767 } }
I20240503 02:14:32.286514 14473 leader_election.cc:302] T 00000000000000000000000000000000 P e0c1752ef8e54d56a0d646d3bccdaf63 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e0c1752ef8e54d56a0d646d3bccdaf63; no voters:
I20240503 02:14:32.287319 14473 leader_election.cc:288] T 00000000000000000000000000000000 P e0c1752ef8e54d56a0d646d3bccdaf63 [CANDIDATE]: Term 2 election: Requested vote from peers
I20240503 02:14:32.287518 14478 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e0c1752ef8e54d56a0d646d3bccdaf63 [term 2 FOLLOWER]: Leader election won for term 2
I20240503 02:14:32.288375 14478 raft_consensus.cc:686] T 00000000000000000000000000000000 P e0c1752ef8e54d56a0d646d3bccdaf63 [term 2 LEADER]: Becoming Leader. State: Replica: e0c1752ef8e54d56a0d646d3bccdaf63, State: Running, Role: LEADER
I20240503 02:14:32.289072 14478 consensus_queue.cc:237] T 00000000000000000000000000000000 P e0c1752ef8e54d56a0d646d3bccdaf63 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e0c1752ef8e54d56a0d646d3bccdaf63" member_type: VOTER last_known_addr { host: "127.9.151.190" port: 38767 } }
I20240503 02:14:32.289800 14473 sys_catalog.cc:564] T 00000000000000000000000000000000 P e0c1752ef8e54d56a0d646d3bccdaf63 [sys.catalog]: configured and running, proceeding with master startup.
I20240503 02:14:32.291796 14479 sys_catalog.cc:455] T 00000000000000000000000000000000 P e0c1752ef8e54d56a0d646d3bccdaf63 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "e0c1752ef8e54d56a0d646d3bccdaf63" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e0c1752ef8e54d56a0d646d3bccdaf63" member_type: VOTER last_known_addr { host: "127.9.151.190" port: 38767 } } }
I20240503 02:14:32.292219 14479 sys_catalog.cc:458] T 00000000000000000000000000000000 P e0c1752ef8e54d56a0d646d3bccdaf63 [sys.catalog]: This master's current role is: LEADER
I20240503 02:14:32.292265 14480 sys_catalog.cc:455] T 00000000000000000000000000000000 P e0c1752ef8e54d56a0d646d3bccdaf63 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e0c1752ef8e54d56a0d646d3bccdaf63. Latest consensus state: current_term: 2 leader_uuid: "e0c1752ef8e54d56a0d646d3bccdaf63" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e0c1752ef8e54d56a0d646d3bccdaf63" member_type: VOTER last_known_addr { host: "127.9.151.190" port: 38767 } } }
I20240503 02:14:32.292516 14480 sys_catalog.cc:458] T 00000000000000000000000000000000 P e0c1752ef8e54d56a0d646d3bccdaf63 [sys.catalog]: This master's current role is: LEADER
I20240503 02:14:32.296869 14484 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20240503 02:14:32.306495 14484 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20240503 02:14:32.308188 14484 catalog_manager.cc:1259] Loaded cluster ID: 0e481206868d4979948cc7ff4ad193ec
I20240503 02:14:32.308297 14484 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20240503 02:14:32.311605 14484 catalog_manager.cc:1504] Loading token signing keys...
I20240503 02:14:32.312991 14484 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e0c1752ef8e54d56a0d646d3bccdaf63: Loaded TSK: 0
I20240503 02:14:32.314651 14484 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20240503 02:14:32.935432 14262 heartbeater.cc:347] Connected to a master server at 127.9.151.190:38767
I20240503 02:14:32.935707 14262 heartbeater.cc:503] Master 127.9.151.190:38767 was elected leader, sending a full tablet report...
I20240503 02:14:32.938177 14438 master_service.cc:432] Got heartbeat from unknown tserver (permanent_uuid: "27f29b79cf9b42f4a5b9838965bbb72d" instance_seqno: 1714702470876754) as {username='slave'} at 127.9.151.129:56591; Asking this server to re-register.
I20240503 02:14:32.939389 14262 heartbeater.cc:464] Registering TS with master...
I20240503 02:14:32.939769 14262 heartbeater.cc:511] Master 127.9.151.190:38767 requested a full tablet report, sending...
I20240503 02:14:32.941684 14438 ts_manager.cc:194] Registered new tserver with Master: 27f29b79cf9b42f4a5b9838965bbb72d (127.9.151.129:33313)
I20240503 02:14:32.945240 9822 external_mini_cluster.cc:930] 1 TS(s) registered with all masters
I20240503 02:14:32.945421 9822 test_util.cc:268] Using random seed: 1204526847
I20240503 02:14:32.969749 14438 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:33944:
name: "test-workload"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
rows: "<redacted>""\004\001\000fff\006\004\001\000\314\314\314\014\004\001\000233\023\004\001\000\230\231\231\031\004\001\000\376\377\377\037\004\001\000dff&\004\001\000\312\314\314,\004\001\0000333\004\001\000\226\231\2319\004\001\000\374\377\377?\004\001\000bffF\004\001\000\310\314\314L\004\001\000.33S\004\001\000\224\231\231Y\004\001\000\372\377\377_\004\001\000`fff\004\001\000\306\314\314l\004\001\000,33s\004\001\000\222\231\231y"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20240503 02:14:33.012643 14194 tablet_service.cc:1467] Processing CreateTablet for tablet f884dde21ea246a792634883aed3cb82 (DEFAULT_TABLE table=test-workload [id=4a2689bb8f05412eb946811a226159b4]), partition=RANGE (key) PARTITION 322122546 <= VALUES < 429496728
I20240503 02:14:33.012627 14187 tablet_service.cc:1467] Processing CreateTablet for tablet cefdd1e872814ddbb522f4013d55b26c (DEFAULT_TABLE table=test-workload [id=4a2689bb8f05412eb946811a226159b4]), partition=RANGE (key) PARTITION 1073741820 <= VALUES < 1181116002
I20240503 02:14:33.013793 14186 tablet_service.cc:1467] Processing CreateTablet for tablet 97c56e26ef4f47f4acf50d0b8d8116dc (DEFAULT_TABLE table=test-workload [id=4a2689bb8f05412eb946811a226159b4]), partition=RANGE (key) PARTITION 1181116002 <= VALUES < 1288490184
I20240503 02:14:33.014614 14191 tablet_service.cc:1467] Processing CreateTablet for tablet 9d03aee673644af6a0b56ffdc5fb1e99 (DEFAULT_TABLE table=test-workload [id=4a2689bb8f05412eb946811a226159b4]), partition=RANGE (key) PARTITION 644245092 <= VALUES < 751619274
I20240503 02:14:33.012692 14195 tablet_service.cc:1467] Processing CreateTablet for tablet 92cd1f3c9cdc43e0b9d56e3375488f50 (DEFAULT_TABLE table=test-workload [id=4a2689bb8f05412eb946811a226159b4]), partition=RANGE (key) PARTITION 214748364 <= VALUES < 322122546
I20240503 02:14:33.015218 14178 tablet_service.cc:1467] Processing CreateTablet for tablet ac0fc766d4b04fb1a7c996c57d383349 (DEFAULT_TABLE table=test-workload [id=4a2689bb8f05412eb946811a226159b4]), partition=RANGE (key) PARTITION 2040109458 <= VALUES
I20240503 02:14:33.016604 14197 tablet_service.cc:1467] Processing CreateTablet for tablet ef909312fe9545b18bf9f918f52928c2 (DEFAULT_TABLE table=test-workload [id=4a2689bb8f05412eb946811a226159b4]), partition=RANGE (key) PARTITION VALUES < 107374182
I20240503 02:14:33.016718 14190 tablet_service.cc:1467] Processing CreateTablet for tablet 02b52f763d8a412788ceabba8278d71f (DEFAULT_TABLE table=test-workload [id=4a2689bb8f05412eb946811a226159b4]), partition=RANGE (key) PARTITION 751619274 <= VALUES < 858993456
I20240503 02:14:33.016799 14189 tablet_service.cc:1467] Processing CreateTablet for tablet 24e81fa4c7b249aca961df89f56a8557 (DEFAULT_TABLE table=test-workload [id=4a2689bb8f05412eb946811a226159b4]), partition=RANGE (key) PARTITION 858993456 <= VALUES < 966367638
I20240503 02:14:33.016853 14188 tablet_service.cc:1467] Processing CreateTablet for tablet 4ff002252c4d46d2af2a4956acd90b3f (DEFAULT_TABLE table=test-workload [id=4a2689bb8f05412eb946811a226159b4]), partition=RANGE (key) PARTITION 966367638 <= VALUES < 1073741820
I20240503 02:14:33.018115 14194 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f884dde21ea246a792634883aed3cb82. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:14:33.017450 14192 tablet_service.cc:1467] Processing CreateTablet for tablet fe8b26bb37ab4b948fd500b648bf6b5a (DEFAULT_TABLE table=test-workload [id=4a2689bb8f05412eb946811a226159b4]), partition=RANGE (key) PARTITION 536870910 <= VALUES < 644245092
I20240503 02:14:33.017793 14182 tablet_service.cc:1467] Processing CreateTablet for tablet 6069a864c5ff43348881ff477218578d (DEFAULT_TABLE table=test-workload [id=4a2689bb8f05412eb946811a226159b4]), partition=RANGE (key) PARTITION 1610612730 <= VALUES < 1717986912
I20240503 02:14:33.017529 14193 tablet_service.cc:1467] Processing CreateTablet for tablet 9d424d86fde04f2dbdd9434ee5755150 (DEFAULT_TABLE table=test-workload [id=4a2689bb8f05412eb946811a226159b4]), partition=RANGE (key) PARTITION 429496728 <= VALUES < 536870910
I20240503 02:14:33.017968 14185 tablet_service.cc:1467] Processing CreateTablet for tablet b0df8ae119f948eaa5461faa42007713 (DEFAULT_TABLE table=test-workload [id=4a2689bb8f05412eb946811a226159b4]), partition=RANGE (key) PARTITION 1288490184 <= VALUES < 1395864366
I20240503 02:14:33.019721 14180 tablet_service.cc:1467] Processing CreateTablet for tablet fc8f5043623a48418283e5ecd50d58e2 (DEFAULT_TABLE table=test-workload [id=4a2689bb8f05412eb946811a226159b4]), partition=RANGE (key) PARTITION 1825361094 <= VALUES < 1932735276
I20240503 02:14:33.018210 14184 tablet_service.cc:1467] Processing CreateTablet for tablet be3ab5a5dd774bdba45e6d7061019195 (DEFAULT_TABLE table=test-workload [id=4a2689bb8f05412eb946811a226159b4]), partition=RANGE (key) PARTITION 1395864366 <= VALUES < 1503238548
I20240503 02:14:33.018174 14196 tablet_service.cc:1467] Processing CreateTablet for tablet b26fbd5ae3384b6cb444331bd8e75aef (DEFAULT_TABLE table=test-workload [id=4a2689bb8f05412eb946811a226159b4]), partition=RANGE (key) PARTITION 107374182 <= VALUES < 214748364
I20240503 02:14:33.020578 14193 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9d424d86fde04f2dbdd9434ee5755150. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:14:33.020853 14178 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ac0fc766d4b04fb1a7c996c57d383349. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:14:33.021109 14187 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cefdd1e872814ddbb522f4013d55b26c. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:14:33.018414 14179 tablet_service.cc:1467] Processing CreateTablet for tablet fa681a37b30c4a0b879d3f3b184405fc (DEFAULT_TABLE table=test-workload [id=4a2689bb8f05412eb946811a226159b4]), partition=RANGE (key) PARTITION 1932735276 <= VALUES < 2040109458
I20240503 02:14:33.021365 14182 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6069a864c5ff43348881ff477218578d. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:14:33.021620 14184 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet be3ab5a5dd774bdba45e6d7061019195. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:14:33.017915 14183 tablet_service.cc:1467] Processing CreateTablet for tablet a40f48ddab6e48d5ba3707ef78923ee5 (DEFAULT_TABLE table=test-workload [id=4a2689bb8f05412eb946811a226159b4]), partition=RANGE (key) PARTITION 1503238548 <= VALUES < 1610612730
I20240503 02:14:33.021835 14192 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fe8b26bb37ab4b948fd500b648bf6b5a. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:14:33.022042 14196 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b26fbd5ae3384b6cb444331bd8e75aef. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:14:33.019806 14181 tablet_service.cc:1467] Processing CreateTablet for tablet e50f84673efd4108859e8c1360cad36f (DEFAULT_TABLE table=test-workload [id=4a2689bb8f05412eb946811a226159b4]), partition=RANGE (key) PARTITION 1717986912 <= VALUES < 1825361094
I20240503 02:14:33.022295 14179 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fa681a37b30c4a0b879d3f3b184405fc. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:14:33.022605 14181 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e50f84673efd4108859e8c1360cad36f. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:14:33.022924 14183 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a40f48ddab6e48d5ba3707ef78923ee5. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:14:33.023218 14189 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 24e81fa4c7b249aca961df89f56a8557. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:14:33.023738 14190 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 02b52f763d8a412788ceabba8278d71f. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:14:33.024127 14188 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4ff002252c4d46d2af2a4956acd90b3f. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:14:33.025524 14186 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 97c56e26ef4f47f4acf50d0b8d8116dc. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:14:33.027274 14191 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9d03aee673644af6a0b56ffdc5fb1e99. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:14:33.027916 14180 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fc8f5043623a48418283e5ecd50d58e2. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:14:33.028550 14185 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b0df8ae119f948eaa5461faa42007713. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:14:33.028894 14195 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 92cd1f3c9cdc43e0b9d56e3375488f50. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:14:33.034247 14197 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ef909312fe9545b18bf9f918f52928c2. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:14:33.143464 14508 tablet_bootstrap.cc:492] T 9d03aee673644af6a0b56ffdc5fb1e99 P 27f29b79cf9b42f4a5b9838965bbb72d: Bootstrap starting.
I20240503 02:14:33.147207 14508 tablet_bootstrap.cc:654] T 9d03aee673644af6a0b56ffdc5fb1e99 P 27f29b79cf9b42f4a5b9838965bbb72d: Neither blocks nor log segments found. Creating new log.
I20240503 02:14:33.148377 14508 log.cc:824] T 9d03aee673644af6a0b56ffdc5fb1e99 P 27f29b79cf9b42f4a5b9838965bbb72d: Log is configured to *not* fsync() on all Append() calls
I20240503 02:14:33.151880 14508 tablet_bootstrap.cc:492] T 9d03aee673644af6a0b56ffdc5fb1e99 P 27f29b79cf9b42f4a5b9838965bbb72d: No bootstrap required, opened a new log
I20240503 02:14:33.152189 14508 ts_tablet_manager.cc:1367] T 9d03aee673644af6a0b56ffdc5fb1e99 P 27f29b79cf9b42f4a5b9838965bbb72d: Time spent bootstrapping tablet: real 0.009s user 0.007s sys 0.000s
I20240503 02:14:33.158962 14508 raft_consensus.cc:348] T 9d03aee673644af6a0b56ffdc5fb1e99 P 27f29b79cf9b42f4a5b9838965bbb72d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27f29b79cf9b42f4a5b9838965bbb72d" member_type: VOTER last_known_addr { host: "127.9.151.129" port: 33313 } }
I20240503 02:14:33.159399 14508 raft_consensus.cc:374] T 9d03aee673644af6a0b56ffdc5fb1e99 P 27f29b79cf9b42f4a5b9838965bbb72d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:14:33.159493 14508 raft_consensus.cc:729] T 9d03aee673644af6a0b56ffdc5fb1e99 P 27f29b79cf9b42f4a5b9838965bbb72d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 27f29b79cf9b42f4a5b9838965bbb72d, State: Initialized, Role: FOLLOWER
I20240503 02:14:33.160281 14508 consensus_queue.cc:260] T 9d03aee673644af6a0b56ffdc5fb1e99 P 27f29b79cf9b42f4a5b9838965bbb72d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27f29b79cf9b42f4a5b9838965bbb72d" member_type: VOTER last_known_addr { host: "127.9.151.129" port: 33313 } }
I20240503 02:14:33.160588 14508 raft_consensus.cc:388] T 9d03aee673644af6a0b56ffdc5fb1e99 P 27f29b79cf9b42f4a5b9838965bbb72d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20240503 02:14:33.160759 14508 raft_consensus.cc:482] T 9d03aee673644af6a0b56ffdc5fb1e99 P 27f29b79cf9b42f4a5b9838965bbb72d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20240503 02:14:33.160953 14508 raft_consensus.cc:3037] T 9d03aee673644af6a0b56ffdc5fb1e99 P 27f29b79cf9b42f4a5b9838965bbb72d [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:14:33.163345 14508 raft_consensus.cc:504] T 9d03aee673644af6a0b56ffdc5fb1e99 P 27f29b79cf9b42f4a5b9838965bbb72d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27f29b79cf9b42f4a5b9838965bbb72d" member_type: VOTER last_known_addr { host: "127.9.151.129" port: 33313 } }
I20240503 02:14:33.163924 14508 leader_election.cc:302] T 9d03aee673644af6a0b56ffdc5fb1e99 P 27f29b79cf9b42f4a5b9838965bbb72d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 27f29b79cf9b42f4a5b9838965bbb72d; no voters:
I20240503 02:14:33.164727 14508 leader_election.cc:288] T 9d03aee673644af6a0b56ffdc5fb1e99 P 27f29b79cf9b42f4a5b9838965bbb72d [CANDIDATE]: Term 1 election: Requested vote from peers
I20240503 02:14:33.164937 14510 raft_consensus.cc:2781] T 9d03aee673644af6a0b56ffdc5fb1e99 P 27f29b79cf9b42f4a5b9838965bbb72d [term 1 FOLLOWER]: Leader election won for term 1
I20240503 02:14:33.166131 14510 raft_consensus.cc:686] T 9d03aee673644af6a0b56ffdc5fb1e99 P 27f29b79cf9b42f4a5b9838965bbb72d [term 1 LEADER]: Becoming Leader. State: Replica: 27f29b79cf9b42f4a5b9838965bbb72d, State: Running, Role: LEADER
I20240503 02:14:33.166473 14508 ts_tablet_manager.cc:1398] T 9d03aee673644af6a0b56ffdc5fb1e99 P 27f29b79cf9b42f4a5b9838965bbb72d: Time spent starting tablet: real 0.014s user 0.015s sys 0.000s
I20240503 02:14:33.166771 14510 consensus_queue.cc:237] T 9d03aee673644af6a0b56ffdc5fb1e99 P 27f29b79cf9b42f4a5b9838965bbb72d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27f29b79cf9b42f4a5b9838965bbb72d" member_type: VOTER last_known_addr { host: "127.9.151.129" port: 33313 } }
I20240503 02:14:33.166941 14508 tablet_bootstrap.cc:492] T 02b52f763d8a412788ceabba8278d71f P 27f29b79cf9b42f4a5b9838965bbb72d: Bootstrap starting.
I20240503 02:14:33.168550 14508 tablet_bootstrap.cc:654] T 02b52f763d8a412788ceabba8278d71f P 27f29b79cf9b42f4a5b9838965bbb72d: Neither blocks nor log segments found. Creating new log.
I20240503 02:14:33.170996 14508 tablet_bootstrap.cc:492] T 02b52f763d8a412788ceabba8278d71f P 27f29b79cf9b42f4a5b9838965bbb72d: No bootstrap required, opened a new log
I20240503 02:14:33.171206 14508 ts_tablet_manager.cc:1367] T 02b52f763d8a412788ceabba8278d71f P 27f29b79cf9b42f4a5b9838965bbb72d: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20240503 02:14:33.172120 14508 raft_consensus.cc:348] T 02b52f763d8a412788ceabba8278d71f P 27f29b79cf9b42f4a5b9838965bbb72d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27f29b79cf9b42f4a5b9838965bbb72d" member_type: VOTER last_known_addr { host: "127.9.151.129" port: 33313 } }
I20240503 02:14:33.172302 14508 raft_consensus.cc:374] T 02b52f763d8a412788ceabba8278d71f P 27f29b79cf9b42f4a5b9838965bbb72d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:14:33.172386 14508 raft_consensus.cc:729] T 02b52f763d8a412788ceabba8278d71f P 27f29b79cf9b42f4a5b9838965bbb72d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 27f29b79cf9b42f4a5b9838965bbb72d, State: Initialized, Role: FOLLOWER
I20240503 02:14:33.172688 14508 consensus_queue.cc:260] T 02b52f763d8a412788ceabba8278d71f P 27f29b79cf9b42f4a5b9838965bbb72d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27f29b79cf9b42f4a5b9838965bbb72d" member_type: VOTER last_known_addr { host: "127.9.151.129" port: 33313 } }
I20240503 02:14:33.172860 14508 raft_consensus.cc:388] T 02b52f763d8a412788ceabba8278d71f P 27f29b79cf9b42f4a5b9838965bbb72d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20240503 02:14:33.172938 14508 raft_consensus.cc:482] T 02b52f763d8a412788ceabba8278d71f P 27f29b79cf9b42f4a5b9838965bbb72d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20240503 02:14:33.173055 14508 raft_consensus.cc:3037] T 02b52f763d8a412788ceabba8278d71f P 27f29b79cf9b42f4a5b9838965bbb72d [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:14:33.173182 14438 catalog_manager.cc:5484] T 9d03aee673644af6a0b56ffdc5fb1e99 P 27f29b79cf9b42f4a5b9838965bbb72d reported cstate change: term changed from 0 to 1, leader changed from <none> to 27f29b79cf9b42f4a5b9838965bbb72d (127.9.151.129). New cstate: current_term: 1 leader_uuid: "27f29b79cf9b42f4a5b9838965bbb72d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27f29b79cf9b42f4a5b9838965bbb72d" member_type: VOTER last_known_addr { host: "127.9.151.129" port: 33313 } health_report { overall_health: HEALTHY } } }
I20240503 02:14:33.175069 14508 raft_consensus.cc:504] T 02b52f763d8a412788ceabba8278d71f P 27f29b79cf9b42f4a5b9838965bbb72d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27f29b79cf9b42f4a5b9838965bbb72d" member_type: VOTER last_known_addr { host: "127.9.151.129" port: 33313 } }
I20240503 02:14:33.175324 14508 leader_election.cc:302] T 02b52f763d8a412788ceabba8278d71f P 27f29b79cf9b42f4a5b9838965bbb72d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 27f29b79cf9b42f4a5b9838965bbb72d; no voters:
I20240503 02:14:33.175514 14508 leader_election.cc:288] T 02b52f763d8a412788ceabba8278d71f P 27f29b79cf9b42f4a5b9838965bbb72d [CANDIDATE]: Term 1 election: Requested vote from peers
I20240503 02:14:33.175591 14510 raft_consensus.cc:2781] T 02b52f763d8a412788ceabba8278d71f P 27f29b79cf9b42f4a5b9838965bbb72d [term 1 FOLLOWER]: Leader election won for term 1
I20240503 02:14:33.175750 14510 raft_consensus.cc:686] T 02b52f763d8a412788ceabba8278d71f P 27f29b79cf9b42f4a5b9838965bbb72d [term 1 LEADER]: Becoming Leader. State: Replica: 27f29b79cf9b42f4a5b9838965bbb72d, State: Running, Role: LEADER
I20240503 02:14:33.176024 14508 ts_tablet_manager.cc:1398] T 02b52f763d8a412788ceabba8278d71f P 27f29b79cf9b42f4a5b9838965bbb72d: Time spent starting tablet: real 0.005s user 0.004s sys 0.000s
I20240503 02:14:33.175985 14510 consensus_queue.cc:237] T 02b52f763d8a412788ceabba8278d71f P 27f29b79cf9b42f4a5b9838965bbb72d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27f29b79cf9b42f4a5b9838965bbb72d" member_type: VOTER last_known_addr { host: "127.9.151.129" port: 33313 } }
I20240503 02:14:33.176302 14508 tablet_bootstrap.cc:492] T 6069a864c5ff43348881ff477218578d P 27f29b79cf9b42f4a5b9838965bbb72d: Bootstrap starting.
I20240503 02:14:33.178144 14508 tablet_bootstrap.cc:654] T 6069a864c5ff43348881ff477218578d P 27f29b79cf9b42f4a5b9838965bbb72d: Neither blocks nor log segments found. Creating new log.
I20240503 02:14:33.180106 14508 tablet_bootstrap.cc:492] T 6069a864c5ff43348881ff477218578d P 27f29b79cf9b42f4a5b9838965bbb72d: No bootstrap required, opened a new log
I20240503 02:14:33.180267 14508 ts_tablet_manager.cc:1367] T 6069a864c5ff43348881ff477218578d P 27f29b79cf9b42f4a5b9838965bbb72d: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.003s
I20240503 02:14:33.179876 14438 catalog_manager.cc:5484] T 02b52f763d8a412788ceabba8278d71f P 27f29b79cf9b42f4a5b9838965bbb72d reported cstate change: term changed from 0 to 1, leader changed from <none> to 27f29b79cf9b42f4a5b9838965bbb72d (127.9.151.129). New cstate: current_term: 1 leader_uuid: "27f29b79cf9b42f4a5b9838965bbb72d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27f29b79cf9b42f4a5b9838965bbb72d" member_type: VOTER last_known_addr { host: "127.9.151.129" port: 33313 } health_report { overall_health: HEALTHY } } }
I20240503 02:14:33.180858 14508 raft_consensus.cc:348] T 6069a864c5ff43348881ff477218578d P 27f29b79cf9b42f4a5b9838965bbb72d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27f29b79cf9b42f4a5b9838965bbb72d" member_type: VOTER last_known_addr { host: "127.9.151.129" port: 33313 } }
I20240503 02:14:33.181018 14508 raft_consensus.cc:374] T 6069a864c5ff43348881ff477218578d P 27f29b79cf9b42f4a5b9838965bbb72d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:14:33.181066 14508 raft_consensus.cc:729] T 6069a864c5ff43348881ff477218578d P 27f29b79cf9b42f4a5b9838965bbb72d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 27f29b79cf9b42f4a5b9838965bbb72d, State: Initialized, Role: FOLLOWER
I20240503 02:14:33.181317 14508 consensus_queue.cc:260] T 6069a864c5ff43348881ff477218578d P 27f29b79cf9b42f4a5b9838965bbb72d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27f29b79cf9b42f4a5b9838965bbb72d" member_type: VOTER last_known_addr { host: "127.9.151.129" port: 33313 } }
I20240503 02:14:33.181471 14508 raft_consensus.cc:388] T 6069a864c5ff43348881ff477218578d P 27f29b79cf9b42f4a5b9838965bbb72d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20240503 02:14:33.181558 14508 raft_consensus.cc:482] T 6069a864c5ff43348881ff477218578d P 27f29b79cf9b42f4a5b9838965bbb72d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20240503 02:14:33.181646 14508 raft_consensus.cc:3037] T 6069a864c5ff43348881ff477218578d P 27f29b79cf9b42f4a5b9838965bbb72d [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:14:33.183895 14508 raft_consensus.cc:504] T 6069a864c5ff43348881ff477218578d P 27f29b79cf9b42f4a5b9838965bbb72d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27f29b79cf9b42f4a5b9838965bbb72d" member_type: VOTER last_known_addr { host: "127.9.151.129" port: 33313 } }
I20240503 02:14:33.184149 14508 leader_election.cc:302] T 6069a864c5ff43348881ff477218578d P 27f29b79cf9b42f4a5b9838965bbb72d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 27f29b79cf9b42f4a5b9838965bbb72d; no voters:
I20240503 02:14:33.184398 14508 leader_election.cc:288] T 6069a864c5ff43348881ff477218578d P 27f29b79cf9b42f4a5b9838965bbb72d [CANDIDATE]: Term 1 election: Requested vote from peers
I20240503 02:14:33.184552 14510 raft_consensus.cc:2781] T 6069a864c5ff43348881ff477218578d P 27f29b79cf9b42f4a5b9838965bbb72d [term 1 FOLLOWER]: Leader election won for term 1
I20240503 02:14:33.184758 14510 raft_consensus.cc:686] T 6069a864c5ff43348881ff477218578d P 27f29b79cf9b42f4a5b9838965bbb72d [term 1 LEADER]: Becoming Leader. State: Replica: 27f29b79cf9b42f4a5b9838965bbb72d, State: Running, Role: LEADER
I20240503 02:14:33.184974 14508 ts_tablet_manager.cc:1398] T 6069a864c5ff43348881ff477218578d P 27f29b79cf9b42f4a5b9838965bbb72d: Time spent starting tablet: real 0.005s user 0.004s sys 0.001s
I20240503 02:14:33.185250 14508 tablet_bootstrap.cc:492] T 9d424d86fde04f2dbdd9434ee5755150 P 27f29b79cf9b42f4a5b9838965bbb72d: Bootstrap starting.
I20240503 02:14:33.185165 14510 consensus_queue.cc:237] T 6069a864c5ff43348881ff477218578d P 27f29b79cf9b42f4a5b9838965bbb72d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27f29b79cf9b42f4a5b9838965bbb72d" member_type: VOTER last_known_addr { host: "127.9.151.129" port: 33313 } }
I20240503 02:14:33.187016 14508 tablet_bootstrap.cc:654] T 9d424d86fde04f2dbdd9434ee5755150 P 27f29b79cf9b42f4a5b9838965bbb72d: Neither blocks nor log segments found. Creating new log.
I20240503 02:14:33.188270 14438 catalog_manager.cc:5484] T 6069a864c5ff43348881ff477218578d P 27f29b79cf9b42f4a5b9838965bbb72d reported cstate change: term changed from 0 to 1, leader changed from <none> to 27f29b79cf9b42f4a5b9838965bbb72d (127.9.151.129). New cstate: current_term: 1 leader_uuid: "27f29b79cf9b42f4a5b9838965bbb72d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27f29b79cf9b42f4a5b9838965bbb72d" member_type: VOTER last_known_addr { host: "127.9.151.129" port: 33313 } health_report { overall_health: HEALTHY } } }
I20240503 02:14:33.188874 14508 tablet_bootstrap.cc:492] T 9d424d86fde04f2dbdd9434ee5755150 P 27f29b79cf9b42f4a5b9838965bbb72d: No bootstrap required, opened a new log
I20240503 02:14:33.189003 14508 ts_tablet_manager.cc:1367] T 9d424d86fde04f2dbdd9434ee5755150 P 27f29b79cf9b42f4a5b9838965bbb72d: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20240503 02:14:33.189704 14508 raft_consensus.cc:348] T 9d424d86fde04f2dbdd9434ee5755150 P 27f29b79cf9b42f4a5b9838965bbb72d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27f29b79cf9b42f4a5b9838965bbb72d" member_type: VOTER last_known_addr { host: "127.9.151.129" port: 33313 } }
I20240503 02:14:33.189893 14508 raft_consensus.cc:374] T 9d424d86fde04f2dbdd9434ee5755150 P 27f29b79cf9b42f4a5b9838965bbb72d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:14:33.190006 14508 raft_consensus.cc:729] T 9d424d86fde04f2dbdd9434ee5755150 P 27f29b79cf9b42f4a5b9838965bbb72d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 27f29b79cf9b42f4a5b9838965bbb72d, State: Initialized, Role: FOLLOWER
I20240503 02:14:33.190259 14508 consensus_queue.cc:260] T 9d424d86fde04f2dbdd9434ee5755150 P 27f29b79cf9b42f4a5b9838965bbb72d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27f29b79cf9b42f4a5b9838965bbb72d" member_type: VOTER last_known_addr { host: "127.9.151.129" port: 33313 } }
I20240503 02:14:33.190462 14508 raft_consensus.cc:388] T 9d424d86fde04f2dbdd9434ee5755150 P 27f29b79cf9b42f4a5b9838965bbb72d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20240503 02:14:33.190536 14508 raft_consensus.cc:482] T 9d424d86fde04f2dbdd9434ee5755150 P 27f29b79cf9b42f4a5b9838965bbb72d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20240503 02:14:33.190647 14508 raft_consensus.cc:3037] T 9d424d86fde04f2dbdd9434ee5755150 P 27f29b79cf9b42f4a5b9838965bbb72d [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:14:33.193106 14508 raft_consensus.cc:504] T 9d424d86fde04f2dbdd9434ee5755150 P 27f29b79cf9b42f4a5b9838965bbb72d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27f29b79cf9b42f4a5b9838965bbb72d" member_type: VOTER last_known_addr { host: "127.9.151.129" port: 33313 } }
I20240503 02:14:33.193437 14508 leader_election.cc:302] T 9d424d86fde04f2dbdd9434ee5755150 P 27f29b79cf9b42f4a5b9838965bbb72d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 27f29b79cf9b42f4a5b9838965bbb72d; no voters:
I20240503 02:14:33.193660 14508 leader_election.cc:288] T 9d424d86fde04f2dbdd9434ee5755150 P 27f29b79cf9b42f4a5b9838965bbb72d [CANDIDATE]: Term 1 election: Requested vote from peers
I20240503 02:14:33.193801 14510 raft_consensus.cc:2781] T 9d424d86fde04f2dbdd9434ee5755150 P 27f29b79cf9b42f4a5b9838965bbb72d [term 1 FOLLOWER]: Leader election won for term 1
I20240503 02:14:33.193994 14510 raft_consensus.cc:686] T 9d424d86fde04f2dbdd9434ee5755150 P 27f29b79cf9b42f4a5b9838965bbb72d [term 1 LEADER]: Becoming Leader. State: Replica: 27f29b79cf9b42f4a5b9838965bbb72d, State: Running, Role: LEADER
I20240503 02:14:33.194168 14508 ts_tablet_manager.cc:1398] T 9d424d86fde04f2dbdd9434ee5755150 P 27f29b79cf9b42f4a5b9838965bbb72d: Time spent starting tablet: real 0.005s user 0.002s sys 0.000s
I20240503 02:14:33.194265 14510 consensus_queue.cc:237] T 9d424d86fde04f2dbdd9434ee5755150 P 27f29b79cf9b42f4a5b9838965bbb72d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27f29b79cf9b42f4a5b9838965bbb72d" member_type: VOTER last_known_addr { host: "127.9.151.129" port: 33313 } }
I20240503 02:14:33.194456 14508 tablet_bootstrap.cc:492] T a40f48ddab6e48d5ba3707ef78923ee5 P 27f29b79cf9b42f4a5b9838965bbb72d: Bootstrap starting.
I20240503 02:14:33.196254 14508 tablet_bootstrap.cc:654] T a40f48ddab6e48d5ba3707ef78923ee5 P 27f29b79cf9b42f4a5b9838965bbb72d: Neither blocks nor log segments found. Creating new log.
I20240503 02:14:33.196934 14438 catalog_manager.cc:5484] T 9d424d86fde04f2dbdd9434ee5755150 P 27f29b79cf9b42f4a5b9838965bbb72d reported cstate change: term changed from 0 to 1, leader changed from <none> to 27f29b79cf9b42f4a5b9838965bbb72d (127.9.151.129). New cstate: current_term: 1 leader_uuid: "27f29b79cf9b42f4a5b9838965bbb72d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27f29b79cf9b42f4a5b9838965bbb72d" member_type: VOTER last_known_addr { host: "127.9.151.129" port: 33313 } health_report { overall_health: HEALTHY } } }
I20240503 02:14:33.198383 14508 tablet_bootstrap.cc:492] T a40f48ddab6e48d5ba3707ef78923ee5 P 27f29b79cf9b42f4a5b9838965bbb72d: No bootstrap required, opened a new log
I20240503 02:14:33.198554 14508 ts_tablet_manager.cc:1367] T a40f48ddab6e48d5ba3707ef78923ee5 P 27f29b79cf9b42f4a5b9838965bbb72d: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20240503 02:14:33.199378 14508 raft_consensus.cc:348] T a40f48ddab6e48d5ba3707ef78923ee5 P 27f29b79cf9b42f4a5b9838965bbb72d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27f29b79cf9b42f4a5b9838965bbb72d" member_type: VOTER last_known_addr { host: "127.9.151.129" port: 33313 } }
I20240503 02:14:33.199548 14508 raft_consensus.cc:374] T a40f48ddab6e48d5ba3707ef78923ee5 P 27f29b79cf9b42f4a5b9838965bbb72d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:14:33.199630 14508 raft_consensus.cc:729] T a40f48ddab6e48d5ba3707ef78923ee5 P 27f29b79cf9b42f4a5b9838965bbb72d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 27f29b79cf9b42f4a5b9838965bbb72d, State: Initialized, Role: FOLLOWER
I20240503 02:14:33.199893 14508 consensus_queue.cc:260] T a40f48ddab6e48d5ba3707ef78923ee5 P 27f29b79cf9b42f4a5b9838965bbb72d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27f29b79cf9b42f4a5b9838965bbb72d" member_type: VOTER last_known_addr { host: "127.9.151.129" port: 33313 } }
I20240503 02:14:33.200053 14508 raft_consensus.cc:388] T a40f48ddab6e48d5ba3707ef78923ee5 P 27f29b79cf9b42f4a5b9838965bbb72d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20240503 02:14:33.200147 14508 raft_consensus.cc:482] T a40f48ddab6e48d5ba3707ef78923ee5 P 27f29b79cf9b42f4a5b9838965bbb72d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20240503 02:14:33.200263 14508 raft_consensus.cc:3037] T a40f48ddab6e48d5ba3707ef78923ee5 P 27f29b79cf9b42f4a5b9838965bbb72d [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:14:33.202634 14508 raft_consensus.cc:504] T a40f48ddab6e48d5ba3707ef78923ee5 P 27f29b79cf9b42f4a5b9838965bbb72d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27f29b79cf9b42f4a5b9838965bbb72d" member_type: VOTER last_known_addr { host: "127.9.151.129" port: 33313 } }
I20240503 02:14:33.202916 14508 leader_election.cc:302] T a40f48ddab6e48d5ba3707ef78923ee5 P 27f29b79cf9b42f4a5b9838965bbb72d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 27f29b79cf9b42f4a5b9838965bbb72d; no voters:
I20240503 02:14:33.203161 14508 leader_election.cc:288] T a40f48ddab6e48d5ba3707ef78923ee5 P 27f29b79cf9b42f4a5b9838965bbb72d [CANDIDATE]: Term 1 election: Requested vote from peers
I20240503 02:14:33.203383 14510 raft_consensus.cc:2781] T a40f48ddab6e48d5ba3707ef78923ee5 P 27f29b79cf9b42f4a5b9838965bbb72d [term 1 FOLLOWER]: Leader election won for term 1
I20240503 02:14:33.203605 14510 raft_consensus.cc:686] T a40f48ddab6e48d5ba3707ef78923ee5 P 27f29b79cf9b42f4a5b9838965bbb72d [term 1 LEADER]: Becoming Leader. State: Replica: 27f29b79cf9b42f4a5b9838965bbb72d, State: Running, Role: LEADER
I20240503 02:14:33.203662 14508 ts_tablet_manager.cc:1398] T a40f48ddab6e48d5ba3707ef78923ee5 P 27f29b79cf9b42f4a5b9838965bbb72d: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20240503 02:14:33.203990 14508 tablet_bootstrap.cc:492] T 24e81fa4c7b249aca961df89f56a8557 P 27f29b79cf9b42f4a5b9838965bbb72d: Bootstrap starting.
I20240503 02:14:33.203912 14510 consensus_queue.cc:237] T a40f48ddab6e48d5ba3707ef78923ee5 P 27f29b79cf9b42f4a5b9838965bbb72d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27f29b79cf9b42f4a5b9838965bbb72d" member_type: VOTER last_known_addr { host: "127.9.151.129" port: 33313 } }
I20240503 02:14:33.206133 14508 tablet_bootstrap.cc:654] T 24e81fa4c7b249aca961df89f56a8557 P 27f29b79cf9b42f4a5b9838965bbb72d: Neither blocks nor log segments found. Creating new log.
I20240503 02:14:33.207340 14438 catalog_manager.cc:5484] T a40f48ddab6e48d5ba3707ef78923ee5 P 27f29b79cf9b42f4a5b9838965bbb72d reported cstate change: term changed from 0 to 1, leader changed from <none> to 27f29b79cf9b42f4a5b9838965bbb72d (127.9.151.129). New cstate: current_term: 1 leader_uuid: "27f29b79cf9b42f4a5b9838965bbb72d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27f29b79cf9b42f4a5b9838965bbb72d" member_type: VOTER last_known_addr { host: "127.9.151.129" port: 33313 } health_report { overall_health: HEALTHY } } }
I20240503 02:14:33.208233 14508 tablet_bootstrap.cc:492] T 24e81fa4c7b249aca961df89f56a8557 P 27f29b79cf9b42f4a5b9838965bbb72d: No bootstrap required, opened a new log
I20240503 02:14:33.208427 14508 ts_tablet_manager.cc:1367] T 24e81fa4c7b249aca961df89f56a8557 P 27f29b79cf9b42f4a5b9838965bbb72d: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.002s
I20240503 02:14:33.209228 14508 raft_consensus.cc:348] T 24e81fa4c7b249aca961df89f56a8557 P 27f29b79cf9b42f4a5b9838965bbb72d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27f29b79cf9b42f4a5b9838965bbb72d" member_type: VOTER last_known_addr { host: "127.9.151.129" port: 33313 } }
I20240503 02:14:33.209383 14508 raft_consensus.cc:374] T 24e81fa4c7b249aca961df89f56a8557 P 27f29b79cf9b42f4a5b9838965bbb72d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:14:33.209443 14508 raft_consensus.cc:729] T 24e81fa4c7b249aca961df89f56a8557 P 27f29b79cf9b42f4a5b9838965bbb72d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 27f29b79cf9b42f4a5b9838965bbb72d, State: Initialized, Role: FOLLOWER
I20240503 02:14:33.209677 14508 consensus_queue.cc:260] T 24e81fa4c7b249aca961df89f56a8557 P 27f29b79cf9b42f4a5b9838965bbb72d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27f29b79cf9b42f4a5b9838965bbb72d" member_type: VOTER last_known_addr { host: "127.9.151.129" port: 33313 } }
I20240503 02:14:33.209833 14508 raft_consensus.cc:388] T 24e81fa4c7b249aca961df89f56a8557 P 27f29b79cf9b42f4a5b9838965bbb72d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20240503 02:14:33.209892 14508 raft_consensus.cc:482] T 24e81fa4c7b249aca961df89f56a8557 P 27f29b79cf9b42f4a5b9838965bbb72d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20240503 02:14:33.209973 14508 raft_consensus.cc:3037] T 24e81fa4c7b249aca961df89f56a8557 P 27f29b79cf9b42f4a5b9838965bbb72d [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:14:33.212404 14508 raft_consensus.cc:504] T 24e81fa4c7b249aca961df89f56a8557 P 27f29b79cf9b42f4a5b9838965bbb72d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27f29b79cf9b42f4a5b9838965bbb72d" member_type: VOTER last_known_addr { host: "127.9.151.129" port: 33313 } }
I20240503 02:14:33.212677 14508 leader_election.cc:302] T 24e81fa4c7b249aca961df89f56a8557 P 27f29b79cf9b42f4a5b9838965bbb72d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 27f29b79cf9b42f4a5b9838965bbb72d; no voters:
I20240503 02:14:33.212908 14508 leader_election.cc:288] T 24e81fa4c7b249aca961df89f56a8557 P 27f29b79cf9b42f4a5b9838965bbb72d [CANDIDATE]: Term 1 election: Requested vote from peers
I20240503 02:14:33.213042 14510 raft_consensus.cc:2781] T 24e81fa4c7b249aca961df89f56a8557 P 27f29b79cf9b42f4a5b9838965bbb72d [term 1 FOLLOWER]: Leader election won for term 1
I20240503 02:14:33.213233 14510 raft_consensus.cc:686] T 24e81fa4c7b249aca961df89f56a8557 P 27f29b79cf9b42f4a5b9838965bbb72d [term 1 LEADER]: Becoming Leader. State: Replica: 27f29b79cf9b42f4a5b9838965bbb72d, State: Running, Role: LEADER
I20240503 02:14:33.213380 14508 ts_tablet_manager.cc:1398] T 24e81fa4c7b249aca961df89f56a8557 P 27f29b79cf9b42f4a5b9838965bbb72d: Time spent starting tablet: real 0.005s user 0.002s sys 0.001s
I20240503 02:14:33.213495 14510 consensus_queue.cc:237] T 24e81fa4c7b249aca961df89f56a8557 P 27f29b79cf9b42f4a5b9838965bbb72d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27f29b79cf9b42f4a5b9838965bbb72d" member_type: VOTER last_known_addr { host: "127.9.151.129" port: 33313 } }
I20240503 02:14:33.213641 14508 tablet_bootstrap.cc:492] T cefdd1e872814ddbb522f4013d55b26c P 27f29b79cf9b42f4a5b9838965bbb72d: Bootstrap starting.
I20240503 02:14:33.215397 14508 tablet_bootstrap.cc:654] T cefdd1e872814ddbb522f4013d55b26c P 27f29b79cf9b42f4a5b9838965bbb72d: Neither blocks nor log segments found. Creating new log.
I20240503 02:14:33.216555 14438 catalog_manager.cc:5484] T 24e81fa4c7b249aca961df89f56a8557 P 27f29b79cf9b42f4a5b9838965bbb72d reported cstate change: term changed from 0 to 1, leader changed from <none> to 27f29b79cf9b42f4a5b9838965bbb72d (127.9.151.129). New cstate: current_term: 1 leader_uuid: "27f29b79cf9b42f4a5b9838965bbb72d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27f29b79cf9b42f4a5b9838965bbb72d" member_type: VOTER last_known_addr { host: "127.9.151.129" port: 33313 } health_report { overall_health: HEALTHY } } }
I20240503 02:14:33.217978 14508 tablet_bootstrap.cc:492] T cefdd1e872814ddbb522f4013d55b26c P 27f29b79cf9b42f4a5b9838965bbb72d: No bootstrap required, opened a new log
I20240503 02:14:33.218137 14508 ts_tablet_manager.cc:1367] T cefdd1e872814ddbb522f4013d55b26c P 27f29b79cf9b42f4a5b9838965bbb72d: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.003s
I20240503 02:14:33.218935 14508 raft_consensus.cc:348] T cefdd1e872814ddbb522f4013d55b26c P 27f29b79cf9b42f4a5b9838965bbb72d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27f29b79cf9b42f4a5b9838965bbb72d" member_type: VOTER last_known_addr { host: "127.9.151.129" port: 33313 } }
I20240503 02:14:33.219184 14508 raft_consensus.cc:374] T cefdd1e872814ddbb522f4013d55b26c P 27f29b79cf9b42f4a5b9838965bbb72d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:14:33.219271 14508 raft_consensus.cc:729] T cefdd1e872814ddbb522f4013d55b26c P 27f29b79cf9b42f4a5b9838965bbb72d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 27f29b79cf9b42f4a5b9838965bbb72d, State: Initialized, Role: FOLLOWER
I20240503 02:14:33.219518 14508 consensus_queue.cc:260] T cefdd1e872814ddbb522f4013d55b26c P 27f29b79cf9b42f4a5b9838965bbb72d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27f29b79cf9b42f4a5b9838965bbb72d" member_type: VOTER last_known_addr { host: "127.9.151.129" port: 33313 } }
I20240503 02:14:33.219682 14508 raft_consensus.cc:388] T cefdd1e872814ddbb522f4013d55b26c P 27f29b79cf9b42f4a5b9838965bbb72d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20240503 02:14:33.219800 14508 raft_consensus.cc:482] T cefdd1e872814ddbb522f4013d55b26c P 27f29b79cf9b42f4a5b9838965bbb72d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20240503 02:14:33.219903 14508 raft_consensus.cc:3037] T cefdd1e872814ddbb522f4013d55b26c P 27f29b79cf9b42f4a5b9838965bbb72d [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:14:33.222433 14508 raft_consensus.cc:504] T cefdd1e872814ddbb522f4013d55b26c P 27f29b79cf9b42f4a5b9838965bbb72d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27f29b79cf9b42f4a5b9838965bbb72d" member_type: VOTER last_known_addr { host: "127.9.151.129" port: 33313 } }
I20240503 02:14:33.222749 14508 leader_election.cc:302] T cefdd1e872814ddbb522f4013d55b26c P 27f29b79cf9b42f4a5b9838965bbb72d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 27f29b79cf9b42f4a5b9838965bbb72d; no voters:
I20240503 02:14:33.222977 14508 leader_election.cc:288] T cefdd1e872814ddbb522f4013d55b26c P 27f29b79cf9b42f4a5b9838965bbb72d [CANDIDATE]: Term 1 election: Requested vote from peers
I20240503 02:14:33.223088 14510 raft_consensus.cc:2781] T cefdd1e872814ddbb522f4013d55b26c P 27f29b79cf9b42f4a5b9838965bbb72d [term 1 FOLLOWER]: Leader election won for term 1
I20240503 02:14:33.223277 14510 raft_consensus.cc:686] T cefdd1e872814ddbb522f4013d55b26c P 27f29b79cf9b42f4a5b9838965bbb72d [term 1 LEADER]: Becoming Leader. State: Replica: 27f29b79cf9b42f4a5b9838965bbb72d, State: Running, Role: LEADER
I20240503 02:14:33.223610 14508 ts_tablet_manager.cc:1398] T cefdd1e872814ddbb522f4013d55b26c P 27f29b79cf9b42f4a5b9838965bbb72d: Time spent starting tablet: real 0.005s user 0.005s sys 0.001s
I20240503 02:14:33.223589 14510 consensus_queue.cc:237] T cefdd1e872814ddbb522f4013d55b26c P 27f29b79cf9b42f4a5b9838965bbb72d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27f29b79cf9b42f4a5b9838965bbb72d" member_type: VOTER last_known_addr { host: "127.9.151.129" port: 33313 } }
I20240503 02:14:33.223944 14508 tablet_bootstrap.cc:492] T ac0fc766d4b04fb1a7c996c57d383349 P 27f29b79cf9b42f4a5b9838965bbb72d: Bootstrap starting.
I20240503 02:14:33.225914 14508 tablet_bootstrap.cc:654] T ac0fc766d4b04fb1a7c996c57d383349 P 27f29b79cf9b42f4a5b9838965bbb72d: Neither blocks nor log segments found. Creating new log.
I20240503 02:14:33.227260 14438 catalog_manager.cc:5484] T cefdd1e872814ddbb522f4013d55b26c P 27f29b79cf9b42f4a5b9838965bbb72d reported cstate change: term changed from 0 to 1, leader changed from <none> to 27f29b79cf9b42f4a5b9838965bbb72d (127.9.151.129). New cstate: current_term: 1 leader_uuid: "27f29b79cf9b42f4a5b9838965bbb72d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27f29b79cf9b42f4a5b9838965bbb72d" member_type: VOTER last_known_addr { host: "127.9.151.129" port: 33313 } health_report { overall_health: HEALTHY } } }
I20240503 02:14:33.228561 14508 tablet_bootstrap.cc:492] T ac0fc766d4b04fb1a7c996c57d383349 P 27f29b79cf9b42f4a5b9838965bbb72d: No bootstrap required, opened a new log
I20240503 02:14:33.228713 14508 ts_tablet_manager.cc:1367] T ac0fc766d4b04fb1a7c996c57d383349 P 27f29b79cf9b42f4a5b9838965bbb72d: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20240503 02:14:33.229532 14508 raft_consensus.cc:348] T ac0fc766d4b04fb1a7c996c57d383349 P 27f29b79cf9b42f4a5b9838965bbb72d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27f29b79cf9b42f4a5b9838965bbb72d" member_type: VOTER last_known_addr { host: "127.9.151.129" port: 33313 } }
I20240503 02:14:33.229712 14508 raft_consensus.cc:374] T ac0fc766d4b04fb1a7c996c57d383349 P 27f29b79cf9b42f4a5b9838965bbb72d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:14:33.229799 14508 raft_consensus.cc:729] T ac0fc766d4b04fb1a7c996c57d383349 P 27f29b79cf9b42f4a5b9838965bbb72d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 27f29b79cf9b42f4a5b9838965bbb72d, State: Initialized, Role: FOLLOWER
I20240503 02:14:33.230015 14508 consensus_queue.cc:260] T ac0fc766d4b04fb1a7c996c57d383349 P 27f29b79cf9b42f4a5b9838965bbb72d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27f29b79cf9b42f4a5b9838965bbb72d" member_type: VOTER last_known_addr { host: "127.9.151.129" port: 33313 } }
I20240503 02:14:33.230209 14508 raft_consensus.cc:388] T ac0fc766d4b04fb1a7c996c57d383349 P 27f29b79cf9b42f4a5b9838965bbb72d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20240503 02:14:33.230288 14508 raft_consensus.cc:482] T ac0fc766d4b04fb1a7c996c57d383349 P 27f29b79cf9b42f4a5b9838965bbb72d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20240503 02:14:33.230396 14508 raft_consensus.cc:3037] T ac0fc766d4b04fb1a7c996c57d383349 P 27f29b79cf9b42f4a5b9838965bbb72d [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:14:33.232872 14508 raft_consensus.cc:504] T ac0fc766d4b04fb1a7c996c57d383349 P 27f29b79cf9b42f4a5b9838965bbb72d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27f29b79cf9b42f4a5b9838965bbb72d" member_type: VOTER last_known_addr { host: "127.9.151.129" port: 33313 } }
I20240503 02:14:33.233166 14508 leader_election.cc:302] T ac0fc766d4b04fb1a7c996c57d383349 P 27f29b79cf9b42f4a5b9838965bbb72d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 27f29b79cf9b42f4a5b9838965bbb72d; no voters:
I20240503 02:14:33.233381 14508 leader_election.cc:288] T ac0fc766d4b04fb1a7c996c57d383349 P 27f29b79cf9b42f4a5b9838965bbb72d [CANDIDATE]: Term 1 election: Requested vote from peers
I20240503 02:14:33.233537 14510 raft_consensus.cc:2781] T ac0fc766d4b04fb1a7c996c57d383349 P 27f29b79cf9b42f4a5b9838965bbb72d [term 1 FOLLOWER]: Leader election won for term 1
I20240503 02:14:33.233891 14508 ts_tablet_manager.cc:1398] T ac0fc766d4b04fb1a7c996c57d383349 P 27f29b79cf9b42f4a5b9838965bbb72d: Time spent starting tablet: real 0.005s user 0.003s sys 0.000s
I20240503 02:14:33.234151 14510 raft_consensus.cc:686] T ac0fc766d4b04fb1a7c996c57d383349 P 27f29b79cf9b42f4a5b9838965bbb72d [term 1 LEADER]: Becoming Leader. State: Replica: 27f29b79cf9b42f4a5b9838965bbb72d, State: Running, Role: LEADER
I20240503 02:14:33.234258 14508 tablet_bootstrap.cc:492] T fa681a37b30c4a0b879d3f3b184405fc P 27f29b79cf9b42f4a5b9838965bbb72d: Bootstrap starting.
I20240503 02:14:33.234555 14510 consensus_queue.cc:237] T ac0fc766d4b04fb1a7c996c57d383349 P 27f29b79cf9b42f4a5b9838965bbb72d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27f29b79cf9b42f4a5b9838965bbb72d" member_type: VOTER last_known_addr { host: "127.9.151.129" port: 33313 } }
I20240503 02:14:33.236732 14508 tablet_bootstrap.cc:654] T fa681a37b30c4a0b879d3f3b184405fc P 27f29b79cf9b42f4a5b9838965bbb72d: Neither blocks nor log segments found. Creating new log.
I20240503 02:14:33.238586 14438 catalog_manager.cc:5484] T ac0fc766d4b04fb1a7c996c57d383349 P 27f29b79cf9b42f4a5b9838965bbb72d reported cstate change: term changed from 0 to 1, leader changed from <none> to 27f29b79cf9b42f4a5b9838965bbb72d (127.9.151.129). New cstate: current_term: 1 leader_uuid: "27f29b79cf9b42f4a5b9838965bbb72d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27f29b79cf9b42f4a5b9838965bbb72d" member_type: VOTER last_known_addr { host: "127.9.151.129" port: 33313 } health_report { overall_health: HEALTHY } } }
I20240503 02:14:33.239485 14508 tablet_bootstrap.cc:492] T fa681a37b30c4a0b879d3f3b184405fc P 27f29b79cf9b42f4a5b9838965bbb72d: No bootstrap required, opened a new log
I20240503 02:14:33.239717 14508 ts_tablet_manager.cc:1367] T fa681a37b30c4a0b879d3f3b184405fc P 27f29b79cf9b42f4a5b9838965bbb72d: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20240503 02:14:33.240666 14508 raft_consensus.cc:348] T fa681a37b30c4a0b879d3f3b184405fc P 27f29b79cf9b42f4a5b9838965bbb72d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27f29b79cf9b42f4a5b9838965bbb72d" member_type: VOTER last_known_addr { host: "127.9.151.129" port: 33313 } }
I20240503 02:14:33.241031 14508 raft_consensus.cc:374] T fa681a37b30c4a0b879d3f3b184405fc P 27f29b79cf9b42f4a5b9838965bbb72d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:14:33.241119 14508 raft_consensus.cc:729] T fa681a37b30c4a0b879d3f3b184405fc P 27f29b79cf9b42f4a5b9838965bbb72d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 27f29b79cf9b42f4a5b9838965bbb72d, State: Initialized, Role: FOLLOWER
I20240503 02:14:33.241469 14508 consensus_queue.cc:260] T fa681a37b30c4a0b879d3f3b184405fc P 27f29b79cf9b42f4a5b9838965bbb72d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27f29b79cf9b42f4a5b9838965bbb72d" member_type: VOTER last_known_addr { host: "127.9.151.129" port: 33313 } }
I20240503 02:14:33.241652 14508 raft_consensus.cc:388] T fa681a37b30c4a0b879d3f3b184405fc P 27f29b79cf9b42f4a5b9838965bbb72d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20240503 02:14:33.241755 14508 raft_consensus.cc:482] T fa681a37b30c4a0b879d3f3b184405fc P 27f29b79cf9b42f4a5b9838965bbb72d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20240503 02:14:33.241868 14508 raft_consensus.cc:3037] T fa681a37b30c4a0b879d3f3b184405fc P 27f29b79cf9b42f4a5b9838965bbb72d [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:14:33.244943 14508 raft_consensus.cc:504] T fa681a37b30c4a0b879d3f3b184405fc P 27f29b79cf9b42f4a5b9838965bbb72d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27f29b79cf9b42f4a5b9838965bbb72d" member_type: VOTER last_known_addr { host: "127.9.151.129" port: 33313 } }
I20240503 02:14:33.245237 14508 leader_election.cc:302] T fa681a37b30c4a0b879d3f3b184405fc P 27f29b79cf9b42f4a5b9838965bbb72d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 27f29b79cf9b42f4a5b9838965bbb72d; no voters:
I20240503 02:14:33.245474 14508 leader_election.cc:288] T fa681a37b30c4a0b879d3f3b184405fc P 27f29b79cf9b42f4a5b9838965bbb72d [CANDIDATE]: Term 1 election: Requested vote from peers
I20240503 02:14:33.245566 14510 raft_consensus.cc:2781] T fa681a37b30c4a0b879d3f3b184405fc P 27f29b79cf9b42f4a5b9838965bbb72d [term 1 FOLLOWER]: Leader election won for term 1
I20240503 02:14:33.245764 14510 raft_consensus.cc:686] T fa681a37b30c4a0b879d3f3b184405fc P 27f29b79cf9b42f4a5b9838965bbb72d [term 1 LEADER]: Becoming Leader. State: Replica: 27f29b79cf9b42f4a5b9838965bbb72d, State: Running, Role: LEADER
I20240503 02:14:33.246058 14510 consensus_queue.cc:237] T fa681a37b30c4a0b879d3f3b184405fc P 27f29b79cf9b42f4a5b9838965bbb72d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27f29b79cf9b42f4a5b9838965bbb72d" member_type: VOTER last_known_addr { host: "127.9.151.129" port: 33313 } }
I20240503 02:14:33.246167 14508 ts_tablet_manager.cc:1398] T fa681a37b30c4a0b879d3f3b184405fc P 27f29b79cf9b42f4a5b9838965bbb72d: Time spent starting tablet: real 0.006s user 0.003s sys 0.004s
I20240503 02:14:33.246542 14508 tablet_bootstrap.cc:492] T b0df8ae119f948eaa5461faa42007713 P 27f29b79cf9b42f4a5b9838965bbb72d: Bootstrap starting.
I20240503 02:14:33.248425 14508 tablet_bootstrap.cc:654] T b0df8ae119f948eaa5461faa42007713 P 27f29b79cf9b42f4a5b9838965bbb72d: Neither blocks nor log segments found. Creating new log.
I20240503 02:14:33.248710 14438 catalog_manager.cc:5484] T fa681a37b30c4a0b879d3f3b184405fc P 27f29b79cf9b42f4a5b9838965bbb72d reported cstate change: term changed from 0 to 1, leader changed from <none> to 27f29b79cf9b42f4a5b9838965bbb72d (127.9.151.129). New cstate: current_term: 1 leader_uuid: "27f29b79cf9b42f4a5b9838965bbb72d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27f29b79cf9b42f4a5b9838965bbb72d" member_type: VOTER last_known_addr { host: "127.9.151.129" port: 33313 } health_report { overall_health: HEALTHY } } }
I20240503 02:14:33.251191 14508 tablet_bootstrap.cc:492] T b0df8ae119f948eaa5461faa42007713 P 27f29b79cf9b42f4a5b9838965bbb72d: No bootstrap required, opened a new log
I20240503 02:14:33.251392 14508 ts_tablet_manager.cc:1367] T b0df8ae119f948eaa5461faa42007713 P 27f29b79cf9b42f4a5b9838965bbb72d: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20240503 02:14:33.252305 14508 raft_consensus.cc:348] T b0df8ae119f948eaa5461faa42007713 P 27f29b79cf9b42f4a5b9838965bbb72d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27f29b79cf9b42f4a5b9838965bbb72d" member_type: VOTER last_known_addr { host: "127.9.151.129" port: 33313 } }
I20240503 02:14:33.252537 14508 raft_consensus.cc:374] T b0df8ae119f948eaa5461faa42007713 P 27f29b79cf9b42f4a5b9838965bbb72d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:14:33.252636 14508 raft_consensus.cc:729] T b0df8ae119f948eaa5461faa42007713 P 27f29b79cf9b42f4a5b9838965bbb72d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 27f29b79cf9b42f4a5b9838965bbb72d, State: Initialized, Role: FOLLOWER
I20240503 02:14:33.252910 14508 consensus_queue.cc:260] T b0df8ae119f948eaa5461faa42007713 P 27f29b79cf9b42f4a5b9838965bbb72d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27f29b79cf9b42f4a5b9838965bbb72d" member_type: VOTER last_known_addr { host: "127.9.151.129" port: 33313 } }
I20240503 02:14:33.253119 14508 raft_consensus.cc:388] T b0df8ae119f948eaa5461faa42007713 P 27f29b79cf9b42f4a5b9838965bbb72d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20240503 02:14:33.253237 14508 raft_consensus.cc:482] T b0df8ae119f948eaa5461faa42007713 P 27f29b79cf9b42f4a5b9838965bbb72d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20240503 02:14:33.253373 14508 raft_consensus.cc:3037] T b0df8ae119f948eaa5461faa42007713 P 27f29b79cf9b42f4a5b9838965bbb72d [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:14:33.255463 14508 raft_consensus.cc:504] T b0df8ae119f948eaa5461faa42007713 P 27f29b79cf9b42f4a5b9838965bbb72d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27f29b79cf9b42f4a5b9838965bbb72d" member_type: VOTER last_known_addr { host: "127.9.151.129" port: 33313 } }
I20240503 02:14:33.255826 14508 leader_election.cc:302] T b0df8ae119f948eaa5461faa42007713 P 27f29b79cf9b42f4a5b9838965bbb72d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 27f29b79cf9b42f4a5b9838965bbb72d; no voters:
I20240503 02:14:33.256098 14508 leader_election.cc:288] T b0df8ae119f948eaa5461faa42007713 P 27f29b79cf9b42f4a5b9838965bbb72d [CANDIDATE]: Term 1 election: Requested vote from peers
I20240503 02:14:33.256217 14510 raft_consensus.cc:2781] T b0df8ae119f948eaa5461faa42007713 P 27f29b79cf9b42f4a5b9838965bbb72d [term 1 FOLLOWER]: Leader election won for term 1
I20240503 02:14:33.256438 14510 raft_consensus.cc:686] T b0df8ae119f948eaa5461faa42007713 P 27f29b79cf9b42f4a5b9838965bbb72d [term 1 LEADER]: Becoming Leader. State: Replica: 27f29b79cf9b42f4a5b9838965bbb72d, State: Running, Role: LEADER
I20240503 02:14:33.256714 14508 ts_tablet_manager.cc:1398] T b0df8ae119f948eaa5461faa42007713 P 27f29b79cf9b42f4a5b9838965bbb72d: Time spent starting tablet: real 0.005s user 0.004s sys 0.000s
I20240503 02:14:33.256752 14510 consensus_queue.cc:237] T b0df8ae119f948eaa5461faa42007713 P 27f29b79cf9b42f4a5b9838965bbb72d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27f29b79cf9b42f4a5b9838965bbb72d" member_type: VOTER last_known_addr { host: "127.9.151.129" port: 33313 } }
I20240503 02:14:33.257056 14508 tablet_bootstrap.cc:492] T fc8f5043623a48418283e5ecd50d58e2 P 27f29b79cf9b42f4a5b9838965bbb72d: Bootstrap starting.
I20240503 02:14:33.259341 14508 tablet_bootstrap.cc:654] T fc8f5043623a48418283e5ecd50d58e2 P 27f29b79cf9b42f4a5b9838965bbb72d: Neither blocks nor log segments found. Creating new log.
I20240503 02:14:33.260114 14438 catalog_manager.cc:5484] T b0df8ae119f948eaa5461faa42007713 P 27f29b79cf9b42f4a5b9838965bbb72d reported cstate change: term changed from 0 to 1, leader changed from <none> to 27f29b79cf9b42f4a5b9838965bbb72d (127.9.151.129). New cstate: current_term: 1 leader_uuid: "27f29b79cf9b42f4a5b9838965bbb72d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27f29b79cf9b42f4a5b9838965bbb72d" member_type: VOTER last_known_addr { host: "127.9.151.129" port: 33313 } health_report { overall_health: HEALTHY } } }
I20240503 02:14:33.261380 14508 tablet_bootstrap.cc:492] T fc8f5043623a48418283e5ecd50d58e2 P 27f29b79cf9b42f4a5b9838965bbb72d: No bootstrap required, opened a new log
I20240503 02:14:33.261567 14508 ts_tablet_manager.cc:1367] T fc8f5043623a48418283e5ecd50d58e2 P 27f29b79cf9b42f4a5b9838965bbb72d: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.003s
I20240503 02:14:33.262581 14508 raft_consensus.cc:348] T fc8f5043623a48418283e5ecd50d58e2 P 27f29b79cf9b42f4a5b9838965bbb72d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27f29b79cf9b42f4a5b9838965bbb72d" member_type: VOTER last_known_addr { host: "127.9.151.129" port: 33313 } }
I20240503 02:14:33.262882 14508 raft_consensus.cc:374] T fc8f5043623a48418283e5ecd50d58e2 P 27f29b79cf9b42f4a5b9838965bbb72d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:14:33.263095 14508 raft_consensus.cc:729] T fc8f5043623a48418283e5ecd50d58e2 P 27f29b79cf9b42f4a5b9838965bbb72d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 27f29b79cf9b42f4a5b9838965bbb72d, State: Initialized, Role: FOLLOWER
I20240503 02:14:33.263478 14508 consensus_queue.cc:260] T fc8f5043623a48418283e5ecd50d58e2 P 27f29b79cf9b42f4a5b9838965bbb72d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27f29b79cf9b42f4a5b9838965bbb72d" member_type: VOTER last_known_addr { host: "127.9.151.129" port: 33313 } }
I20240503 02:14:33.263701 14508 raft_consensus.cc:388] T fc8f5043623a48418283e5ecd50d58e2 P 27f29b79cf9b42f4a5b9838965bbb72d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20240503 02:14:33.263916 14508 raft_consensus.cc:482] T fc8f5043623a48418283e5ecd50d58e2 P 27f29b79cf9b42f4a5b9838965bbb72d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20240503 02:14:33.264060 14508 raft_consensus.cc:3037] T fc8f5043623a48418283e5ecd50d58e2 P 27f29b79cf9b42f4a5b9838965bbb72d [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:14:33.267138 14508 raft_consensus.cc:504] T fc8f5043623a48418283e5ecd50d58e2 P 27f29b79cf9b42f4a5b9838965bbb72d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27f29b79cf9b42f4a5b9838965bbb72d" member_type: VOTER last_known_addr { host: "127.9.151.129" port: 33313 } }
I20240503 02:14:33.267715 14508 leader_election.cc:302] T fc8f5043623a48418283e5ecd50d58e2 P 27f29b79cf9b42f4a5b9838965bbb72d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 27f29b79cf9b42f4a5b9838965bbb72d; no voters:
I20240503 02:14:33.268025 14508 leader_election.cc:288] T fc8f5043623a48418283e5ecd50d58e2 P 27f29b79cf9b42f4a5b9838965bbb72d [CANDIDATE]: Term 1 election: Requested vote from peers
I20240503 02:14:33.268143 14510 raft_consensus.cc:2781] T fc8f5043623a48418283e5ecd50d58e2 P 27f29b79cf9b42f4a5b9838965bbb72d [term 1 FOLLOWER]: Leader election won for term 1
I20240503 02:14:33.268330 14510 raft_consensus.cc:686] T fc8f5043623a48418283e5ecd50d58e2 P 27f29b79cf9b42f4a5b9838965bbb72d [term 1 LEADER]: Becoming Leader. State: Replica: 27f29b79cf9b42f4a5b9838965bbb72d, State: Running, Role: LEADER
I20240503 02:14:33.268584 14508 ts_tablet_manager.cc:1398] T fc8f5043623a48418283e5ecd50d58e2 P 27f29b79cf9b42f4a5b9838965bbb72d: Time spent starting tablet: real 0.007s user 0.006s sys 0.000s
I20240503 02:14:33.268612 14510 consensus_queue.cc:237] T fc8f5043623a48418283e5ecd50d58e2 P 27f29b79cf9b42f4a5b9838965bbb72d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27f29b79cf9b42f4a5b9838965bbb72d" member_type: VOTER last_known_addr { host: "127.9.151.129" port: 33313 } }
I20240503 02:14:33.268898 14508 tablet_bootstrap.cc:492] T ef909312fe9545b18bf9f918f52928c2 P 27f29b79cf9b42f4a5b9838965bbb72d: Bootstrap starting.
I20240503 02:14:33.270715 14508 tablet_bootstrap.cc:654] T ef909312fe9545b18bf9f918f52928c2 P 27f29b79cf9b42f4a5b9838965bbb72d: Neither blocks nor log segments found. Creating new log.
I20240503 02:14:33.271833 14438 catalog_manager.cc:5484] T fc8f5043623a48418283e5ecd50d58e2 P 27f29b79cf9b42f4a5b9838965bbb72d reported cstate change: term changed from 0 to 1, leader changed from <none> to 27f29b79cf9b42f4a5b9838965bbb72d (127.9.151.129). New cstate: current_term: 1 leader_uuid: "27f29b79cf9b42f4a5b9838965bbb72d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27f29b79cf9b42f4a5b9838965bbb72d" member_type: VOTER last_known_addr { host: "127.9.151.129" port: 33313 } health_report { overall_health: HEALTHY } } }
I20240503 02:14:33.272769 14508 tablet_bootstrap.cc:492] T ef909312fe9545b18bf9f918f52928c2 P 27f29b79cf9b42f4a5b9838965bbb72d: No bootstrap required, opened a new log
I20240503 02:14:33.272927 14508 ts_tablet_manager.cc:1367] T ef909312fe9545b18bf9f918f52928c2 P 27f29b79cf9b42f4a5b9838965bbb72d: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20240503 02:14:33.273516 14508 raft_consensus.cc:348] T ef909312fe9545b18bf9f918f52928c2 P 27f29b79cf9b42f4a5b9838965bbb72d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27f29b79cf9b42f4a5b9838965bbb72d" member_type: VOTER last_known_addr { host: "127.9.151.129" port: 33313 } }
I20240503 02:14:33.273694 14508 raft_consensus.cc:374] T ef909312fe9545b18bf9f918f52928c2 P 27f29b79cf9b42f4a5b9838965bbb72d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:14:33.273795 14508 raft_consensus.cc:729] T ef909312fe9545b18bf9f918f52928c2 P 27f29b79cf9b42f4a5b9838965bbb72d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 27f29b79cf9b42f4a5b9838965bbb72d, State: Initialized, Role: FOLLOWER
I20240503 02:14:33.274035 14508 consensus_queue.cc:260] T ef909312fe9545b18bf9f918f52928c2 P 27f29b79cf9b42f4a5b9838965bbb72d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27f29b79cf9b42f4a5b9838965bbb72d" member_type: VOTER last_known_addr { host: "127.9.151.129" port: 33313 } }
I20240503 02:14:33.274227 14508 raft_consensus.cc:388] T ef909312fe9545b18bf9f918f52928c2 P 27f29b79cf9b42f4a5b9838965bbb72d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20240503 02:14:33.274313 14508 raft_consensus.cc:482] T ef909312fe9545b18bf9f918f52928c2 P 27f29b79cf9b42f4a5b9838965bbb72d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20240503 02:14:33.274431 14508 raft_consensus.cc:3037] T ef909312fe9545b18bf9f918f52928c2 P 27f29b79cf9b42f4a5b9838965bbb72d [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:14:33.277079 14508 raft_consensus.cc:504] T ef909312fe9545b18bf9f918f52928c2 P 27f29b79cf9b42f4a5b9838965bbb72d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27f29b79cf9b42f4a5b9838965bbb72d" member_type: VOTER last_known_addr { host: "127.9.151.129" port: 33313 } }
I20240503 02:14:33.277379 14508 leader_election.cc:302] T ef909312fe9545b18bf9f918f52928c2 P 27f29b79cf9b42f4a5b9838965bbb72d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 27f29b79cf9b42f4a5b9838965bbb72d; no voters:
I20240503 02:14:33.277642 14508 leader_election.cc:288] T ef909312fe9545b18bf9f918f52928c2 P 27f29b79cf9b42f4a5b9838965bbb72d [CANDIDATE]: Term 1 election: Requested vote from peers
I20240503 02:14:33.277741 14510 raft_consensus.cc:2781] T ef909312fe9545b18bf9f918f52928c2 P 27f29b79cf9b42f4a5b9838965bbb72d [term 1 FOLLOWER]: Leader election won for term 1
I20240503 02:14:33.277971 14510 raft_consensus.cc:686] T ef909312fe9545b18bf9f918f52928c2 P 27f29b79cf9b42f4a5b9838965bbb72d [term 1 LEADER]: Becoming Leader. State: Replica: 27f29b79cf9b42f4a5b9838965bbb72d, State: Running, Role: LEADER
I20240503 02:14:33.278259 14508 ts_tablet_manager.cc:1398] T ef909312fe9545b18bf9f918f52928c2 P 27f29b79cf9b42f4a5b9838965bbb72d: Time spent starting tablet: real 0.005s user 0.000s sys 0.003s
I20240503 02:14:33.278265 14510 consensus_queue.cc:237] T ef909312fe9545b18bf9f918f52928c2 P 27f29b79cf9b42f4a5b9838965bbb72d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27f29b79cf9b42f4a5b9838965bbb72d" member_type: VOTER last_known_addr { host: "127.9.151.129" port: 33313 } }
I20240503 02:14:33.278582 14508 tablet_bootstrap.cc:492] T 97c56e26ef4f47f4acf50d0b8d8116dc P 27f29b79cf9b42f4a5b9838965bbb72d: Bootstrap starting.
I20240503 02:14:33.281072 14508 tablet_bootstrap.cc:654] T 97c56e26ef4f47f4acf50d0b8d8116dc P 27f29b79cf9b42f4a5b9838965bbb72d: Neither blocks nor log segments found. Creating new log.
I20240503 02:14:33.282511 14438 catalog_manager.cc:5484] T ef909312fe9545b18bf9f918f52928c2 P 27f29b79cf9b42f4a5b9838965bbb72d reported cstate change: term changed from 0 to 1, leader changed from <none> to 27f29b79cf9b42f4a5b9838965bbb72d (127.9.151.129). New cstate: current_term: 1 leader_uuid: "27f29b79cf9b42f4a5b9838965bbb72d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27f29b79cf9b42f4a5b9838965bbb72d" member_type: VOTER last_known_addr { host: "127.9.151.129" port: 33313 } health_report { overall_health: HEALTHY } } }
I20240503 02:14:33.283159 14508 tablet_bootstrap.cc:492] T 97c56e26ef4f47f4acf50d0b8d8116dc P 27f29b79cf9b42f4a5b9838965bbb72d: No bootstrap required, opened a new log
I20240503 02:14:33.283452 14508 ts_tablet_manager.cc:1367] T 97c56e26ef4f47f4acf50d0b8d8116dc P 27f29b79cf9b42f4a5b9838965bbb72d: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20240503 02:14:33.284323 14508 raft_consensus.cc:348] T 97c56e26ef4f47f4acf50d0b8d8116dc P 27f29b79cf9b42f4a5b9838965bbb72d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27f29b79cf9b42f4a5b9838965bbb72d" member_type: VOTER last_known_addr { host: "127.9.151.129" port: 33313 } }
I20240503 02:14:33.284497 14508 raft_consensus.cc:374] T 97c56e26ef4f47f4acf50d0b8d8116dc P 27f29b79cf9b42f4a5b9838965bbb72d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:14:33.284586 14508 raft_consensus.cc:729] T 97c56e26ef4f47f4acf50d0b8d8116dc P 27f29b79cf9b42f4a5b9838965bbb72d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 27f29b79cf9b42f4a5b9838965bbb72d, State: Initialized, Role: FOLLOWER
I20240503 02:14:33.284874 14508 consensus_queue.cc:260] T 97c56e26ef4f47f4acf50d0b8d8116dc P 27f29b79cf9b42f4a5b9838965bbb72d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27f29b79cf9b42f4a5b9838965bbb72d" member_type: VOTER last_known_addr { host: "127.9.151.129" port: 33313 } }
I20240503 02:14:33.285053 14508 raft_consensus.cc:388] T 97c56e26ef4f47f4acf50d0b8d8116dc P 27f29b79cf9b42f4a5b9838965bbb72d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20240503 02:14:33.285161 14508 raft_consensus.cc:482] T 97c56e26ef4f47f4acf50d0b8d8116dc P 27f29b79cf9b42f4a5b9838965bbb72d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20240503 02:14:33.285262 14508 raft_consensus.cc:3037] T 97c56e26ef4f47f4acf50d0b8d8116dc P 27f29b79cf9b42f4a5b9838965bbb72d [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:14:33.287585 14508 raft_consensus.cc:504] T 97c56e26ef4f47f4acf50d0b8d8116dc P 27f29b79cf9b42f4a5b9838965bbb72d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27f29b79cf9b42f4a5b9838965bbb72d" member_type: VOTER last_known_addr { host: "127.9.151.129" port: 33313 } }
I20240503 02:14:33.287997 14508 leader_election.cc:302] T 97c56e26ef4f47f4acf50d0b8d8116dc P 27f29b79cf9b42f4a5b9838965bbb72d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 27f29b79cf9b42f4a5b9838965bbb72d; no voters:
I20240503 02:14:33.288247 14508 leader_election.cc:288] T 97c56e26ef4f47f4acf50d0b8d8116dc P 27f29b79cf9b42f4a5b9838965bbb72d [CANDIDATE]: Term 1 election: Requested vote from peers
I20240503 02:14:33.288347 14510 raft_consensus.cc:2781] T 97c56e26ef4f47f4acf50d0b8d8116dc P 27f29b79cf9b42f4a5b9838965bbb72d [term 1 FOLLOWER]: Leader election won for term 1
I20240503 02:14:33.288512 14510 raft_consensus.cc:686] T 97c56e26ef4f47f4acf50d0b8d8116dc P 27f29b79cf9b42f4a5b9838965bbb72d [term 1 LEADER]: Becoming Leader. State: Replica: 27f29b79cf9b42f4a5b9838965bbb72d, State: Running, Role: LEADER
I20240503 02:14:33.288786 14510 consensus_queue.cc:237] T 97c56e26ef4f47f4acf50d0b8d8116dc P 27f29b79cf9b42f4a5b9838965bbb72d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27f29b79cf9b42f4a5b9838965bbb72d" member_type: VOTER last_known_addr { host: "127.9.151.129" port: 33313 } }
I20240503 02:14:33.289077 14508 ts_tablet_manager.cc:1398] T 97c56e26ef4f47f4acf50d0b8d8116dc P 27f29b79cf9b42f4a5b9838965bbb72d: Time spent starting tablet: real 0.005s user 0.001s sys 0.003s
I20240503 02:14:33.289418 14508 tablet_bootstrap.cc:492] T be3ab5a5dd774bdba45e6d7061019195 P 27f29b79cf9b42f4a5b9838965bbb72d: Bootstrap starting.
I20240503 02:14:33.291558 14508 tablet_bootstrap.cc:654] T be3ab5a5dd774bdba45e6d7061019195 P 27f29b79cf9b42f4a5b9838965bbb72d: Neither blocks nor log segments found. Creating new log.
I20240503 02:14:33.291981 14438 catalog_manager.cc:5484] T 97c56e26ef4f47f4acf50d0b8d8116dc P 27f29b79cf9b42f4a5b9838965bbb72d reported cstate change: term changed from 0 to 1, leader changed from <none> to 27f29b79cf9b42f4a5b9838965bbb72d (127.9.151.129). New cstate: current_term: 1 leader_uuid: "27f29b79cf9b42f4a5b9838965bbb72d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27f29b79cf9b42f4a5b9838965bbb72d" member_type: VOTER last_known_addr { host: "127.9.151.129" port: 33313 } health_report { overall_health: HEALTHY } } }
I20240503 02:14:33.293534 14508 tablet_bootstrap.cc:492] T be3ab5a5dd774bdba45e6d7061019195 P 27f29b79cf9b42f4a5b9838965bbb72d: No bootstrap required, opened a new log
I20240503 02:14:33.293701 14508 ts_tablet_manager.cc:1367] T be3ab5a5dd774bdba45e6d7061019195 P 27f29b79cf9b42f4a5b9838965bbb72d: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20240503 02:14:33.294450 14508 raft_consensus.cc:348] T be3ab5a5dd774bdba45e6d7061019195 P 27f29b79cf9b42f4a5b9838965bbb72d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27f29b79cf9b42f4a5b9838965bbb72d" member_type: VOTER last_known_addr { host: "127.9.151.129" port: 33313 } }
I20240503 02:14:33.294641 14508 raft_consensus.cc:374] T be3ab5a5dd774bdba45e6d7061019195 P 27f29b79cf9b42f4a5b9838965bbb72d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:14:33.294777 14508 raft_consensus.cc:729] T be3ab5a5dd774bdba45e6d7061019195 P 27f29b79cf9b42f4a5b9838965bbb72d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 27f29b79cf9b42f4a5b9838965bbb72d, State: Initialized, Role: FOLLOWER
I20240503 02:14:33.295073 14508 consensus_queue.cc:260] T be3ab5a5dd774bdba45e6d7061019195 P 27f29b79cf9b42f4a5b9838965bbb72d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27f29b79cf9b42f4a5b9838965bbb72d" member_type: VOTER last_known_addr { host: "127.9.151.129" port: 33313 } }
I20240503 02:14:33.295262 14508 raft_consensus.cc:388] T be3ab5a5dd774bdba45e6d7061019195 P 27f29b79cf9b42f4a5b9838965bbb72d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20240503 02:14:33.295347 14508 raft_consensus.cc:482] T be3ab5a5dd774bdba45e6d7061019195 P 27f29b79cf9b42f4a5b9838965bbb72d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20240503 02:14:33.295464 14508 raft_consensus.cc:3037] T be3ab5a5dd774bdba45e6d7061019195 P 27f29b79cf9b42f4a5b9838965bbb72d [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:14:33.297777 14508 raft_consensus.cc:504] T be3ab5a5dd774bdba45e6d7061019195 P 27f29b79cf9b42f4a5b9838965bbb72d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27f29b79cf9b42f4a5b9838965bbb72d" member_type: VOTER last_known_addr { host: "127.9.151.129" port: 33313 } }
I20240503 02:14:33.298084 14508 leader_election.cc:302] T be3ab5a5dd774bdba45e6d7061019195 P 27f29b79cf9b42f4a5b9838965bbb72d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 27f29b79cf9b42f4a5b9838965bbb72d; no voters:
I20240503 02:14:33.298324 14508 leader_election.cc:288] T be3ab5a5dd774bdba45e6d7061019195 P 27f29b79cf9b42f4a5b9838965bbb72d [CANDIDATE]: Term 1 election: Requested vote from peers
I20240503 02:14:33.298597 14512 raft_consensus.cc:2781] T be3ab5a5dd774bdba45e6d7061019195 P 27f29b79cf9b42f4a5b9838965bbb72d [term 1 FOLLOWER]: Leader election won for term 1
I20240503 02:14:33.298842 14512 raft_consensus.cc:686] T be3ab5a5dd774bdba45e6d7061019195 P 27f29b79cf9b42f4a5b9838965bbb72d [term 1 LEADER]: Becoming Leader. State: Replica: 27f29b79cf9b42f4a5b9838965bbb72d, State: Running, Role: LEADER
I20240503 02:14:33.298962 14508 ts_tablet_manager.cc:1398] T be3ab5a5dd774bdba45e6d7061019195 P 27f29b79cf9b42f4a5b9838965bbb72d: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20240503 02:14:33.299293 14508 tablet_bootstrap.cc:492] T 92cd1f3c9cdc43e0b9d56e3375488f50 P 27f29b79cf9b42f4a5b9838965bbb72d: Bootstrap starting.
I20240503 02:14:33.299206 14512 consensus_queue.cc:237] T be3ab5a5dd774bdba45e6d7061019195 P 27f29b79cf9b42f4a5b9838965bbb72d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27f29b79cf9b42f4a5b9838965bbb72d" member_type: VOTER last_known_addr { host: "127.9.151.129" port: 33313 } }
I20240503 02:14:33.301046 14508 tablet_bootstrap.cc:654] T 92cd1f3c9cdc43e0b9d56e3375488f50 P 27f29b79cf9b42f4a5b9838965bbb72d: Neither blocks nor log segments found. Creating new log.
I20240503 02:14:33.303242 14438 catalog_manager.cc:5484] T be3ab5a5dd774bdba45e6d7061019195 P 27f29b79cf9b42f4a5b9838965bbb72d reported cstate change: term changed from 0 to 1, leader changed from <none> to 27f29b79cf9b42f4a5b9838965bbb72d (127.9.151.129). New cstate: current_term: 1 leader_uuid: "27f29b79cf9b42f4a5b9838965bbb72d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27f29b79cf9b42f4a5b9838965bbb72d" member_type: VOTER last_known_addr { host: "127.9.151.129" port: 33313 } health_report { overall_health: HEALTHY } } }
I20240503 02:14:33.305457 14508 tablet_bootstrap.cc:492] T 92cd1f3c9cdc43e0b9d56e3375488f50 P 27f29b79cf9b42f4a5b9838965bbb72d: No bootstrap required, opened a new log
I20240503 02:14:33.305650 14508 ts_tablet_manager.cc:1367] T 92cd1f3c9cdc43e0b9d56e3375488f50 P 27f29b79cf9b42f4a5b9838965bbb72d: Time spent bootstrapping tablet: real 0.006s user 0.002s sys 0.004s
I20240503 02:14:33.306418 14508 raft_consensus.cc:348] T 92cd1f3c9cdc43e0b9d56e3375488f50 P 27f29b79cf9b42f4a5b9838965bbb72d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27f29b79cf9b42f4a5b9838965bbb72d" member_type: VOTER last_known_addr { host: "127.9.151.129" port: 33313 } }
I20240503 02:14:33.306658 14508 raft_consensus.cc:374] T 92cd1f3c9cdc43e0b9d56e3375488f50 P 27f29b79cf9b42f4a5b9838965bbb72d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:14:33.306778 14508 raft_consensus.cc:729] T 92cd1f3c9cdc43e0b9d56e3375488f50 P 27f29b79cf9b42f4a5b9838965bbb72d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 27f29b79cf9b42f4a5b9838965bbb72d, State: Initialized, Role: FOLLOWER
I20240503 02:14:33.307101 14508 consensus_queue.cc:260] T 92cd1f3c9cdc43e0b9d56e3375488f50 P 27f29b79cf9b42f4a5b9838965bbb72d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27f29b79cf9b42f4a5b9838965bbb72d" member_type: VOTER last_known_addr { host: "127.9.151.129" port: 33313 } }
I20240503 02:14:33.307302 14508 raft_consensus.cc:388] T 92cd1f3c9cdc43e0b9d56e3375488f50 P 27f29b79cf9b42f4a5b9838965bbb72d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20240503 02:14:33.307408 14508 raft_consensus.cc:482] T 92cd1f3c9cdc43e0b9d56e3375488f50 P 27f29b79cf9b42f4a5b9838965bbb72d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20240503 02:14:33.307519 14508 raft_consensus.cc:3037] T 92cd1f3c9cdc43e0b9d56e3375488f50 P 27f29b79cf9b42f4a5b9838965bbb72d [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:14:33.309926 14508 raft_consensus.cc:504] T 92cd1f3c9cdc43e0b9d56e3375488f50 P 27f29b79cf9b42f4a5b9838965bbb72d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27f29b79cf9b42f4a5b9838965bbb72d" member_type: VOTER last_known_addr { host: "127.9.151.129" port: 33313 } }
I20240503 02:14:33.310158 14508 leader_election.cc:302] T 92cd1f3c9cdc43e0b9d56e3375488f50 P 27f29b79cf9b42f4a5b9838965bbb72d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 27f29b79cf9b42f4a5b9838965bbb72d; no voters:
I20240503 02:14:33.310328 14508 leader_election.cc:288] T 92cd1f3c9cdc43e0b9d56e3375488f50 P 27f29b79cf9b42f4a5b9838965bbb72d [CANDIDATE]: Term 1 election: Requested vote from peers
I20240503 02:14:33.310448 14512 raft_consensus.cc:2781] T 92cd1f3c9cdc43e0b9d56e3375488f50 P 27f29b79cf9b42f4a5b9838965bbb72d [term 1 FOLLOWER]: Leader election won for term 1
I20240503 02:14:33.310717 14512 raft_consensus.cc:686] T 92cd1f3c9cdc43e0b9d56e3375488f50 P 27f29b79cf9b42f4a5b9838965bbb72d [term 1 LEADER]: Becoming Leader. State: Replica: 27f29b79cf9b42f4a5b9838965bbb72d, State: Running, Role: LEADER
I20240503 02:14:33.310776 14508 ts_tablet_manager.cc:1398] T 92cd1f3c9cdc43e0b9d56e3375488f50 P 27f29b79cf9b42f4a5b9838965bbb72d: Time spent starting tablet: real 0.005s user 0.002s sys 0.003s
I20240503 02:14:33.311038 14508 tablet_bootstrap.cc:492] T fe8b26bb37ab4b948fd500b648bf6b5a P 27f29b79cf9b42f4a5b9838965bbb72d: Bootstrap starting.
I20240503 02:14:33.310981 14512 consensus_queue.cc:237] T 92cd1f3c9cdc43e0b9d56e3375488f50 P 27f29b79cf9b42f4a5b9838965bbb72d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27f29b79cf9b42f4a5b9838965bbb72d" member_type: VOTER last_known_addr { host: "127.9.151.129" port: 33313 } }
I20240503 02:14:33.312784 14508 tablet_bootstrap.cc:654] T fe8b26bb37ab4b948fd500b648bf6b5a P 27f29b79cf9b42f4a5b9838965bbb72d: Neither blocks nor log segments found. Creating new log.
I20240503 02:14:33.313887 14438 catalog_manager.cc:5484] T 92cd1f3c9cdc43e0b9d56e3375488f50 P 27f29b79cf9b42f4a5b9838965bbb72d reported cstate change: term changed from 0 to 1, leader changed from <none> to 27f29b79cf9b42f4a5b9838965bbb72d (127.9.151.129). New cstate: current_term: 1 leader_uuid: "27f29b79cf9b42f4a5b9838965bbb72d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27f29b79cf9b42f4a5b9838965bbb72d" member_type: VOTER last_known_addr { host: "127.9.151.129" port: 33313 } health_report { overall_health: HEALTHY } } }
I20240503 02:14:33.314807 14508 tablet_bootstrap.cc:492] T fe8b26bb37ab4b948fd500b648bf6b5a P 27f29b79cf9b42f4a5b9838965bbb72d: No bootstrap required, opened a new log
I20240503 02:14:33.314963 14508 ts_tablet_manager.cc:1367] T fe8b26bb37ab4b948fd500b648bf6b5a P 27f29b79cf9b42f4a5b9838965bbb72d: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.001s
I20240503 02:14:33.315747 14508 raft_consensus.cc:348] T fe8b26bb37ab4b948fd500b648bf6b5a P 27f29b79cf9b42f4a5b9838965bbb72d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27f29b79cf9b42f4a5b9838965bbb72d" member_type: VOTER last_known_addr { host: "127.9.151.129" port: 33313 } }
I20240503 02:14:33.315894 14508 raft_consensus.cc:374] T fe8b26bb37ab4b948fd500b648bf6b5a P 27f29b79cf9b42f4a5b9838965bbb72d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:14:33.315955 14508 raft_consensus.cc:729] T fe8b26bb37ab4b948fd500b648bf6b5a P 27f29b79cf9b42f4a5b9838965bbb72d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 27f29b79cf9b42f4a5b9838965bbb72d, State: Initialized, Role: FOLLOWER
I20240503 02:14:33.316164 14508 consensus_queue.cc:260] T fe8b26bb37ab4b948fd500b648bf6b5a P 27f29b79cf9b42f4a5b9838965bbb72d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27f29b79cf9b42f4a5b9838965bbb72d" member_type: VOTER last_known_addr { host: "127.9.151.129" port: 33313 } }
I20240503 02:14:33.316361 14508 raft_consensus.cc:388] T fe8b26bb37ab4b948fd500b648bf6b5a P 27f29b79cf9b42f4a5b9838965bbb72d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20240503 02:14:33.316452 14508 raft_consensus.cc:482] T fe8b26bb37ab4b948fd500b648bf6b5a P 27f29b79cf9b42f4a5b9838965bbb72d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20240503 02:14:33.316606 14508 raft_consensus.cc:3037] T fe8b26bb37ab4b948fd500b648bf6b5a P 27f29b79cf9b42f4a5b9838965bbb72d [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:14:33.318832 14508 raft_consensus.cc:504] T fe8b26bb37ab4b948fd500b648bf6b5a P 27f29b79cf9b42f4a5b9838965bbb72d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27f29b79cf9b42f4a5b9838965bbb72d" member_type: VOTER last_known_addr { host: "127.9.151.129" port: 33313 } }
I20240503 02:14:33.319103 14508 leader_election.cc:302] T fe8b26bb37ab4b948fd500b648bf6b5a P 27f29b79cf9b42f4a5b9838965bbb72d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 27f29b79cf9b42f4a5b9838965bbb72d; no voters:
I20240503 02:14:33.319304 14508 leader_election.cc:288] T fe8b26bb37ab4b948fd500b648bf6b5a P 27f29b79cf9b42f4a5b9838965bbb72d [CANDIDATE]: Term 1 election: Requested vote from peers
I20240503 02:14:33.319406 14512 raft_consensus.cc:2781] T fe8b26bb37ab4b948fd500b648bf6b5a P 27f29b79cf9b42f4a5b9838965bbb72d [term 1 FOLLOWER]: Leader election won for term 1
I20240503 02:14:33.319543 14512 raft_consensus.cc:686] T fe8b26bb37ab4b948fd500b648bf6b5a P 27f29b79cf9b42f4a5b9838965bbb72d [term 1 LEADER]: Becoming Leader. State: Replica: 27f29b79cf9b42f4a5b9838965bbb72d, State: Running, Role: LEADER
I20240503 02:14:33.319845 14508 ts_tablet_manager.cc:1398] T fe8b26bb37ab4b948fd500b648bf6b5a P 27f29b79cf9b42f4a5b9838965bbb72d: Time spent starting tablet: real 0.005s user 0.001s sys 0.003s
I20240503 02:14:33.319936 14512 consensus_queue.cc:237] T fe8b26bb37ab4b948fd500b648bf6b5a P 27f29b79cf9b42f4a5b9838965bbb72d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27f29b79cf9b42f4a5b9838965bbb72d" member_type: VOTER last_known_addr { host: "127.9.151.129" port: 33313 } }
I20240503 02:14:33.320212 14508 tablet_bootstrap.cc:492] T 4ff002252c4d46d2af2a4956acd90b3f P 27f29b79cf9b42f4a5b9838965bbb72d: Bootstrap starting.
I20240503 02:14:33.323042 14438 catalog_manager.cc:5484] T fe8b26bb37ab4b948fd500b648bf6b5a P 27f29b79cf9b42f4a5b9838965bbb72d reported cstate change: term changed from 0 to 1, leader changed from <none> to 27f29b79cf9b42f4a5b9838965bbb72d (127.9.151.129). New cstate: current_term: 1 leader_uuid: "27f29b79cf9b42f4a5b9838965bbb72d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27f29b79cf9b42f4a5b9838965bbb72d" member_type: VOTER last_known_addr { host: "127.9.151.129" port: 33313 } health_report { overall_health: HEALTHY } } }
I20240503 02:14:33.347002 14508 tablet_bootstrap.cc:654] T 4ff002252c4d46d2af2a4956acd90b3f P 27f29b79cf9b42f4a5b9838965bbb72d: Neither blocks nor log segments found. Creating new log.
I20240503 02:14:33.349160 14508 tablet_bootstrap.cc:492] T 4ff002252c4d46d2af2a4956acd90b3f P 27f29b79cf9b42f4a5b9838965bbb72d: No bootstrap required, opened a new log
I20240503 02:14:33.349488 14508 ts_tablet_manager.cc:1367] T 4ff002252c4d46d2af2a4956acd90b3f P 27f29b79cf9b42f4a5b9838965bbb72d: Time spent bootstrapping tablet: real 0.029s user 0.003s sys 0.001s
I20240503 02:14:33.350226 14508 raft_consensus.cc:348] T 4ff002252c4d46d2af2a4956acd90b3f P 27f29b79cf9b42f4a5b9838965bbb72d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27f29b79cf9b42f4a5b9838965bbb72d" member_type: VOTER last_known_addr { host: "127.9.151.129" port: 33313 } }
I20240503 02:14:33.350430 14508 raft_consensus.cc:374] T 4ff002252c4d46d2af2a4956acd90b3f P 27f29b79cf9b42f4a5b9838965bbb72d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:14:33.350518 14508 raft_consensus.cc:729] T 4ff002252c4d46d2af2a4956acd90b3f P 27f29b79cf9b42f4a5b9838965bbb72d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 27f29b79cf9b42f4a5b9838965bbb72d, State: Initialized, Role: FOLLOWER
I20240503 02:14:33.350868 14508 consensus_queue.cc:260] T 4ff002252c4d46d2af2a4956acd90b3f P 27f29b79cf9b42f4a5b9838965bbb72d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27f29b79cf9b42f4a5b9838965bbb72d" member_type: VOTER last_known_addr { host: "127.9.151.129" port: 33313 } }
I20240503 02:14:33.351020 14508 raft_consensus.cc:388] T 4ff002252c4d46d2af2a4956acd90b3f P 27f29b79cf9b42f4a5b9838965bbb72d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20240503 02:14:33.351120 14508 raft_consensus.cc:482] T 4ff002252c4d46d2af2a4956acd90b3f P 27f29b79cf9b42f4a5b9838965bbb72d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20240503 02:14:33.351222 14508 raft_consensus.cc:3037] T 4ff002252c4d46d2af2a4956acd90b3f P 27f29b79cf9b42f4a5b9838965bbb72d [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:14:33.353327 14508 raft_consensus.cc:504] T 4ff002252c4d46d2af2a4956acd90b3f P 27f29b79cf9b42f4a5b9838965bbb72d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27f29b79cf9b42f4a5b9838965bbb72d" member_type: VOTER last_known_addr { host: "127.9.151.129" port: 33313 } }
I20240503 02:14:33.353718 14508 leader_election.cc:302] T 4ff002252c4d46d2af2a4956acd90b3f P 27f29b79cf9b42f4a5b9838965bbb72d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 27f29b79cf9b42f4a5b9838965bbb72d; no voters:
I20240503 02:14:33.353956 14508 leader_election.cc:288] T 4ff002252c4d46d2af2a4956acd90b3f P 27f29b79cf9b42f4a5b9838965bbb72d [CANDIDATE]: Term 1 election: Requested vote from peers
I20240503 02:14:33.354084 14512 raft_consensus.cc:2781] T 4ff002252c4d46d2af2a4956acd90b3f P 27f29b79cf9b42f4a5b9838965bbb72d [term 1 FOLLOWER]: Leader election won for term 1
I20240503 02:14:33.354285 14512 raft_consensus.cc:686] T 4ff002252c4d46d2af2a4956acd90b3f P 27f29b79cf9b42f4a5b9838965bbb72d [term 1 LEADER]: Becoming Leader. State: Replica: 27f29b79cf9b42f4a5b9838965bbb72d, State: Running, Role: LEADER
I20240503 02:14:33.354461 14508 ts_tablet_manager.cc:1398] T 4ff002252c4d46d2af2a4956acd90b3f P 27f29b79cf9b42f4a5b9838965bbb72d: Time spent starting tablet: real 0.005s user 0.002s sys 0.000s
I20240503 02:14:33.354537 14512 consensus_queue.cc:237] T 4ff002252c4d46d2af2a4956acd90b3f P 27f29b79cf9b42f4a5b9838965bbb72d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27f29b79cf9b42f4a5b9838965bbb72d" member_type: VOTER last_known_addr { host: "127.9.151.129" port: 33313 } }
I20240503 02:14:33.354787 14508 tablet_bootstrap.cc:492] T e50f84673efd4108859e8c1360cad36f P 27f29b79cf9b42f4a5b9838965bbb72d: Bootstrap starting.
I20240503 02:14:33.356735 14508 tablet_bootstrap.cc:654] T e50f84673efd4108859e8c1360cad36f P 27f29b79cf9b42f4a5b9838965bbb72d: Neither blocks nor log segments found. Creating new log.
I20240503 02:14:33.357430 14438 catalog_manager.cc:5484] T 4ff002252c4d46d2af2a4956acd90b3f P 27f29b79cf9b42f4a5b9838965bbb72d reported cstate change: term changed from 0 to 1, leader changed from <none> to 27f29b79cf9b42f4a5b9838965bbb72d (127.9.151.129). New cstate: current_term: 1 leader_uuid: "27f29b79cf9b42f4a5b9838965bbb72d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27f29b79cf9b42f4a5b9838965bbb72d" member_type: VOTER last_known_addr { host: "127.9.151.129" port: 33313 } health_report { overall_health: HEALTHY } } }
I20240503 02:14:33.358611 14508 tablet_bootstrap.cc:492] T e50f84673efd4108859e8c1360cad36f P 27f29b79cf9b42f4a5b9838965bbb72d: No bootstrap required, opened a new log
I20240503 02:14:33.358798 14508 ts_tablet_manager.cc:1367] T e50f84673efd4108859e8c1360cad36f P 27f29b79cf9b42f4a5b9838965bbb72d: Time spent bootstrapping tablet: real 0.004s user 0.004s sys 0.000s
I20240503 02:14:33.359606 14508 raft_consensus.cc:348] T e50f84673efd4108859e8c1360cad36f P 27f29b79cf9b42f4a5b9838965bbb72d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27f29b79cf9b42f4a5b9838965bbb72d" member_type: VOTER last_known_addr { host: "127.9.151.129" port: 33313 } }
I20240503 02:14:33.359786 14508 raft_consensus.cc:374] T e50f84673efd4108859e8c1360cad36f P 27f29b79cf9b42f4a5b9838965bbb72d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:14:33.359887 14508 raft_consensus.cc:729] T e50f84673efd4108859e8c1360cad36f P 27f29b79cf9b42f4a5b9838965bbb72d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 27f29b79cf9b42f4a5b9838965bbb72d, State: Initialized, Role: FOLLOWER
I20240503 02:14:33.360142 14508 consensus_queue.cc:260] T e50f84673efd4108859e8c1360cad36f P 27f29b79cf9b42f4a5b9838965bbb72d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27f29b79cf9b42f4a5b9838965bbb72d" member_type: VOTER last_known_addr { host: "127.9.151.129" port: 33313 } }
I20240503 02:14:33.360356 14508 raft_consensus.cc:388] T e50f84673efd4108859e8c1360cad36f P 27f29b79cf9b42f4a5b9838965bbb72d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20240503 02:14:33.360437 14508 raft_consensus.cc:482] T e50f84673efd4108859e8c1360cad36f P 27f29b79cf9b42f4a5b9838965bbb72d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20240503 02:14:33.360545 14508 raft_consensus.cc:3037] T e50f84673efd4108859e8c1360cad36f P 27f29b79cf9b42f4a5b9838965bbb72d [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:14:33.362601 14508 raft_consensus.cc:504] T e50f84673efd4108859e8c1360cad36f P 27f29b79cf9b42f4a5b9838965bbb72d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27f29b79cf9b42f4a5b9838965bbb72d" member_type: VOTER last_known_addr { host: "127.9.151.129" port: 33313 } }
I20240503 02:14:33.362898 14508 leader_election.cc:302] T e50f84673efd4108859e8c1360cad36f P 27f29b79cf9b42f4a5b9838965bbb72d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 27f29b79cf9b42f4a5b9838965bbb72d; no voters:
I20240503 02:14:33.363266 14508 leader_election.cc:288] T e50f84673efd4108859e8c1360cad36f P 27f29b79cf9b42f4a5b9838965bbb72d [CANDIDATE]: Term 1 election: Requested vote from peers
I20240503 02:14:33.363579 14512 raft_consensus.cc:2781] T e50f84673efd4108859e8c1360cad36f P 27f29b79cf9b42f4a5b9838965bbb72d [term 1 FOLLOWER]: Leader election won for term 1
I20240503 02:14:33.363828 14512 raft_consensus.cc:686] T e50f84673efd4108859e8c1360cad36f P 27f29b79cf9b42f4a5b9838965bbb72d [term 1 LEADER]: Becoming Leader. State: Replica: 27f29b79cf9b42f4a5b9838965bbb72d, State: Running, Role: LEADER
I20240503 02:14:33.364172 14508 ts_tablet_manager.cc:1398] T e50f84673efd4108859e8c1360cad36f P 27f29b79cf9b42f4a5b9838965bbb72d: Time spent starting tablet: real 0.005s user 0.004s sys 0.000s
I20240503 02:14:33.364199 14512 consensus_queue.cc:237] T e50f84673efd4108859e8c1360cad36f P 27f29b79cf9b42f4a5b9838965bbb72d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27f29b79cf9b42f4a5b9838965bbb72d" member_type: VOTER last_known_addr { host: "127.9.151.129" port: 33313 } }
I20240503 02:14:33.364651 14508 tablet_bootstrap.cc:492] T f884dde21ea246a792634883aed3cb82 P 27f29b79cf9b42f4a5b9838965bbb72d: Bootstrap starting.
I20240503 02:14:33.366637 14508 tablet_bootstrap.cc:654] T f884dde21ea246a792634883aed3cb82 P 27f29b79cf9b42f4a5b9838965bbb72d: Neither blocks nor log segments found. Creating new log.
I20240503 02:14:33.368093 14438 catalog_manager.cc:5484] T e50f84673efd4108859e8c1360cad36f P 27f29b79cf9b42f4a5b9838965bbb72d reported cstate change: term changed from 0 to 1, leader changed from <none> to 27f29b79cf9b42f4a5b9838965bbb72d (127.9.151.129). New cstate: current_term: 1 leader_uuid: "27f29b79cf9b42f4a5b9838965bbb72d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27f29b79cf9b42f4a5b9838965bbb72d" member_type: VOTER last_known_addr { host: "127.9.151.129" port: 33313 } health_report { overall_health: HEALTHY } } }
I20240503 02:14:33.368631 14508 tablet_bootstrap.cc:492] T f884dde21ea246a792634883aed3cb82 P 27f29b79cf9b42f4a5b9838965bbb72d: No bootstrap required, opened a new log
I20240503 02:14:33.368827 14508 ts_tablet_manager.cc:1367] T f884dde21ea246a792634883aed3cb82 P 27f29b79cf9b42f4a5b9838965bbb72d: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.003s
I20240503 02:14:33.369590 14508 raft_consensus.cc:348] T f884dde21ea246a792634883aed3cb82 P 27f29b79cf9b42f4a5b9838965bbb72d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27f29b79cf9b42f4a5b9838965bbb72d" member_type: VOTER last_known_addr { host: "127.9.151.129" port: 33313 } }
I20240503 02:14:33.369832 14508 raft_consensus.cc:374] T f884dde21ea246a792634883aed3cb82 P 27f29b79cf9b42f4a5b9838965bbb72d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:14:33.369906 14508 raft_consensus.cc:729] T f884dde21ea246a792634883aed3cb82 P 27f29b79cf9b42f4a5b9838965bbb72d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 27f29b79cf9b42f4a5b9838965bbb72d, State: Initialized, Role: FOLLOWER
I20240503 02:14:33.370179 14508 consensus_queue.cc:260] T f884dde21ea246a792634883aed3cb82 P 27f29b79cf9b42f4a5b9838965bbb72d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27f29b79cf9b42f4a5b9838965bbb72d" member_type: VOTER last_known_addr { host: "127.9.151.129" port: 33313 } }
I20240503 02:14:33.370313 14508 raft_consensus.cc:388] T f884dde21ea246a792634883aed3cb82 P 27f29b79cf9b42f4a5b9838965bbb72d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20240503 02:14:33.370399 14508 raft_consensus.cc:482] T f884dde21ea246a792634883aed3cb82 P 27f29b79cf9b42f4a5b9838965bbb72d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20240503 02:14:33.370497 14508 raft_consensus.cc:3037] T f884dde21ea246a792634883aed3cb82 P 27f29b79cf9b42f4a5b9838965bbb72d [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:14:33.372666 14508 raft_consensus.cc:504] T f884dde21ea246a792634883aed3cb82 P 27f29b79cf9b42f4a5b9838965bbb72d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27f29b79cf9b42f4a5b9838965bbb72d" member_type: VOTER last_known_addr { host: "127.9.151.129" port: 33313 } }
I20240503 02:14:33.372997 14508 leader_election.cc:302] T f884dde21ea246a792634883aed3cb82 P 27f29b79cf9b42f4a5b9838965bbb72d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 27f29b79cf9b42f4a5b9838965bbb72d; no voters:
I20240503 02:14:33.373262 14512 raft_consensus.cc:2781] T f884dde21ea246a792634883aed3cb82 P 27f29b79cf9b42f4a5b9838965bbb72d [term 1 FOLLOWER]: Leader election won for term 1
I20240503 02:14:33.373430 14512 raft_consensus.cc:686] T f884dde21ea246a792634883aed3cb82 P 27f29b79cf9b42f4a5b9838965bbb72d [term 1 LEADER]: Becoming Leader. State: Replica: 27f29b79cf9b42f4a5b9838965bbb72d, State: Running, Role: LEADER
I20240503 02:14:33.373646 14512 consensus_queue.cc:237] T f884dde21ea246a792634883aed3cb82 P 27f29b79cf9b42f4a5b9838965bbb72d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27f29b79cf9b42f4a5b9838965bbb72d" member_type: VOTER last_known_addr { host: "127.9.151.129" port: 33313 } }
I20240503 02:14:33.374383 14508 leader_election.cc:288] T f884dde21ea246a792634883aed3cb82 P 27f29b79cf9b42f4a5b9838965bbb72d [CANDIDATE]: Term 1 election: Requested vote from peers
I20240503 02:14:33.374893 14508 ts_tablet_manager.cc:1398] T f884dde21ea246a792634883aed3cb82 P 27f29b79cf9b42f4a5b9838965bbb72d: Time spent starting tablet: real 0.006s user 0.001s sys 0.004s
I20240503 02:14:33.375198 14508 tablet_bootstrap.cc:492] T b26fbd5ae3384b6cb444331bd8e75aef P 27f29b79cf9b42f4a5b9838965bbb72d: Bootstrap starting.
I20240503 02:14:33.376732 14508 tablet_bootstrap.cc:654] T b26fbd5ae3384b6cb444331bd8e75aef P 27f29b79cf9b42f4a5b9838965bbb72d: Neither blocks nor log segments found. Creating new log.
I20240503 02:14:33.377092 14438 catalog_manager.cc:5484] T f884dde21ea246a792634883aed3cb82 P 27f29b79cf9b42f4a5b9838965bbb72d reported cstate change: term changed from 0 to 1, leader changed from <none> to 27f29b79cf9b42f4a5b9838965bbb72d (127.9.151.129). New cstate: current_term: 1 leader_uuid: "27f29b79cf9b42f4a5b9838965bbb72d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27f29b79cf9b42f4a5b9838965bbb72d" member_type: VOTER last_known_addr { host: "127.9.151.129" port: 33313 } health_report { overall_health: HEALTHY } } }
I20240503 02:14:33.378499 14508 tablet_bootstrap.cc:492] T b26fbd5ae3384b6cb444331bd8e75aef P 27f29b79cf9b42f4a5b9838965bbb72d: No bootstrap required, opened a new log
I20240503 02:14:33.378645 14508 ts_tablet_manager.cc:1367] T b26fbd5ae3384b6cb444331bd8e75aef P 27f29b79cf9b42f4a5b9838965bbb72d: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20240503 02:14:33.379406 14508 raft_consensus.cc:348] T b26fbd5ae3384b6cb444331bd8e75aef P 27f29b79cf9b42f4a5b9838965bbb72d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27f29b79cf9b42f4a5b9838965bbb72d" member_type: VOTER last_known_addr { host: "127.9.151.129" port: 33313 } }
I20240503 02:14:33.379566 14508 raft_consensus.cc:374] T b26fbd5ae3384b6cb444331bd8e75aef P 27f29b79cf9b42f4a5b9838965bbb72d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240503 02:14:33.379678 14508 raft_consensus.cc:729] T b26fbd5ae3384b6cb444331bd8e75aef P 27f29b79cf9b42f4a5b9838965bbb72d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 27f29b79cf9b42f4a5b9838965bbb72d, State: Initialized, Role: FOLLOWER
I20240503 02:14:33.379918 14508 consensus_queue.cc:260] T b26fbd5ae3384b6cb444331bd8e75aef P 27f29b79cf9b42f4a5b9838965bbb72d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27f29b79cf9b42f4a5b9838965bbb72d" member_type: VOTER last_known_addr { host: "127.9.151.129" port: 33313 } }
I20240503 02:14:33.380081 14508 raft_consensus.cc:388] T b26fbd5ae3384b6cb444331bd8e75aef P 27f29b79cf9b42f4a5b9838965bbb72d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20240503 02:14:33.380156 14508 raft_consensus.cc:482] T b26fbd5ae3384b6cb444331bd8e75aef P 27f29b79cf9b42f4a5b9838965bbb72d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20240503 02:14:33.380278 14508 raft_consensus.cc:3037] T b26fbd5ae3384b6cb444331bd8e75aef P 27f29b79cf9b42f4a5b9838965bbb72d [term 0 FOLLOWER]: Advancing to term 1
I20240503 02:14:33.382179 14508 raft_consensus.cc:504] T b26fbd5ae3384b6cb444331bd8e75aef P 27f29b79cf9b42f4a5b9838965bbb72d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27f29b79cf9b42f4a5b9838965bbb72d" member_type: VOTER last_known_addr { host: "127.9.151.129" port: 33313 } }
I20240503 02:14:33.382462 14508 leader_election.cc:302] T b26fbd5ae3384b6cb444331bd8e75aef P 27f29b79cf9b42f4a5b9838965bbb72d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 27f29b79cf9b42f4a5b9838965bbb72d; no voters:
I20240503 02:14:33.382647 14508 leader_election.cc:288] T b26fbd5ae3384b6cb444331bd8e75aef P 27f29b79cf9b42f4a5b9838965bbb72d [CANDIDATE]: Term 1 election: Requested vote from peers
I20240503 02:14:33.382853 14512 raft_consensus.cc:2781] T b26fbd5ae3384b6cb444331bd8e75aef P 27f29b79cf9b42f4a5b9838965bbb72d [term 1 FOLLOWER]: Leader election won for term 1
I20240503 02:14:33.383033 14512 raft_consensus.cc:686] T b26fbd5ae3384b6cb444331bd8e75aef P 27f29b79cf9b42f4a5b9838965bbb72d [term 1 LEADER]: Becoming Leader. State: Replica: 27f29b79cf9b42f4a5b9838965bbb72d, State: Running, Role: LEADER
I20240503 02:14:33.383250 14508 ts_tablet_manager.cc:1398] T b26fbd5ae3384b6cb444331bd8e75aef P 27f29b79cf9b42f4a5b9838965bbb72d: Time spent starting tablet: real 0.004s user 0.004s sys 0.000s
I20240503 02:14:33.383322 14512 consensus_queue.cc:237] T b26fbd5ae3384b6cb444331bd8e75aef P 27f29b79cf9b42f4a5b9838965bbb72d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27f29b79cf9b42f4a5b9838965bbb72d" member_type: VOTER last_known_addr { host: "127.9.151.129" port: 33313 } }
I20240503 02:14:33.386121 14438 catalog_manager.cc:5484] T b26fbd5ae3384b6cb444331bd8e75aef P 27f29b79cf9b42f4a5b9838965bbb72d reported cstate change: term changed from 0 to 1, leader changed from <none> to 27f29b79cf9b42f4a5b9838965bbb72d (127.9.151.129). New cstate: current_term: 1 leader_uuid: "27f29b79cf9b42f4a5b9838965bbb72d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27f29b79cf9b42f4a5b9838965bbb72d" member_type: VOTER last_known_addr { host: "127.9.151.129" port: 33313 } health_report { overall_health: HEALTHY } } }
W20240503 02:14:33.759207 14534 log.cc:924] Time spent T fc8f5043623a48418283e5ecd50d58e2 P 27f29b79cf9b42f4a5b9838965bbb72d: Append to log took a long time: real 0.075s user 0.000s sys 0.000s
W20240503 02:14:33.785471 14532 log.cc:924] Time spent T b0df8ae119f948eaa5461faa42007713 P 27f29b79cf9b42f4a5b9838965bbb72d: Append to log took a long time: real 0.087s user 0.000s sys 0.000s
I20240503 02:14:37.476161 9822 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknrxqHk/build/asan/bin/kudu
/tmp/dist-test-tasknrxqHk/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknrxqHk/test-tmp/tablet_copy_client_session-itest.0.TabletCopyClientSessionITest.TestTabletCopyWithBusySource.1714702442467385-9822-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasknrxqHk/test-tmp/tablet_copy_client_session-itest.0.TabletCopyClientSessionITest.TestTabletCopyWithBusySource.1714702442467385-9822-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknrxqHk/test-tmp/tablet_copy_client_session-itest.0.TabletCopyClientSessionITest.TestTabletCopyWithBusySource.1714702442467385-9822-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasknrxqHk/test-tmp/tablet_copy_client_session-itest.0.TabletCopyClientSessionITest.TestTabletCopyWithBusySource.1714702442467385-9822-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.151.130:41429
--local_ip_for_outbound_sockets=127.9.151.130
--tserver_master_addrs=127.9.151.190:38767
--webserver_port=40243
--webserver_interface=127.9.151.130
--builtin_ntp_servers=127.9.151.148:43735
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--num_tablets_to_copy_simultaneously=20 with env {}
W20240503 02:14:37.846578 14605 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240503 02:14:37.846957 14605 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240503 02:14:37.847113 14605 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240503 02:14:37.855635 14605 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240503 02:14:37.855912 14605 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.151.130
I20240503 02:14:37.865870 14605 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.151.148:43735
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknrxqHk/test-tmp/tablet_copy_client_session-itest.0.TabletCopyClientSessionITest.TestTabletCopyWithBusySource.1714702442467385-9822-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasknrxqHk/test-tmp/tablet_copy_client_session-itest.0.TabletCopyClientSessionITest.TestTabletCopyWithBusySource.1714702442467385-9822-0/minicluster-data/ts-1/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.151.130:41429
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknrxqHk/test-tmp/tablet_copy_client_session-itest.0.TabletCopyClientSessionITest.TestTabletCopyWithBusySource.1714702442467385-9822-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.151.130
--webserver_port=40243
--tserver_master_addrs=127.9.151.190:38767
--num_tablets_to_copy_simultaneously=20
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.151.130
--log_dir=/tmp/dist-test-tasknrxqHk/test-tmp/tablet_copy_client_session-itest.0.TabletCopyClientSessionITest.TestTabletCopyWithBusySource.1714702442467385-9822-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c39e72a98e124e5a12080bac42105b996594d806
build type FASTDEBUG
built by None at 03 May 2024 01:43:22 UTC on efd1337a2aa3
build id 99
ASAN enabled
I20240503 02:14:37.867846 14605 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240503 02:14:37.870435 14605 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240503 02:14:37.886161 14613 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240503 02:14:37.886938 14611 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240503 02:14:37.887291 14610 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240503 02:14:37.887921 14605 server_base.cc:1034] running on GCE node
I20240503 02:14:37.888645 14605 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240503 02:14:37.889834 14605 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240503 02:14:37.891117 14605 hybrid_clock.cc:648] HybridClock initialized: now 1714702477891022 us; error 63 us; skew 500 ppm
I20240503 02:14:37.891471 14605 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20240503 02:14:37.894165 14605 webserver.cc:460] Webserver started at http://127.9.151.130:40243/ using document root <none> and password file <none>
I20240503 02:14:37.895098 14605 fs_manager.cc:359] Metadata directory not provided
I20240503 02:14:37.895251 14605 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240503 02:14:37.902141 14605 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.006s sys 0.000s
I20240503 02:14:37.905786 14619 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240503 02:14:37.908005 14605 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.001s sys 0.002s
I20240503 02:14:37.908300 14605 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknrxqHk/test-tmp/tablet_copy_client_session-itest.0.TabletCopyClientSessionITest.TestTabletCopyWithBusySource.1714702442467385-9822-0/minicluster-data/ts-1/data,/tmp/dist-test-tasknrxqHk/test-tmp/tablet_copy_client_session-itest.0.TabletCopyClientSessionITest.TestTabletCopyWithBusySource.1714702442467385-9822-0/minicluster-data/ts-1/wal
uuid: "5f665335495c43dcbc53e072f6b1f44f"
format_stamp: "Formatted at 2024-05-03 02:14:31 on dist-test-slave-twlh"
I20240503 02:14:37.909442 14605 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknrxqHk/test-tmp/tablet_copy_client_session-itest.0.TabletCopyClientSessionITest.TestTabletCopyWithBusySource.1714702442467385-9822-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasknrxqHk/test-tmp/tablet_copy_client_session-itest.0.TabletCopyClientSessionITest.TestTabletCopyWithBusySource.1714702442467385-9822-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasknrxqHk/test-tmp/tablet_copy_client_session-itest.0.TabletCopyClientSessionITest.TestTabletCopyWithBusySource.1714702442467385-9822-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240503 02:14:37.928603 14605 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240503 02:14:37.929574 14605 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240503 02:14:37.929847 14605 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240503 02:14:37.930964 14605 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240503 02:14:37.932683 14605 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240503 02:14:37.932775 14605 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20240503 02:14:37.932902 14605 ts_tablet_manager.cc:590] Registered 0 tablets
I20240503 02:14:37.932978 14605 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20240503 02:14:37.982547 14605 rpc_server.cc:307] RPC server started. Bound to: 127.9.151.130:41429
I20240503 02:14:37.982604 14731 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.151.130:41429 every 8 connection(s)
I20240503 02:14:37.984422 14605 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknrxqHk/test-tmp/tablet_copy_client_session-itest.0.TabletCopyClientSessionITest.TestTabletCopyWithBusySource.1714702442467385-9822-0/minicluster-data/ts-1/data/info.pb
I20240503 02:14:37.994292 9822 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknrxqHk/build/asan/bin/kudu as pid 14605
I20240503 02:14:37.996466 9822 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasknrxqHk/build/asan/bin/kudu pid 14135
I20240503 02:14:37.999048 14257 generic_service.cc:196] Checking for leaks (request via RPC)
I20240503 02:14:38.003001 14732 heartbeater.cc:347] Connected to a master server at 127.9.151.190:38767
I20240503 02:14:38.003443 14732 heartbeater.cc:464] Registering TS with master...
I20240503 02:14:38.004561 14732 heartbeater.cc:511] Master 127.9.151.190:38767 requested a full tablet report, sending...
I20240503 02:14:38.006984 14433 ts_manager.cc:194] Registered new tserver with Master: 5f665335495c43dcbc53e072f6b1f44f (127.9.151.130:41429)
I20240503 02:14:38.008982 14433 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.9.151.130:42135
I20240503 02:14:38.235740 9822 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknrxqHk/build/asan/bin/kudu with pid 14135
I20240503 02:14:38.276314 9822 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknrxqHk/build/asan/bin/kudu
/tmp/dist-test-tasknrxqHk/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknrxqHk/test-tmp/tablet_copy_client_session-itest.0.TabletCopyClientSessionITest.TestTabletCopyWithBusySource.1714702442467385-9822-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasknrxqHk/test-tmp/tablet_copy_client_session-itest.0.TabletCopyClientSessionITest.TestTabletCopyWithBusySource.1714702442467385-9822-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknrxqHk/test-tmp/tablet_copy_client_session-itest.0.TabletCopyClientSessionITest.TestTabletCopyWithBusySource.1714702442467385-9822-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasknrxqHk/test-tmp/tablet_copy_client_session-itest.0.TabletCopyClientSessionITest.TestTabletCopyWithBusySource.1714702442467385-9822-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.151.129:33313
--local_ip_for_outbound_sockets=127.9.151.129
--tserver_master_addrs=127.9.151.190:38767
--webserver_port=43777
--webserver_interface=127.9.151.129
--builtin_ntp_servers=127.9.151.148:43735
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--num_tablets_to_copy_simultaneously=20
--rpc_service_queue_length=1
--rpc_num_service_threads=1 with env {}
W20240503 02:14:38.650456 14737 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20240503 02:14:38.651019 14737 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20240503 02:14:38.651237 14737 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20240503 02:14:38.660146 14737 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20240503 02:14:38.660434 14737 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.151.129
I20240503 02:14:38.671061 14737 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.151.148:43735
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknrxqHk/test-tmp/tablet_copy_client_session-itest.0.TabletCopyClientSessionITest.TestTabletCopyWithBusySource.1714702442467385-9822-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasknrxqHk/test-tmp/tablet_copy_client_session-itest.0.TabletCopyClientSessionITest.TestTabletCopyWithBusySource.1714702442467385-9822-0/minicluster-data/ts-0/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.151.129:33313
--rpc_num_service_threads=1
--rpc_server_allow_ephemeral_ports=true
--rpc_service_queue_length=1
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknrxqHk/test-tmp/tablet_copy_client_session-itest.0.TabletCopyClientSessionITest.TestTabletCopyWithBusySource.1714702442467385-9822-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.151.129
--webserver_port=43777
--tserver_master_addrs=127.9.151.190:38767
--num_tablets_to_copy_simultaneously=20
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.151.129
--log_dir=/tmp/dist-test-tasknrxqHk/test-tmp/tablet_copy_client_session-itest.0.TabletCopyClientSessionITest.TestTabletCopyWithBusySource.1714702442467385-9822-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c39e72a98e124e5a12080bac42105b996594d806
build type FASTDEBUG
built by None at 03 May 2024 01:43:22 UTC on efd1337a2aa3
build id 99
ASAN enabled
I20240503 02:14:38.673038 14737 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240503 02:14:38.675099 14737 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240503 02:14:38.689078 14742 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240503 02:14:38.689739 14743 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240503 02:14:38.690330 14745 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240503 02:14:38.691061 14737 server_base.cc:1034] running on GCE node
I20240503 02:14:38.691877 14737 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20240503 02:14:38.692947 14737 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20240503 02:14:38.694281 14737 hybrid_clock.cc:648] HybridClock initialized: now 1714702478694169 us; error 47 us; skew 500 ppm
I20240503 02:14:38.694980 14737 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20240503 02:14:38.697991 14737 webserver.cc:460] Webserver started at http://127.9.151.129:43777/ using document root <none> and password file <none>
I20240503 02:14:38.698915 14737 fs_manager.cc:359] Metadata directory not provided
I20240503 02:14:38.699074 14737 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240503 02:14:38.705883 14737 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.005s sys 0.000s
I20240503 02:14:38.712582 14751 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20240503 02:14:38.714221 14737 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.000s sys 0.005s
I20240503 02:14:38.714459 14737 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknrxqHk/test-tmp/tablet_copy_client_session-itest.0.TabletCopyClientSessionITest.TestTabletCopyWithBusySource.1714702442467385-9822-0/minicluster-data/ts-0/data,/tmp/dist-test-tasknrxqHk/test-tmp/tablet_copy_client_session-itest.0.TabletCopyClientSessionITest.TestTabletCopyWithBusySource.1714702442467385-9822-0/minicluster-data/ts-0/wal
uuid: "27f29b79cf9b42f4a5b9838965bbb72d"
format_stamp: "Formatted at 2024-05-03 02:14:30 on dist-test-slave-twlh"
I20240503 02:14:38.715632 14737 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknrxqHk/test-tmp/tablet_copy_client_session-itest.0.TabletCopyClientSessionITest.TestTabletCopyWithBusySource.1714702442467385-9822-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasknrxqHk/test-tmp/tablet_copy_client_session-itest.0.TabletCopyClientSessionITest.TestTabletCopyWithBusySource.1714702442467385-9822-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasknrxqHk/test-tmp/tablet_copy_client_session-itest.0.TabletCopyClientSessionITest.TestTabletCopyWithBusySource.1714702442467385-9822-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240503 02:14:38.730422 14737 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20240503 02:14:38.731643 14737 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240503 02:14:38.732019 14737 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240503 02:14:38.733599 14737 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240503 02:14:38.736619 14758 ts_tablet_manager.cc:522] Loading tablet metadata (0/20 complete)
I20240503 02:14:38.782466 14737 ts_tablet_manager.cc:559] Loaded tablet metadata (20 total tablets, 20 live tablets)
I20240503 02:14:38.782624 14737 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.047s user 0.003s sys 0.000s
I20240503 02:14:38.782769 14737 ts_tablet_manager.cc:574] Registering tablets (0/20 complete)
I20240503 02:14:38.786401 14758 tablet_bootstrap.cc:492] T 97c56e26ef4f47f4acf50d0b8d8116dc P 27f29b79cf9b42f4a5b9838965bbb72d: Bootstrap starting.
I20240503 02:14:38.830140 14737 ts_tablet_manager.cc:590] Registered 20 tablets
I20240503 02:14:38.830258 14737 ts_tablet_manager.cc:569] Time spent register tablets: real 0.048s user 0.041s sys 0.003s
I20240503 02:14:38.839623 14758 log.cc:824] T 97c56e26ef4f47f4acf50d0b8d8116dc P 27f29b79cf9b42f4a5b9838965bbb72d: Log is configured to *not* fsync() on all Append() calls
I20240503 02:14:38.839704 14737 rpc_server.cc:307] RPC server started. Bound to: 127.9.151.129:33313
I20240503 02:14:38.839746 14769 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.151.129:33313 every 8 connection(s)
I20240503 02:14:38.841136 14737 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknrxqHk/test-tmp/tablet_copy_client_session-itest.0.TabletCopyClientSessionITest.TestTabletCopyWithBusySource.1714702442467385-9822-0/minicluster-data/ts-0/data/info.pb
I20240503 02:14:38.845536 9822 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknrxqHk/build/asan/bin/kudu as pid 14737
I20240503 02:14:38.856048 14771 heartbeater.cc:347] Connected to a master server at 127.9.151.190:38767
I20240503 02:14:38.857620 14771 heartbeater.cc:464] Registering TS with master...
I20240503 02:14:38.858974 14771 heartbeater.cc:511] Master 127.9.151.190:38767 requested a full tablet report, sending...
I20240503 02:14:38.862869 14433 ts_manager.cc:194] Re-registered known tserver with Master: 27f29b79cf9b42f4a5b9838965bbb72d (127.9.151.129:33313)
I20240503 02:14:38.868121 14433 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.9.151.129:39187
I20240503 02:14:39.008191 14758 tablet_bootstrap.cc:492] T 97c56e26ef4f47f4acf50d0b8d8116dc P 27f29b79cf9b42f4a5b9838965bbb72d: Bootstrap replayed 1/1 log segments. Stats: ops{read=188 overwritten=0 applied=188 ignored=0} inserts{seen=512 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240503 02:14:39.008865 14758 tablet_bootstrap.cc:492] T 97c56e26ef4f47f4acf50d0b8d8116dc P 27f29b79cf9b42f4a5b9838965bbb72d: Bootstrap complete.
I20240503 02:14:39.010154 14758 ts_tablet_manager.cc:1367] T 97c56e26ef4f47f4acf50d0b8d8116dc P 27f29b79cf9b42f4a5b9838965bbb72d: Time spent bootstrapping tablet: real 0.224s user 0.202s sys 0.016s
I20240503 02:14:39.013840 14732 heartbeater.cc:503] Master 127.9.151.190:38767 was elected leader, sending a full tablet report...
I20240503 02:14:39.016155 14758 raft_consensus.cc:348] T 97c56e26ef4f47f4acf50d0b8d8116dc P 27f29b79cf9b42f4a5b9838965bbb72d [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27f29b79cf9b42f4a5b9838965bbb72d" member_type: VOTER last_known_addr { host: "127.9.151.129" port: 33313 } }
I20240503 02:14:39.017000 14758 raft_consensus.cc:729] T 97c56e26ef4f47f4acf50d0b8d8116dc P 27f29b79cf9b42f4a5b9838965bbb72d [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 27f29b79cf9b42f4a5b9838965bbb72d, State: Initialized, Role: FOLLOWER
I20240503 02:14:39.017617 14758 consensus_queue.cc:260] T 97c56e26ef4f47f4acf50d0b8d8116dc P 27f29b79cf9b42f4a5b9838965bbb72d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 188, Last appended: 1.188, Last appended by leader: 188, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27f29b79cf9b42f4a5b9838965bbb72d" member_type: VOTER last_known_addr { host: "127.9.151.129" port: 33313 } }
I20240503 02:14:39.017814 14758 raft_consensus.cc:388] T 97c56e26ef4f47f4acf50d0b8d8116dc P 27f29b79cf9b42f4a5b9838965bbb72d [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20240503 02:14:39.017906 14758 raft_consensus.cc:482] T 97c56e26ef4f47f4acf50d0b8d8116dc P 27f29b79cf9b42f4a5b9838965bbb72d [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20240503 02:14:39.018045 14758 raft_consensus.cc:3037] T 97c56e26ef4f47f4acf50d0b8d8116dc P 27f29b79cf9b42f4a5b9838965bbb72d [term 1 FOLLOWER]: Advancing to term 2
I20240503 02:14:39.021888 14758 raft_consensus.cc:504] T 97c56e26ef4f47f4acf50d0b8d8116dc P 27f29b79cf9b42f4a5b9838965bbb72d [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27f29b79cf9b42f4a5b9838965bbb72d" member_type: VOTER last_known_addr { host: "127.9.151.129" port: 33313 } }
I20240503 02:14:39.022439 14758 leader_election.cc:302] T 97c56e26ef4f47f4acf50d0b8d8116dc P 27f29b79cf9b42f4a5b9838965bbb72d [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 27f29b79cf9b42f4a5b9838965bbb72d; no voters:
I20240503 02:14:39.023066 14758 leader_election.cc:288] T 97c56e26ef4f47f4acf50d0b8d8116dc P 27f29b79cf9b42f4a5b9838965bbb72d [CANDIDATE]: Term 2 election: Requested vote from peers
I20240503 02:14:39.023245 14777 raft_consensus.cc:2781] T 97c56e26ef4f47f4acf50d0b8d8116dc P 27f29b79cf9b42f4a5b9838965bbb72d [term 2 FOLLOWER]: Leader election won for term 2
I20240503 02:14:39.024358 14771 heartbeater.cc:503] Master 127.9.151.190:38767 was elected leader, sending a full tablet report...
I20240503 02:14:39.024453 14777 raft_consensus.cc:686] T 97c56e26ef4f47f4acf50d0b8d8116dc P 27f29b79cf9b42f4a5b9838965bbb72d [term 2 LEADER]: Becoming Leader. State: Replica: 27f29b79cf9b42f4a5b9838965bbb72d, State: Running, Role: LEADER
I20240503 02:14:39.024817 14758 ts_tablet_manager.cc:1398] T 97c56e26ef4f47f4acf50d0b8d8116dc P 27f29b79cf9b42f4a5b9838965bbb72d: Time spent starting tablet: real 0.015s user 0.017s sys 0.000s
I20240503 02:14:39.024989 14777 consensus_queue.cc:237] T 97c56e26ef4f47f4acf50d0b8d8116dc P 27f29b79cf9b42f4a5b9838965bbb72d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 188, Committed index: 188, Last appended: 1.188, Last appended by leader: 188, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27f29b79cf9b42f4a5b9838965bbb72d" member_type: VOTER last_known_addr { host: "127.9.151.129" port: 33313 } }
I20240503 02:14:39.025246 14758 tablet_bootstrap.cc:492] T fa681a37b30c4a0b879d3f3b184405fc P 27f29b79cf9b42f4a5b9838965bbb72d: Bootstrap starting.
I20240503 02:14:39.031213 14433 catalog_manager.cc:5484] T 97c56e26ef4f47f4acf50d0b8d8116dc P 27f29b79cf9b42f4a5b9838965bbb72d reported cstate change: term changed from 1 to 2. New cstate: current_term: 2 leader_uuid: "27f29b79cf9b42f4a5b9838965bbb72d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27f29b79cf9b42f4a5b9838965bbb72d" member_type: VOTER last_known_addr { host: "127.9.151.129" port: 33313 } health_report { overall_health: HEALTHY } } }
I20240503 02:14:39.223598 14758 tablet_bootstrap.cc:492] T fa681a37b30c4a0b879d3f3b184405fc P 27f29b79cf9b42f4a5b9838965bbb72d: Bootstrap replayed 1/1 log segments. Stats: ops{read=192 overwritten=0 applied=192 ignored=0} inserts{seen=528 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240503 02:14:39.224109 14758 tablet_bootstrap.cc:492] T fa681a37b30c4a0b879d3f3b184405fc P 27f29b79cf9b42f4a5b9838965bbb72d: Bootstrap complete.
I20240503 02:14:39.225100 14758 ts_tablet_manager.cc:1367] T fa681a37b30c4a0b879d3f3b184405fc P 27f29b79cf9b42f4a5b9838965bbb72d: Time spent bootstrapping tablet: real 0.200s user 0.181s sys 0.016s
I20240503 02:14:39.226105 14758 raft_consensus.cc:348] T fa681a37b30c4a0b879d3f3b184405fc P 27f29b79cf9b42f4a5b9838965bbb72d [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27f29b79cf9b42f4a5b9838965bbb72d" member_type: VOTER last_known_addr { host: "127.9.151.129" port: 33313 } }
I20240503 02:14:39.226285 14758 raft_consensus.cc:729] T fa681a37b30c4a0b879d3f3b184405fc P 27f29b79cf9b42f4a5b9838965bbb72d [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 27f29b79cf9b42f4a5b9838965bbb72d, State: Initialized, Role: FOLLOWER
I20240503 02:14:39.226591 14758 consensus_queue.cc:260] T fa681a37b30c4a0b879d3f3b184405fc P 27f29b79cf9b42f4a5b9838965bbb72d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 192, Last appended: 1.192, Last appended by leader: 192, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27f29b79cf9b42f4a5b9838965bbb72d" member_type: VOTER last_known_addr { host: "127.9.151.129" port: 33313 } }
I20240503 02:14:39.226817 14758 raft_consensus.cc:388] T fa681a37b30c4a0b879d3f3b184405fc P 27f29b79cf9b42f4a5b9838965bbb72d [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20240503 02:14:39.226948 14758 raft_consensus.cc:482] T fa681a37b30c4a0b879d3f3b184405fc P 27f29b79cf9b42f4a5b9838965bbb72d [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20240503 02:14:39.227052 14758 raft_consensus.cc:3037] T fa681a37b30c4a0b879d3f3b184405fc P 27f29b79cf9b42f4a5b9838965bbb72d [term 1 FOLLOWER]: Advancing to term 2
I20240503 02:14:39.229321 14758 raft_consensus.cc:504] T fa681a37b30c4a0b879d3f3b184405fc P 27f29b79cf9b42f4a5b9838965bbb72d [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27f29b79cf9b42f4a5b9838965bbb72d" member_type: VOTER last_known_addr { host: "127.9.151.129" port: 33313 } }
I20240503 02:14:39.229597 14758 leader_election.cc:302] T fa681a37b30c4a0b879d3f3b184405fc P 27f29b79cf9b42f4a5b9838965bbb72d [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 27f29b79cf9b42f4a5b9838965bbb72d; no voters:
I20240503 02:14:39.229851 14758 leader_election.cc:288] T fa681a37b30c4a0b879d3f3b184405fc P 27f29b79cf9b42f4a5b9838965bbb72d [CANDIDATE]: Term 2 election: Requested vote from peers
I20240503 02:14:39.229939 14777 raft_consensus.cc:2781] T fa681a37b30c4a0b879d3f3b184405fc P 27f29b79cf9b42f4a5b9838965bbb72d [term 2 FOLLOWER]: Leader election won for term 2
I20240503 02:14:39.230136 14777 raft_consensus.cc:686] T fa681a37b30c4a0b879d3f3b184405fc P 27f29b79cf9b42f4a5b9838965bbb72d [term 2 LEADER]: Becoming Leader. State: Replica: 27f29b79cf9b42f4a5b9838965bbb72d, State: Running, Role: LEADER
I20240503 02:14:39.230347 14758 ts_tablet_manager.cc:1398] T fa681a37b30c4a0b879d3f3b184405fc P 27f29b79cf9b42f4a5b9838965bbb72d: Time spent starting tablet: real 0.005s user 0.004s sys 0.000s
I20240503 02:14:39.230412 14777 consensus_queue.cc:237] T fa681a37b30c4a0b879d3f3b184405fc P 27f29b79cf9b42f4a5b9838965bbb72d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 192, Committed index: 192, Last appended: 1.192, Last appended by leader: 192, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27f29b79cf9b42f4a5b9838965bbb72d" member_type: VOTER last_known_addr { host: "127.9.151.129" port: 33313 } }
I20240503 02:14:39.230666 14758 tablet_bootstrap.cc:492] T 9d424d86fde04f2dbdd9434ee5755150 P 27f29b79cf9b42f4a5b9838965bbb72d: Bootstrap starting.
I20240503 02:14:39.237111 14433 catalog_manager.cc:5484] T fa681a37b30c4a0b879d3f3b184405fc P 27f29b79cf9b42f4a5b9838965bbb72d reported cstate change: term changed from 1 to 2. New cstate: current_term: 2 leader_uuid: "27f29b79cf9b42f4a5b9838965bbb72d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27f29b79cf9b42f4a5b9838965bbb72d" member_type: VOTER last_known_addr { host: "127.9.151.129" port: 33313 } health_report { overall_health: HEALTHY } } }
I20240503 02:14:39.415488 14758 tablet_bootstrap.cc:492] T 9d424d86fde04f2dbdd9434ee5755150 P 27f29b79cf9b42f4a5b9838965bbb72d: Bootstrap replayed 1/1 log segments. Stats: ops{read=190 overwritten=0 applied=190 ignored=0} inserts{seen=528 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240503 02:14:39.416033 14758 tablet_bootstrap.cc:492] T 9d424d86fde04f2dbdd9434ee5755150 P 27f29b79cf9b42f4a5b9838965bbb72d: Bootstrap complete.
I20240503 02:14:39.417068 14758 ts_tablet_manager.cc:1367] T 9d424d86fde04f2dbdd9434ee5755150 P 27f29b79cf9b42f4a5b9838965bbb72d: Time spent bootstrapping tablet: real 0.186s user 0.152s sys 0.032s
I20240503 02:14:39.417768 14758 raft_consensus.cc:348] T 9d424d86fde04f2dbdd9434ee5755150 P 27f29b79cf9b42f4a5b9838965bbb72d [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27f29b79cf9b42f4a5b9838965bbb72d" member_type: VOTER last_known_addr { host: "127.9.151.129" port: 33313 } }
I20240503 02:14:39.417954 14758 raft_consensus.cc:729] T 9d424d86fde04f2dbdd9434ee5755150 P 27f29b79cf9b42f4a5b9838965bbb72d [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 27f29b79cf9b42f4a5b9838965bbb72d, State: Initialized, Role: FOLLOWER
I20240503 02:14:39.418200 14758 consensus_queue.cc:260] T 9d424d86fde04f2dbdd9434ee5755150 P 27f29b79cf9b42f4a5b9838965bbb72d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 190, Last appended: 1.190, Last appended by leader: 190, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27f29b79cf9b42f4a5b9838965bbb72d" member_type: VOTER last_known_addr { host: "127.9.151.129" port: 33313 } }
I20240503 02:14:39.418360 14758 raft_consensus.cc:388] T 9d424d86fde04f2dbdd9434ee5755150 P 27f29b79cf9b42f4a5b9838965bbb72d [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20240503 02:14:39.418437 14758 raft_consensus.cc:482] T 9d424d86fde04f2dbdd9434ee5755150 P 27f29b79cf9b42f4a5b9838965bbb72d [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20240503 02:14:39.418561 14758 raft_consensus.cc:3037] T 9d424d86fde04f2dbdd9434ee5755150 P 27f29b79cf9b42f4a5b9838965bbb72d [term 1 FOLLOWER]: Advancing to term 2
I20240503 02:14:39.420600 14758 raft_consensus.cc:504] T 9d424d86fde04f2dbdd9434ee5755150 P 27f29b79cf9b42f4a5b9838965bbb72d [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27f29b79cf9b42f4a5b9838965bbb72d" member_type: VOTER last_known_addr { host: "127.9.151.129" port: 33313 } }
I20240503 02:14:39.420869 14758 leader_election.cc:302] T 9d424d86fde04f2dbdd9434ee5755150 P 27f29b79cf9b42f4a5b9838965bbb72d [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 27f29b79cf9b42f4a5b9838965bbb72d; no voters:
I20240503 02:14:39.421106 14758 leader_election.cc:288] T 9d424d86fde04f2dbdd9434ee5755150 P 27f29b79cf9b42f4a5b9838965bbb72d [CANDIDATE]: Term 2 election: Requested vote from peers
I20240503 02:14:39.421190 14777 raft_consensus.cc:2781] T 9d424d86fde04f2dbdd9434ee5755150 P 27f29b79cf9b42f4a5b9838965bbb72d [term 2 FOLLOWER]: Leader election won for term 2
I20240503 02:14:39.421358 14777 raft_consensus.cc:686] T 9d424d86fde04f2dbdd9434ee5755150 P 27f29b79cf9b42f4a5b9838965bbb72d [term 2 LEADER]: Becoming Leader. State: Replica: 27f29b79cf9b42f4a5b9838965bbb72d, State: Running, Role: LEADER
I20240503 02:14:39.421811 14758 ts_tablet_manager.cc:1398] T 9d424d86fde04f2dbdd9434ee5755150 P 27f29b79cf9b42f4a5b9838965bbb72d: Time spent starting tablet: real 0.005s user 0.004s sys 0.000s
I20240503 02:14:39.421633 14777 consensus_queue.cc:237] T 9d424d86fde04f2dbdd9434ee5755150 P 27f29b79cf9b42f4a5b9838965bbb72d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 190, Committed index: 190, Last appended: 1.190, Last appended by leader: 190, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27f29b79cf9b42f4a5b9838965bbb72d" member_type: VOTER last_known_addr { host: "127.9.151.129" port: 33313 } }
I20240503 02:14:39.422099 14758 tablet_bootstrap.cc:492] T e50f84673efd4108859e8c1360cad36f P 27f29b79cf9b42f4a5b9838965bbb72d: Bootstrap starting.
I20240503 02:14:39.424578 14433 catalog_manager.cc:5484] T 9d424d86fde04f2dbdd9434ee5755150 P 27f29b79cf9b42f4a5b9838965bbb72d reported cstate change: term changed from 1 to 2. New cstate: current_term: 2 leader_uuid: "27f29b79cf9b42f4a5b9838965bbb72d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27f29b79cf9b42f4a5b9838965bbb72d" member_type: VOTER last_known_addr { host: "127.9.151.129" port: 33313 } health_report { overall_health: HEALTHY } } }
I20240503 02:14:39.610229 14758 tablet_bootstrap.cc:492] T e50f84673efd4108859e8c1360cad36f P 27f29b79cf9b42f4a5b9838965bbb72d: Bootstrap replayed 1/1 log segments. Stats: ops{read=192 overwritten=0 applied=192 ignored=0} inserts{seen=541 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240503 02:14:39.610857 14758 tablet_bootstrap.cc:492] T e50f84673efd4108859e8c1360cad36f P 27f29b79cf9b42f4a5b9838965bbb72d: Bootstrap complete.
I20240503 02:14:39.611923 14758 ts_tablet_manager.cc:1367] T e50f84673efd4108859e8c1360cad36f P 27f29b79cf9b42f4a5b9838965bbb72d: Time spent bootstrapping tablet: real 0.190s user 0.169s sys 0.020s
I20240503 02:14:39.612654 14758 raft_consensus.cc:348] T e50f84673efd4108859e8c1360cad36f P 27f29b79cf9b42f4a5b9838965bbb72d [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27f29b79cf9b42f4a5b9838965bbb72d" member_type: VOTER last_known_addr { host: "127.9.151.129" port: 33313 } }
I20240503 02:14:39.612835 14758 raft_consensus.cc:729] T e50f84673efd4108859e8c1360cad36f P 27f29b79cf9b42f4a5b9838965bbb72d [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 27f29b79cf9b42f4a5b9838965bbb72d, State: Initialized, Role: FOLLOWER
I20240503 02:14:39.613091 14758 consensus_queue.cc:260] T e50f84673efd4108859e8c1360cad36f P 27f29b79cf9b42f4a5b9838965bbb72d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 192, Last appended: 1.192, Last appended by leader: 192, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27f29b79cf9b42f4a5b9838965bbb72d" member_type: VOTER last_known_addr { host: "127.9.151.129" port: 33313 } }
I20240503 02:14:39.613253 14758 raft_consensus.cc:388] T e50f84673efd4108859e8c1360cad36f P 27f29b79cf9b42f4a5b9838965bbb72d [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20240503 02:14:39.613361 14758 raft_consensus.cc:482] T e50f84673efd4108859e8c1360cad36f P 27f29b79cf9b42f4a5b9838965bbb72d [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20240503 02:14:39.613476 14758 raft_consensus.cc:3037] T e50f84673efd4108859e8c1360cad36f P 27f29b79cf9b42f4a5b9838965bbb72d [term 1 FOLLOWER]: Advancing to term 2
I20240503 02:14:39.616065 14758 raft_consensus.cc:504] T e50f84673efd4108859e8c1360cad36f P 27f29b79cf9b42f4a5b9838965bbb72d [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27f29b79cf9b42f4a5b9838965bbb72d" member_type: VOTER last_known_addr { host: "127.9.151.129" port: 33313 } }
I20240503 02:14:39.616307 14758 leader_election.cc:302] T e50f84673efd4108859e8c1360cad36f P 27f29b79cf9b42f4a5b9838965bbb72d [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 27f29b79cf9b42f4a5b9838965bbb72d; no voters:
I20240503 02:14:39.616524 14758 leader_election.cc:288] T e50f84673efd4108859e8c1360cad36f P 27f29b79cf9b42f4a5b9838965bbb72d [CANDIDATE]: Term 2 election: Requested vote from peers
I20240503 02:14:39.616628 14777 raft_consensus.cc:2781] T e50f84673efd4108859e8c1360cad36f P 27f29b79cf9b42f4a5b9838965bbb72d [term 2 FOLLOWER]: Leader election won for term 2
I20240503 02:14:39.616825 14777 raft_consensus.cc:686] T e50f84673efd4108859e8c1360cad36f P 27f29b79cf9b42f4a5b9838965bbb72d [term 2 LEADER]: Becoming Leader. State: Replica: 27f29b79cf9b42f4a5b9838965bbb72d, State: Running, Role: LEADER
I20240503 02:14:39.616983 14758 ts_tablet_manager.cc:1398] T e50f84673efd4108859e8c1360cad36f P 27f29b79cf9b42f4a5b9838965bbb72d: Time spent starting tablet: real 0.005s user 0.004s sys 0.000s
I20240503 02:14:39.617094 14777 consensus_queue.cc:237] T e50f84673efd4108859e8c1360cad36f P 27f29b79cf9b42f4a5b9838965bbb72d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 192, Committed index: 192, Last appended: 1.192, Last appended by leader: 192, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27f29b79cf9b42f4a5b9838965bbb72d" member_type: VOTER last_known_addr { host: "127.9.151.129" port: 33313 } }
I20240503 02:14:39.617259 14758 tablet_bootstrap.cc:492] T fc8f5043623a48418283e5ecd50d58e2 P 27f29b79cf9b42f4a5b9838965bbb72d: Bootstrap starting.
I20240503 02:14:39.619486 14433 catalog_manager.cc:5484] T e50f84673efd4108859e8c1360cad36f P 27f29b79cf9b42f4a5b9838965bbb72d reported cstate change: term changed from 1 to 2. New cstate: current_term: 2 leader_uuid: "27f29b79cf9b42f4a5b9838965bbb72d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27f29b79cf9b42f4a5b9838965bbb72d" member_type: VOTER last_known_addr { host: "127.9.151.129" port: 33313 } health_report { overall_health: HEALTHY } } }
I20240503 02:14:39.806797 14758 tablet_bootstrap.cc:492] T fc8f5043623a48418283e5ecd50d58e2 P 27f29b79cf9b42f4a5b9838965bbb72d: Bootstrap replayed 1/1 log segments. Stats: ops{read=197 overwritten=0 applied=197 ignored=0} inserts{seen=545 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240503 02:14:39.807350 14758 tablet_bootstrap.cc:492] T fc8f5043623a48418283e5ecd50d58e2 P 27f29b79cf9b42f4a5b9838965bbb72d: Bootstrap complete.
I20240503 02:14:39.808317 14758 ts_tablet_manager.cc:1367] T fc8f5043623a48418283e5ecd50d58e2 P 27f29b79cf9b42f4a5b9838965bbb72d: Time spent bootstrapping tablet: real 0.191s user 0.161s sys 0.028s
I20240503 02:14:39.809041 14758 raft_consensus.cc:348] T fc8f5043623a48418283e5ecd50d58e2 P 27f29b79cf9b42f4a5b9838965bbb72d [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27f29b79cf9b42f4a5b9838965bbb72d" member_type: VOTER last_known_addr { host: "127.9.151.129" port: 33313 } }
I20240503 02:14:39.809250 14758 raft_consensus.cc:729] T fc8f5043623a48418283e5ecd50d58e2 P 27f29b79cf9b42f4a5b9838965bbb72d [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 27f29b79cf9b42f4a5b9838965bbb72d, State: Initialized, Role: FOLLOWER
I20240503 02:14:39.809523 14758 consensus_queue.cc:260] T fc8f5043623a48418283e5ecd50d58e2 P 27f29b79cf9b42f4a5b9838965bbb72d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 197, Last appended: 1.197, Last appended by leader: 197, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27f29b79cf9b42f4a5b9838965bbb72d" member_type: VOTER last_known_addr { host: "127.9.151.129" port: 33313 } }
I20240503 02:14:39.809693 14758 raft_consensus.cc:388] T fc8f5043623a48418283e5ecd50d58e2 P 27f29b79cf9b42f4a5b9838965bbb72d [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20240503 02:14:39.809774 14758 raft_consensus.cc:482] T fc8f5043623a48418283e5ecd50d58e2 P 27f29b79cf9b42f4a5b9838965bbb72d [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20240503 02:14:39.809901 14758 raft_consensus.cc:3037] T fc8f5043623a48418283e5ecd50d58e2 P 27f29b79cf9b42f4a5b9838965bbb72d [term 1 FOLLOWER]: Advancing to term 2
I20240503 02:14:39.812297 14758 raft_consensus.cc:504] T fc8f5043623a48418283e5ecd50d58e2 P 27f29b79cf9b42f4a5b9838965bbb72d [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27f29b79cf9b42f4a5b9838965bbb72d" member_type: VOTER last_known_addr { host: "127.9.151.129" port: 33313 } }
I20240503 02:14:39.812754 14758 leader_election.cc:302] T fc8f5043623a48418283e5ecd50d58e2 P 27f29b79cf9b42f4a5b9838965bbb72d [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 27f29b79cf9b42f4a5b9838965bbb72d; no voters:
I20240503 02:14:39.813025 14758 leader_election.cc:288] T fc8f5043623a48418283e5ecd50d58e2 P 27f29b79cf9b42f4a5b9838965bbb72d [CANDIDATE]: Term 2 election: Requested vote from peers
I20240503 02:14:39.813154 14777 raft_consensus.cc:2781] T fc8f5043623a48418283e5ecd50d58e2 P 27f29b79cf9b42f4a5b9838965bbb72d [term 2 FOLLOWER]: Leader election won for term 2
I20240503 02:14:39.813359 14777 raft_consensus.cc:686] T fc8f5043623a48418283e5ecd50d58e2 P 27f29b79cf9b42f4a5b9838965bbb72d [term 2 LEADER]: Becoming Leader. State: Replica: 27f29b79cf9b42f4a5b9838965bbb72d, State: Running, Role: LEADER
I20240503 02:14:39.813586 14758 ts_tablet_manager.cc:1398] T fc8f5043623a48418283e5ecd50d58e2 P 27f29b79cf9b42f4a5b9838965bbb72d: Time spent starting tablet: real 0.005s user 0.004s sys 0.000s
I20240503 02:14:39.813660 14777 consensus_queue.cc:237] T fc8f5043623a48418283e5ecd50d58e2 P 27f29b79cf9b42f4a5b9838965bbb72d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 197, Committed index: 197, Last appended: 1.197, Last appended by leader: 197, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27f29b79cf9b42f4a5b9838965bbb72d" member_type: VOTER last_known_addr { host: "127.9.151.129" port: 33313 } }
I20240503 02:14:39.813843 14758 tablet_bootstrap.cc:492] T f884dde21ea246a792634883aed3cb82 P 27f29b79cf9b42f4a5b9838965bbb72d: Bootstrap starting.
I20240503 02:14:39.816608 14433 catalog_manager.cc:5484] T fc8f5043623a48418283e5ecd50d58e2 P 27f29b79cf9b42f4a5b9838965bbb72d reported cstate change: term changed from 1 to 2. New cstate: current_term: 2 leader_uuid: "27f29b79cf9b42f4a5b9838965bbb72d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27f29b79cf9b42f4a5b9838965bbb72d" member_type: VOTER last_known_addr { host: "127.9.151.129" port: 33313 } health_report { overall_health: HEALTHY } } }
I20240503 02:14:40.002468 14758 tablet_bootstrap.cc:492] T f884dde21ea246a792634883aed3cb82 P 27f29b79cf9b42f4a5b9838965bbb72d: Bootstrap replayed 1/1 log segments. Stats: ops{read=194 overwritten=0 applied=194 ignored=0} inserts{seen=516 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240503 02:14:40.003049 14758 tablet_bootstrap.cc:492] T f884dde21ea246a792634883aed3cb82 P 27f29b79cf9b42f4a5b9838965bbb72d: Bootstrap complete.
I20240503 02:14:40.004030 14758 ts_tablet_manager.cc:1367] T f884dde21ea246a792634883aed3cb82 P 27f29b79cf9b42f4a5b9838965bbb72d: Time spent bootstrapping tablet: real 0.190s user 0.161s sys 0.028s
I20240503 02:14:40.004700 14758 raft_consensus.cc:348] T f884dde21ea246a792634883aed3cb82 P 27f29b79cf9b42f4a5b9838965bbb72d [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27f29b79cf9b42f4a5b9838965bbb72d" member_type: VOTER last_known_addr { host: "127.9.151.129" port: 33313 } }
I20240503 02:14:40.004865 14758 raft_consensus.cc:729] T f884dde21ea246a792634883aed3cb82 P 27f29b79cf9b42f4a5b9838965bbb72d [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 27f29b79cf9b42f4a5b9838965bbb72d, State: Initialized, Role: FOLLOWER
I20240503 02:14:40.005128 14758 consensus_queue.cc:260] T f884dde21ea246a792634883aed3cb82 P 27f29b79cf9b42f4a5b9838965bbb72d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 194, Last appended: 1.194, Last appended by leader: 194, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27f29b79cf9b42f4a5b9838965bbb72d" member_type: VOTER last_known_addr { host: "127.9.151.129" port: 33313 } }
I20240503 02:14:40.005280 14758 raft_consensus.cc:388] T f884dde21ea246a792634883aed3cb82 P 27f29b79cf9b42f4a5b9838965bbb72d [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20240503 02:14:40.005379 14758 raft_consensus.cc:482] T f884dde21ea246a792634883aed3cb82 P 27f29b79cf9b42f4a5b9838965bbb72d [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20240503 02:14:40.005470 14758 raft_consensus.cc:3037] T f884dde21ea246a792634883aed3cb82 P 27f29b79cf9b42f4a5b9838965bbb72d [term 1 FOLLOWER]: Advancing to term 2
I20240503 02:14:40.007557 14758 raft_consensus.cc:504] T f884dde21ea246a792634883aed3cb82 P 27f29b79cf9b42f4a5b9838965bbb72d [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27f29b79cf9b42f4a5b9838965bbb72d" member_type: VOTER last_known_addr { host: "127.9.151.129" port: 33313 } }
I20240503 02:14:40.007784 14758 leader_election.cc:302] T f884dde21ea246a792634883aed3cb82 P 27f29b79cf9b42f4a5b9838965bbb72d [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 27f29b79cf9b42f4a5b9838965bbb72d; no voters:
I20240503 02:14:40.007993 14758 leader_election.cc:288] T f884dde21ea246a792634883aed3cb82 P 27f29b79cf9b42f4a5b9838965bbb72d [CANDIDATE]: Term 2 election: Requested vote from peers
I20240503 02:14:40.008101 14777 raft_consensus.cc:2781] T f884dde21ea246a792634883aed3cb82 P 27f29b79cf9b42f4a5b9838965bbb72d [term 2 FOLLOWER]: Leader election won for term 2
I20240503 02:14:40.008306 14777 raft_consensus.cc:686] T f884dde21ea246a792634883aed3cb82 P 27f29b79cf9b42f4a5b9838965bbb72d [term 2 LEADER]: Becoming Leader. State: Replica: 27f29b79cf9b42f4a5b9838965bbb72d, State: Running, Role: LEADER
I20240503 02:14:40.008499 14758 ts_tablet_manager.cc:1398] T f884dde21ea246a792634883aed3cb82 P 27f29b79cf9b42f4a5b9838965bbb72d: Time spent starting tablet: real 0.004s user 0.004s sys 0.000s
I20240503 02:14:40.008562 14777 consensus_queue.cc:237] T f884dde21ea246a792634883aed3cb82 P 27f29b79cf9b42f4a5b9838965bbb72d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 194, Committed index: 194, Last appended: 1.194, Last appended by leader: 194, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27f29b79cf9b42f4a5b9838965bbb72d" member_type: VOTER last_known_addr { host: "127.9.151.129" port: 33313 } }
I20240503 02:14:40.008795 14758 tablet_bootstrap.cc:492] T fe8b26bb37ab4b948fd500b648bf6b5a P 27f29b79cf9b42f4a5b9838965bbb72d: Bootstrap starting.
I20240503 02:14:40.011233 14433 catalog_manager.cc:5484] T f884dde21ea246a792634883aed3cb82 P 27f29b79cf9b42f4a5b9838965bbb72d reported cstate change: term changed from 1 to 2. New cstate: current_term: 2 leader_uuid: "27f29b79cf9b42f4a5b9838965bbb72d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27f29b79cf9b42f4a5b9838965bbb72d" member_type: VOTER last_known_addr { host: "127.9.151.129" port: 33313 } health_report { overall_health: HEALTHY } } }
I20240503 02:14:40.195947 14758 tablet_bootstrap.cc:492] T fe8b26bb37ab4b948fd500b648bf6b5a P 27f29b79cf9b42f4a5b9838965bbb72d: Bootstrap replayed 1/1 log segments. Stats: ops{read=197 overwritten=0 applied=197 ignored=0} inserts{seen=515 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240503 02:14:40.196427 14758 tablet_bootstrap.cc:492] T fe8b26bb37ab4b948fd500b648bf6b5a P 27f29b79cf9b42f4a5b9838965bbb72d: Bootstrap complete.
I20240503 02:14:40.197487 14758 ts_tablet_manager.cc:1367] T fe8b26bb37ab4b948fd500b648bf6b5a P 27f29b79cf9b42f4a5b9838965bbb72d: Time spent bootstrapping tablet: real 0.189s user 0.173s sys 0.012s
I20240503 02:14:40.198294 14758 raft_consensus.cc:348] T fe8b26bb37ab4b948fd500b648bf6b5a P 27f29b79cf9b42f4a5b9838965bbb72d [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27f29b79cf9b42f4a5b9838965bbb72d" member_type: VOTER last_known_addr { host: "127.9.151.129" port: 33313 } }
I20240503 02:14:40.198499 14758 raft_consensus.cc:729] T fe8b26bb37ab4b948fd500b648bf6b5a P 27f29b79cf9b42f4a5b9838965bbb72d [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 27f29b79cf9b42f4a5b9838965bbb72d, State: Initialized, Role: FOLLOWER
I20240503 02:14:40.198808 14758 consensus_queue.cc:260] T fe8b26bb37ab4b948fd500b648bf6b5a P 27f29b79cf9b42f4a5b9838965bbb72d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 197, Last appended: 1.197, Last appended by leader: 197, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27f29b79cf9b42f4a5b9838965bbb72d" member_type: VOTER last_known_addr { host: "127.9.151.129" port: 33313 } }
I20240503 02:14:40.198979 14758 raft_consensus.cc:388] T fe8b26bb37ab4b948fd500b648bf6b5a P 27f29b79cf9b42f4a5b9838965bbb72d [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20240503 02:14:40.199074 14758 raft_consensus.cc:482] T fe8b26bb37ab4b948fd500b648bf6b5a P 27f29b79cf9b42f4a5b9838965bbb72d [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20240503 02:14:40.199213 14758 raft_consensus.cc:3037] T fe8b26bb37ab4b948fd500b648bf6b5a P 27f29b79cf9b42f4a5b9838965bbb72d [term 1 FOLLOWER]: Advancing to term 2
I20240503 02:14:40.201442 14758 raft_consensus.cc:504] T fe8b26bb37ab4b948fd500b648bf6b5a P 27f29b79cf9b42f4a5b9838965bbb72d [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27f29b79cf9b42f4a5b9838965bbb72d" member_type: VOTER last_known_addr { host: "127.9.151.129" port: 33313 } }
I20240503 02:14:40.201697 14758 leader_election.cc:302] T fe8b26bb37ab4b948fd500b648bf6b5a P 27f29b79cf9b42f4a5b9838965bbb72d [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 27f29b79cf9b42f4a5b9838965bbb72d; no voters:
I20240503 02:14:40.201900 14758 leader_election.cc:288] T fe8b26bb37ab4b948fd500b648bf6b5a P 27f29b79cf9b42f4a5b9838965bbb72d [CANDIDATE]: Term 2 election: Requested vote from peers
I20240503 02:14:40.202085 14777 raft_consensus.cc:2781] T fe8b26bb37ab4b948fd500b648bf6b5a P 27f29b79cf9b42f4a5b9838965bbb72d [term 2 FOLLOWER]: Leader election won for term 2
I20240503 02:14:40.202291 14777 raft_consensus.cc:686] T fe8b26bb37ab4b948fd500b648bf6b5a P 27f29b79cf9b42f4a5b9838965bbb72d [term 2 LEADER]: Becoming Leader. State: Replica: 27f29b79cf9b42f4a5b9838965bbb72d, State: Running, Role: LEADER
I20240503 02:14:40.202461 14758 ts_tablet_manager.cc:1398] T fe8b26bb37ab4b948fd500b648bf6b5a P 27f29b79cf9b42f4a5b9838965bbb72d: Time spent starting tablet: real 0.005s user 0.004s sys 0.000s
I20240503 02:14:40.202570 14777 consensus_queue.cc:237] T fe8b26bb37ab4b948fd500b648bf6b5a P 27f29b79cf9b42f4a5b9838965bbb72d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 197, Committed index: 197, Last appended: 1.197, Last appended by leader: 197, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27f29b79cf9b42f4a5b9838965bbb72d" member_type: VOTER last_known_addr { host: "127.9.151.129" port: 33313 } }
I20240503 02:14:40.202750 14758 tablet_bootstrap.cc:492] T a40f48ddab6e48d5ba3707ef78923ee5 P 27f29b79cf9b42f4a5b9838965bbb72d: Bootstrap starting.
I20240503 02:14:40.205796 14438 catalog_manager.cc:5484] T fe8b26bb37ab4b948fd500b648bf6b5a P 27f29b79cf9b42f4a5b9838965bbb72d reported cstate change: term changed from 1 to 2. New cstate: current_term: 2 leader_uuid: "27f29b79cf9b42f4a5b9838965bbb72d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27f29b79cf9b42f4a5b9838965bbb72d" member_type: VOTER last_known_addr { host: "127.9.151.129" port: 33313 } health_report { overall_health: HEALTHY } } }
I20240503 02:14:40.386021 14758 tablet_bootstrap.cc:492] T a40f48ddab6e48d5ba3707ef78923ee5 P 27f29b79cf9b42f4a5b9838965bbb72d: Bootstrap replayed 1/1 log segments. Stats: ops{read=191 overwritten=0 applied=191 ignored=0} inserts{seen=494 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240503 02:14:40.386541 14758 tablet_bootstrap.cc:492] T a40f48ddab6e48d5ba3707ef78923ee5 P 27f29b79cf9b42f4a5b9838965bbb72d: Bootstrap complete.
I20240503 02:14:40.387578 14758 ts_tablet_manager.cc:1367] T a40f48ddab6e48d5ba3707ef78923ee5 P 27f29b79cf9b42f4a5b9838965bbb72d: Time spent bootstrapping tablet: real 0.185s user 0.166s sys 0.016s
I20240503 02:14:40.388270 14758 raft_consensus.cc:348] T a40f48ddab6e48d5ba3707ef78923ee5 P 27f29b79cf9b42f4a5b9838965bbb72d [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27f29b79cf9b42f4a5b9838965bbb72d" member_type: VOTER last_known_addr { host: "127.9.151.129" port: 33313 } }
I20240503 02:14:40.388423 14758 raft_consensus.cc:729] T a40f48ddab6e48d5ba3707ef78923ee5 P 27f29b79cf9b42f4a5b9838965bbb72d [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 27f29b79cf9b42f4a5b9838965bbb72d, State: Initialized, Role: FOLLOWER
I20240503 02:14:40.388641 14758 consensus_queue.cc:260] T a40f48ddab6e48d5ba3707ef78923ee5 P 27f29b79cf9b42f4a5b9838965bbb72d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 191, Last appended: 1.191, Last appended by leader: 191, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27f29b79cf9b42f4a5b9838965bbb72d" member_type: VOTER last_known_addr { host: "127.9.151.129" port: 33313 } }
I20240503 02:14:40.388758 14758 raft_consensus.cc:388] T a40f48ddab6e48d5ba3707ef78923ee5 P 27f29b79cf9b42f4a5b9838965bbb72d [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20240503 02:14:40.388818 14758 raft_consensus.cc:482] T a40f48ddab6e48d5ba3707ef78923ee5 P 27f29b79cf9b42f4a5b9838965bbb72d [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20240503 02:14:40.388989 14758 raft_consensus.cc:3037] T a40f48ddab6e48d5ba3707ef78923ee5 P 27f29b79cf9b42f4a5b9838965bbb72d [term 1 FOLLOWER]: Advancing to term 2
I20240503 02:14:40.391651 14758 raft_consensus.cc:504] T a40f48ddab6e48d5ba3707ef78923ee5 P 27f29b79cf9b42f4a5b9838965bbb72d [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27f29b79cf9b42f4a5b9838965bbb72d" member_type: VOTER last_known_addr { host: "127.9.151.129" port: 33313 } }
I20240503 02:14:40.391922 14758 leader_election.cc:302] T a40f48ddab6e48d5ba3707ef78923ee5 P 27f29b79cf9b42f4a5b9838965bbb72d [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 27f29b79cf9b42f4a5b9838965bbb72d; no voters:
I20240503 02:14:40.392133 14758 leader_election.cc:288] T a40f48ddab6e48d5ba3707ef78923ee5 P 27f29b79cf9b42f4a5b9838965bbb72d [CANDIDATE]: Term 2 election: Requested vote from peers
I20240503 02:14:40.392267 14777 raft_consensus.cc:2781] T a40f48ddab6e48d5ba3707ef78923ee5 P 27f29b79cf9b42f4a5b9838965bbb72d [term 2 FOLLOWER]: Leader election won for term 2
I20240503 02:14:40.392473 14777 raft_consensus.cc:686] T a40f48ddab6e48d5ba3707ef78923ee5 P 27f29b79cf9b42f4a5b9838965bbb72d [term 2 LEADER]: Becoming Leader. State: Replica: 27f29b79cf9b42f4a5b9838965bbb72d, State: Running, Role: LEADER
I20240503 02:14:40.392580 14758 ts_tablet_manager.cc:1398] T a40f48ddab6e48d5ba3707ef78923ee5 P 27f29b79cf9b42f4a5b9838965bbb72d: Time spent starting tablet: real 0.005s user 0.004s sys 0.000s
I20240503 02:14:40.392850 14758 tablet_bootstrap.cc:492] T b0df8ae119f948eaa5461faa42007713 P 27f29b79cf9b42f4a5b9838965bbb72d: Bootstrap starting.
I20240503 02:14:40.392763 14777 consensus_queue.cc:237] T a40f48ddab6e48d5ba3707ef78923ee5 P 27f29b79cf9b42f4a5b9838965bbb72d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 191, Committed index: 191, Last appended: 1.191, Last appended by leader: 191, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27f29b79cf9b42f4a5b9838965bbb72d" member_type: VOTER last_known_addr { host: "127.9.151.129" port: 33313 } }
I20240503 02:14:40.395577 14438 catalog_manager.cc:5484] T a40f48ddab6e48d5ba3707ef78923ee5 P 27f29b79cf9b42f4a5b9838965bbb72d reported cstate change: term changed from 1 to 2. New cstate: current_term: 2 leader_uuid: "27f29b79cf9b42f4a5b9838965bbb72d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27f29b79cf9b42f4a5b9838965bbb72d" member_type: VOTER last_known_addr { host: "127.9.151.129" port: 33313 } health_report { overall_health: HEALTHY } } }
I20240503 02:14:40.576714 14758 tablet_bootstrap.cc:492] T b0df8ae119f948eaa5461faa42007713 P 27f29b79cf9b42f4a5b9838965bbb72d: Bootstrap replayed 1/1 log segments. Stats: ops{read=191 overwritten=0 applied=191 ignored=0} inserts{seen=515 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240503 02:14:40.577209 14758 tablet_bootstrap.cc:492] T b0df8ae119f948eaa5461faa42007713 P 27f29b79cf9b42f4a5b9838965bbb72d: Bootstrap complete.
I20240503 02:14:40.578199 14758 ts_tablet_manager.cc:1367] T b0df8ae119f948eaa5461faa42007713 P 27f29b79cf9b42f4a5b9838965bbb72d: Time spent bootstrapping tablet: real 0.185s user 0.166s sys 0.016s
I20240503 02:14:40.578917 14758 raft_consensus.cc:348] T b0df8ae119f948eaa5461faa42007713 P 27f29b79cf9b42f4a5b9838965bbb72d [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27f29b79cf9b42f4a5b9838965bbb72d" member_type: VOTER last_known_addr { host: "127.9.151.129" port: 33313 } }
I20240503 02:14:40.579103 14758 raft_consensus.cc:729] T b0df8ae119f948eaa5461faa42007713 P 27f29b79cf9b42f4a5b9838965bbb72d [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 27f29b79cf9b42f4a5b9838965bbb72d, State: Initialized, Role: FOLLOWER
I20240503 02:14:40.579344 14758 consensus_queue.cc:260] T b0df8ae119f948eaa5461faa42007713 P 27f29b79cf9b42f4a5b9838965bbb72d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 191, Last appended: 1.191, Last appended by leader: 191, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27f29b79cf9b42f4a5b9838965bbb72d" member_type: VOTER last_known_addr { host: "127.9.151.129" port: 33313 } }
I20240503 02:14:40.579504 14758 raft_consensus.cc:388] T b0df8ae119f948eaa5461faa42007713 P 27f29b79cf9b42f4a5b9838965bbb72d [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20240503 02:14:40.579587 14758 raft_consensus.cc:482] T b0df8ae119f948eaa5461faa42007713 P 27f29b79cf9b42f4a5b9838965bbb72d [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20240503 02:14:40.579708 14758 raft_consensus.cc:3037] T b0df8ae119f948eaa5461faa42007713 P 27f29b79cf9b42f4a5b9838965bbb72d [term 1 FOLLOWER]: Advancing to term 2
I20240503 02:14:40.582089 14758 raft_consensus.cc:504] T b0df8ae119f948eaa5461faa42007713 P 27f29b79cf9b42f4a5b9838965bbb72d [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27f29b79cf9b42f4a5b9838965bbb72d" member_type: VOTER last_known_addr { host: "127.9.151.129" port: 33313 } }
I20240503 02:14:40.582419 14758 leader_election.cc:302] T b0df8ae119f948eaa5461faa42007713 P 27f29b79cf9b42f4a5b9838965bbb72d [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 27f29b79cf9b42f4a5b9838965bbb72d; no voters:
I20240503 02:14:40.582633 14758 leader_election.cc:288] T b0df8ae119f948eaa5461faa42007713 P 27f29b79cf9b42f4a5b9838965bbb72d [CANDIDATE]: Term 2 election: Requested vote from peers
I20240503 02:14:40.582763 14777 raft_consensus.cc:2781] T b0df8ae119f948eaa5461faa42007713 P 27f29b79cf9b42f4a5b9838965bbb72d [term 2 FOLLOWER]: Leader election won for term 2
I20240503 02:14:40.583262 14758 ts_tablet_manager.cc:1398] T b0df8ae119f948eaa5461faa42007713 P 27f29b79cf9b42f4a5b9838965bbb72d: Time spent starting tablet: real 0.005s user 0.008s sys 0.000s
I20240503 02:14:40.583559 14777 raft_consensus.cc:686] T b0df8ae119f948eaa5461faa42007713 P 27f29b79cf9b42f4a5b9838965bbb72d [term 2 LEADER]: Becoming Leader. State: Replica: 27f29b79cf9b42f4a5b9838965bbb72d, State: Running, Role: LEADER
I20240503 02:14:40.583639 14758 tablet_bootstrap.cc:492] T 92cd1f3c9cdc43e0b9d56e3375488f50 P 27f29b79cf9b42f4a5b9838965bbb72d: Bootstrap starting.
I20240503 02:14:40.583861 14777 consensus_queue.cc:237] T b0df8ae119f948eaa5461faa42007713 P 27f29b79cf9b42f4a5b9838965bbb72d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 191, Committed index: 191, Last appended: 1.191, Last appended by leader: 191, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27f29b79cf9b42f4a5b9838965bbb72d" member_type: VOTER last_known_addr { host: "127.9.151.129" port: 33313 } }
I20240503 02:14:40.586884 14438 catalog_manager.cc:5484] T b0df8ae119f948eaa5461faa42007713 P 27f29b79cf9b42f4a5b9838965bbb72d reported cstate change: term changed from 1 to 2. New cstate: current_term: 2 leader_uuid: "27f29b79cf9b42f4a5b9838965bbb72d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27f29b79cf9b42f4a5b9838965bbb72d" member_type: VOTER last_known_addr { host: "127.9.151.129" port: 33313 } health_report { overall_health: HEALTHY } } }
I20240503 02:14:40.772464 14758 tablet_bootstrap.cc:492] T 92cd1f3c9cdc43e0b9d56e3375488f50 P 27f29b79cf9b42f4a5b9838965bbb72d: Bootstrap replayed 1/1 log segments. Stats: ops{read=193 overwritten=0 applied=193 ignored=0} inserts{seen=518 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240503 02:14:40.772991 14758 tablet_bootstrap.cc:492] T 92cd1f3c9cdc43e0b9d56e3375488f50 P 27f29b79cf9b42f4a5b9838965bbb72d: Bootstrap complete.
I20240503 02:14:40.773977 14758 ts_tablet_manager.cc:1367] T 92cd1f3c9cdc43e0b9d56e3375488f50 P 27f29b79cf9b42f4a5b9838965bbb72d: Time spent bootstrapping tablet: real 0.190s user 0.179s sys 0.008s
I20240503 02:14:40.774741 14758 raft_consensus.cc:348] T 92cd1f3c9cdc43e0b9d56e3375488f50 P 27f29b79cf9b42f4a5b9838965bbb72d [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27f29b79cf9b42f4a5b9838965bbb72d" member_type: VOTER last_known_addr { host: "127.9.151.129" port: 33313 } }
I20240503 02:14:40.774894 14758 raft_consensus.cc:729] T 92cd1f3c9cdc43e0b9d56e3375488f50 P 27f29b79cf9b42f4a5b9838965bbb72d [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 27f29b79cf9b42f4a5b9838965bbb72d, State: Initialized, Role: FOLLOWER
I20240503 02:14:40.775116 14758 consensus_queue.cc:260] T 92cd1f3c9cdc43e0b9d56e3375488f50 P 27f29b79cf9b42f4a5b9838965bbb72d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 193, Last appended: 1.193, Last appended by leader: 193, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27f29b79cf9b42f4a5b9838965bbb72d" member_type: VOTER last_known_addr { host: "127.9.151.129" port: 33313 } }
I20240503 02:14:40.775240 14758 raft_consensus.cc:388] T 92cd1f3c9cdc43e0b9d56e3375488f50 P 27f29b79cf9b42f4a5b9838965bbb72d [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20240503 02:14:40.775302 14758 raft_consensus.cc:482] T 92cd1f3c9cdc43e0b9d56e3375488f50 P 27f29b79cf9b42f4a5b9838965bbb72d [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20240503 02:14:40.775378 14758 raft_consensus.cc:3037] T 92cd1f3c9cdc43e0b9d56e3375488f50 P 27f29b79cf9b42f4a5b9838965bbb72d [term 1 FOLLOWER]: Advancing to term 2
I20240503 02:14:40.777328 14758 raft_consensus.cc:504] T 92cd1f3c9cdc43e0b9d56e3375488f50 P 27f29b79cf9b42f4a5b9838965bbb72d [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27f29b79cf9b42f4a5b9838965bbb72d" member_type: VOTER last_known_addr { host: "127.9.151.129" port: 33313 } }
I20240503 02:14:40.777586 14758 leader_election.cc:302] T 92cd1f3c9cdc43e0b9d56e3375488f50 P 27f29b79cf9b42f4a5b9838965bbb72d [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 27f29b79cf9b42f4a5b9838965bbb72d; no voters:
I20240503 02:14:40.777798 14758 leader_election.cc:288] T 92cd1f3c9cdc43e0b9d56e3375488f50 P 27f29b79cf9b42f4a5b9838965bbb72d [CANDIDATE]: Term 2 election: Requested vote from peers
I20240503 02:14:40.777942 14777 raft_consensus.cc:2781] T 92cd1f3c9cdc43e0b9d56e3375488f50 P 27f29b79cf9b42f4a5b9838965bbb72d [term 2 FOLLOWER]: Leader election won for term 2
I20240503 02:14:40.778148 14777 raft_consensus.cc:686] T 92cd1f3c9cdc43e0b9d56e3375488f50 P 27f29b79cf9b42f4a5b9838965bbb72d [term 2 LEADER]: Becoming Leader. State: Replica: 27f29b79cf9b42f4a5b9838965bbb72d, State: Running, Role: LEADER
I20240503 02:14:40.778276 14758 ts_tablet_manager.cc:1398] T 92cd1f3c9cdc43e0b9d56e3375488f50 P 27f29b79cf9b42f4a5b9838965bbb72d: Time spent starting tablet: real 0.004s user 0.004s sys 0.000s
I20240503 02:14:40.778398 14777 consensus_queue.cc:237] T 92cd1f3c9cdc43e0b9d56e3375488f50 P 27f29b79cf9b42f4a5b9838965bbb72d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 193, Committed index: 193, Last appended: 1.193, Last appended by leader: 193, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27f29b79cf9b42f4a5b9838965bbb72d" member_type: VOTER last_known_addr { host: "127.9.151.129" port: 33313 } }
I20240503 02:14:40.778525 14758 tablet_bootstrap.cc:492] T 24e81fa4c7b249aca961df89f56a8557 P 27f29b79cf9b42f4a5b9838965bbb72d: Bootstrap starting.
I20240503 02:14:40.781138 14438 catalog_manager.cc:5484] T 92cd1f3c9cdc43e0b9d56e3375488f50 P 27f29b79cf9b42f4a5b9838965bbb72d reported cstate change: term changed from 1 to 2. New cstate: current_term: 2 leader_uuid: "27f29b79cf9b42f4a5b9838965bbb72d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27f29b79cf9b42f4a5b9838965bbb72d" member_type: VOTER last_known_addr { host: "127.9.151.129" port: 33313 } health_report { overall_health: HEALTHY } } }
I20240503 02:14:40.975915 14758 tablet_bootstrap.cc:492] T 24e81fa4c7b249aca961df89f56a8557 P 27f29b79cf9b42f4a5b9838965bbb72d: Bootstrap replayed 1/1 log segments. Stats: ops{read=191 overwritten=0 applied=191 ignored=0} inserts{seen=549 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240503 02:14:40.976526 14758 tablet_bootstrap.cc:492] T 24e81fa4c7b249aca961df89f56a8557 P 27f29b79cf9b42f4a5b9838965bbb72d: Bootstrap complete.
I20240503 02:14:40.977502 14758 ts_tablet_manager.cc:1367] T 24e81fa4c7b249aca961df89f56a8557 P 27f29b79cf9b42f4a5b9838965bbb72d: Time spent bootstrapping tablet: real 0.199s user 0.164s sys 0.032s
I20240503 02:14:40.978123 14758 raft_consensus.cc:348] T 24e81fa4c7b249aca961df89f56a8557 P 27f29b79cf9b42f4a5b9838965bbb72d [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27f29b79cf9b42f4a5b9838965bbb72d" member_type: VOTER last_known_addr { host: "127.9.151.129" port: 33313 } }
I20240503 02:14:40.978307 14758 raft_consensus.cc:729] T 24e81fa4c7b249aca961df89f56a8557 P 27f29b79cf9b42f4a5b9838965bbb72d [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 27f29b79cf9b42f4a5b9838965bbb72d, State: Initialized, Role: FOLLOWER
I20240503 02:14:40.978515 14758 consensus_queue.cc:260] T 24e81fa4c7b249aca961df89f56a8557 P 27f29b79cf9b42f4a5b9838965bbb72d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 191, Last appended: 1.191, Last appended by leader: 191, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27f29b79cf9b42f4a5b9838965bbb72d" member_type: VOTER last_known_addr { host: "127.9.151.129" port: 33313 } }
I20240503 02:14:40.978703 14758 raft_consensus.cc:388] T 24e81fa4c7b249aca961df89f56a8557 P 27f29b79cf9b42f4a5b9838965bbb72d [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20240503 02:14:40.978780 14758 raft_consensus.cc:482] T 24e81fa4c7b249aca961df89f56a8557 P 27f29b79cf9b42f4a5b9838965bbb72d [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20240503 02:14:40.978864 14758 raft_consensus.cc:3037] T 24e81fa4c7b249aca961df89f56a8557 P 27f29b79cf9b42f4a5b9838965bbb72d [term 1 FOLLOWER]: Advancing to term 2
I20240503 02:14:40.980698 14758 raft_consensus.cc:504] T 24e81fa4c7b249aca961df89f56a8557 P 27f29b79cf9b42f4a5b9838965bbb72d [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27f29b79cf9b42f4a5b9838965bbb72d" member_type: VOTER last_known_addr { host: "127.9.151.129" port: 33313 } }
I20240503 02:14:40.980929 14758 leader_election.cc:302] T 24e81fa4c7b249aca961df89f56a8557 P 27f29b79cf9b42f4a5b9838965bbb72d [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 27f29b79cf9b42f4a5b9838965bbb72d; no voters:
I20240503 02:14:40.981093 14758 leader_election.cc:288] T 24e81fa4c7b249aca961df89f56a8557 P 27f29b79cf9b42f4a5b9838965bbb72d [CANDIDATE]: Term 2 election: Requested vote from peers
I20240503 02:14:40.981225 14777 raft_consensus.cc:2781] T 24e81fa4c7b249aca961df89f56a8557 P 27f29b79cf9b42f4a5b9838965bbb72d [term 2 FOLLOWER]: Leader election won for term 2
I20240503 02:14:40.981418 14777 raft_consensus.cc:686] T 24e81fa4c7b249aca961df89f56a8557 P 27f29b79cf9b42f4a5b9838965bbb72d [term 2 LEADER]: Becoming Leader. State: Replica: 27f29b79cf9b42f4a5b9838965bbb72d, State: Running, Role: LEADER
I20240503 02:14:40.981567 14758 ts_tablet_manager.cc:1398] T 24e81fa4c7b249aca961df89f56a8557 P 27f29b79cf9b42f4a5b9838965bbb72d: Time spent starting tablet: real 0.004s user 0.000s sys 0.004s
I20240503 02:14:40.981716 14777 consensus_queue.cc:237] T 24e81fa4c7b249aca961df89f56a8557 P 27f29b79cf9b42f4a5b9838965bbb72d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 191, Committed index: 191, Last appended: 1.191, Last appended by leader: 191, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27f29b79cf9b42f4a5b9838965bbb72d" member_type: VOTER last_known_addr { host: "127.9.151.129" port: 33313 } }
I20240503 02:14:40.981822 14758 tablet_bootstrap.cc:492] T 6069a864c5ff43348881ff477218578d P 27f29b79cf9b42f4a5b9838965bbb72d: Bootstrap starting.
I20240503 02:14:40.985091 14438 catalog_manager.cc:5484] T 24e81fa4c7b249aca961df89f56a8557 P 27f29b79cf9b42f4a5b9838965bbb72d reported cstate change: term changed from 1 to 2. New cstate: current_term: 2 leader_uuid: "27f29b79cf9b42f4a5b9838965bbb72d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27f29b79cf9b42f4a5b9838965bbb72d" member_type: VOTER last_known_addr { host: "127.9.151.129" port: 33313 } health_report { overall_health: HEALTHY } } }
I20240503 02:14:41.171430 14758 tablet_bootstrap.cc:492] T 6069a864c5ff43348881ff477218578d P 27f29b79cf9b42f4a5b9838965bbb72d: Bootstrap replayed 1/1 log segments. Stats: ops{read=190 overwritten=0 applied=190 ignored=0} inserts{seen=513 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240503 02:14:41.171962 14758 tablet_bootstrap.cc:492] T 6069a864c5ff43348881ff477218578d P 27f29b79cf9b42f4a5b9838965bbb72d: Bootstrap complete.
I20240503 02:14:41.172945 14758 ts_tablet_manager.cc:1367] T 6069a864c5ff43348881ff477218578d P 27f29b79cf9b42f4a5b9838965bbb72d: Time spent bootstrapping tablet: real 0.191s user 0.167s sys 0.020s
I20240503 02:14:41.173604 14758 raft_consensus.cc:348] T 6069a864c5ff43348881ff477218578d P 27f29b79cf9b42f4a5b9838965bbb72d [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27f29b79cf9b42f4a5b9838965bbb72d" member_type: VOTER last_known_addr { host: "127.9.151.129" port: 33313 } }
I20240503 02:14:41.173777 14758 raft_consensus.cc:729] T 6069a864c5ff43348881ff477218578d P 27f29b79cf9b42f4a5b9838965bbb72d [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 27f29b79cf9b42f4a5b9838965bbb72d, State: Initialized, Role: FOLLOWER
I20240503 02:14:41.174010 14758 consensus_queue.cc:260] T 6069a864c5ff43348881ff477218578d P 27f29b79cf9b42f4a5b9838965bbb72d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 190, Last appended: 1.190, Last appended by leader: 190, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27f29b79cf9b42f4a5b9838965bbb72d" member_type: VOTER last_known_addr { host: "127.9.151.129" port: 33313 } }
I20240503 02:14:41.174173 14758 raft_consensus.cc:388] T 6069a864c5ff43348881ff477218578d P 27f29b79cf9b42f4a5b9838965bbb72d [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20240503 02:14:41.174275 14758 raft_consensus.cc:482] T 6069a864c5ff43348881ff477218578d P 27f29b79cf9b42f4a5b9838965bbb72d [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20240503 02:14:41.174360 14758 raft_consensus.cc:3037] T 6069a864c5ff43348881ff477218578d P 27f29b79cf9b42f4a5b9838965bbb72d [term 1 FOLLOWER]: Advancing to term 2
I20240503 02:14:41.176577 14758 raft_consensus.cc:504] T 6069a864c5ff43348881ff477218578d P 27f29b79cf9b42f4a5b9838965bbb72d [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27f29b79cf9b42f4a5b9838965bbb72d" member_type: VOTER last_known_addr { host: "127.9.151.129" port: 33313 } }
I20240503 02:14:41.176839 14758 leader_election.cc:302] T 6069a864c5ff43348881ff477218578d P 27f29b79cf9b42f4a5b9838965bbb72d [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 27f29b79cf9b42f4a5b9838965bbb72d; no voters:
I20240503 02:14:41.177048 14758 leader_election.cc:288] T 6069a864c5ff43348881ff477218578d P 27f29b79cf9b42f4a5b9838965bbb72d [CANDIDATE]: Term 2 election: Requested vote from peers
I20240503 02:14:41.177129 14777 raft_consensus.cc:2781] T 6069a864c5ff43348881ff477218578d P 27f29b79cf9b42f4a5b9838965bbb72d [term 2 FOLLOWER]: Leader election won for term 2
I20240503 02:14:41.177476 14758 ts_tablet_manager.cc:1398] T 6069a864c5ff43348881ff477218578d P 27f29b79cf9b42f4a5b9838965bbb72d: Time spent starting tablet: real 0.004s user 0.004s sys 0.000s
I20240503 02:14:41.177774 14758 tablet_bootstrap.cc:492] T cefdd1e872814ddbb522f4013d55b26c P 27f29b79cf9b42f4a5b9838965bbb72d: Bootstrap starting.
I20240503 02:14:41.177814 14777 raft_consensus.cc:686] T 6069a864c5ff43348881ff477218578d P 27f29b79cf9b42f4a5b9838965bbb72d [term 2 LEADER]: Becoming Leader. State: Replica: 27f29b79cf9b42f4a5b9838965bbb72d, State: Running, Role: LEADER
I20240503 02:14:41.178086 14777 consensus_queue.cc:237] T 6069a864c5ff43348881ff477218578d P 27f29b79cf9b42f4a5b9838965bbb72d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 190, Committed index: 190, Last appended: 1.190, Last appended by leader: 190, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27f29b79cf9b42f4a5b9838965bbb72d" member_type: VOTER last_known_addr { host: "127.9.151.129" port: 33313 } }
I20240503 02:14:41.181739 14438 catalog_manager.cc:5484] T 6069a864c5ff43348881ff477218578d P 27f29b79cf9b42f4a5b9838965bbb72d reported cstate change: term changed from 1 to 2. New cstate: current_term: 2 leader_uuid: "27f29b79cf9b42f4a5b9838965bbb72d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27f29b79cf9b42f4a5b9838965bbb72d" member_type: VOTER last_known_addr { host: "127.9.151.129" port: 33313 } health_report { overall_health: HEALTHY } } }
I20240503 02:14:41.354918 14758 tablet_bootstrap.cc:492] T cefdd1e872814ddbb522f4013d55b26c P 27f29b79cf9b42f4a5b9838965bbb72d: Bootstrap replayed 1/1 log segments. Stats: ops{read=185 overwritten=0 applied=185 ignored=0} inserts{seen=486 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240503 02:14:41.355465 14758 tablet_bootstrap.cc:492] T cefdd1e872814ddbb522f4013d55b26c P 27f29b79cf9b42f4a5b9838965bbb72d: Bootstrap complete.
I20240503 02:14:41.356541 14758 ts_tablet_manager.cc:1367] T cefdd1e872814ddbb522f4013d55b26c P 27f29b79cf9b42f4a5b9838965bbb72d: Time spent bootstrapping tablet: real 0.179s user 0.145s sys 0.032s
I20240503 02:14:41.357192 14758 raft_consensus.cc:348] T cefdd1e872814ddbb522f4013d55b26c P 27f29b79cf9b42f4a5b9838965bbb72d [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27f29b79cf9b42f4a5b9838965bbb72d" member_type: VOTER last_known_addr { host: "127.9.151.129" port: 33313 } }
I20240503 02:14:41.357368 14758 raft_consensus.cc:729] T cefdd1e872814ddbb522f4013d55b26c P 27f29b79cf9b42f4a5b9838965bbb72d [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 27f29b79cf9b42f4a5b9838965bbb72d, State: Initialized, Role: FOLLOWER
I20240503 02:14:41.357623 14758 consensus_queue.cc:260] T cefdd1e872814ddbb522f4013d55b26c P 27f29b79cf9b42f4a5b9838965bbb72d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 185, Last appended: 1.185, Last appended by leader: 185, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27f29b79cf9b42f4a5b9838965bbb72d" member_type: VOTER last_known_addr { host: "127.9.151.129" port: 33313 } }
I20240503 02:14:41.357792 14758 raft_consensus.cc:388] T cefdd1e872814ddbb522f4013d55b26c P 27f29b79cf9b42f4a5b9838965bbb72d [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20240503 02:14:41.357879 14758 raft_consensus.cc:482] T cefdd1e872814ddbb522f4013d55b26c P 27f29b79cf9b42f4a5b9838965bbb72d [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20240503 02:14:41.357992 14758 raft_consensus.cc:3037] T cefdd1e872814ddbb522f4013d55b26c P 27f29b79cf9b42f4a5b9838965bbb72d [term 1 FOLLOWER]: Advancing to term 2
I20240503 02:14:41.360035 14758 raft_consensus.cc:504] T cefdd1e872814ddbb522f4013d55b26c P 27f29b79cf9b42f4a5b9838965bbb72d [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27f29b79cf9b42f4a5b9838965bbb72d" member_type: VOTER last_known_addr { host: "127.9.151.129" port: 33313 } }
I20240503 02:14:41.360284 14758 leader_election.cc:302] T cefdd1e872814ddbb522f4013d55b26c P 27f29b79cf9b42f4a5b9838965bbb72d [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 27f29b79cf9b42f4a5b9838965bbb72d; no voters:
I20240503 02:14:41.360484 14758 leader_election.cc:288] T cefdd1e872814ddbb522f4013d55b26c P 27f29b79cf9b42f4a5b9838965bbb72d [CANDIDATE]: Term 2 election: Requested vote from peers
I20240503 02:14:41.360613 14777 raft_consensus.cc:2781] T cefdd1e872814ddbb522f4013d55b26c P 27f29b79cf9b42f4a5b9838965bbb72d [term 2 FOLLOWER]: Leader election won for term 2
I20240503 02:14:41.360986 14758 ts_tablet_manager.cc:1398] T cefdd1e872814ddbb522f4013d55b26c P 27f29b79cf9b42f4a5b9838965bbb72d: Time spent starting tablet: real 0.004s user 0.000s sys 0.004s
I20240503 02:14:41.360842 14777 raft_consensus.cc:686] T cefdd1e872814ddbb522f4013d55b26c P 27f29b79cf9b42f4a5b9838965bbb72d [term 2 LEADER]: Becoming Leader. State: Replica: 27f29b79cf9b42f4a5b9838965bbb72d, State: Running, Role: LEADER
I20240503 02:14:41.361241 14758 tablet_bootstrap.cc:492] T ac0fc766d4b04fb1a7c996c57d383349 P 27f29b79cf9b42f4a5b9838965bbb72d: Bootstrap starting.
I20240503 02:14:41.361343 14777 consensus_queue.cc:237] T cefdd1e872814ddbb522f4013d55b26c P 27f29b79cf9b42f4a5b9838965bbb72d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 185, Committed index: 185, Last appended: 1.185, Last appended by leader: 185, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27f29b79cf9b42f4a5b9838965bbb72d" member_type: VOTER last_known_addr { host: "127.9.151.129" port: 33313 } }
I20240503 02:14:41.366092 14438 catalog_manager.cc:5484] T cefdd1e872814ddbb522f4013d55b26c P 27f29b79cf9b42f4a5b9838965bbb72d reported cstate change: term changed from 1 to 2. New cstate: current_term: 2 leader_uuid: "27f29b79cf9b42f4a5b9838965bbb72d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27f29b79cf9b42f4a5b9838965bbb72d" member_type: VOTER last_known_addr { host: "127.9.151.129" port: 33313 } health_report { overall_health: HEALTHY } } }
I20240503 02:14:41.540809 14758 tablet_bootstrap.cc:492] T ac0fc766d4b04fb1a7c996c57d383349 P 27f29b79cf9b42f4a5b9838965bbb72d: Bootstrap replayed 1/1 log segments. Stats: ops{read=189 overwritten=0 applied=189 ignored=0} inserts{seen=516 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240503 02:14:41.541281 14758 tablet_bootstrap.cc:492] T ac0fc766d4b04fb1a7c996c57d383349 P 27f29b79cf9b42f4a5b9838965bbb72d: Bootstrap complete.
I20240503 02:14:41.542217 14758 ts_tablet_manager.cc:1367] T ac0fc766d4b04fb1a7c996c57d383349 P 27f29b79cf9b42f4a5b9838965bbb72d: Time spent bootstrapping tablet: real 0.181s user 0.158s sys 0.020s
I20240503 02:14:41.542934 14758 raft_consensus.cc:348] T ac0fc766d4b04fb1a7c996c57d383349 P 27f29b79cf9b42f4a5b9838965bbb72d [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27f29b79cf9b42f4a5b9838965bbb72d" member_type: VOTER last_known_addr { host: "127.9.151.129" port: 33313 } }
I20240503 02:14:41.543098 14758 raft_consensus.cc:729] T ac0fc766d4b04fb1a7c996c57d383349 P 27f29b79cf9b42f4a5b9838965bbb72d [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 27f29b79cf9b42f4a5b9838965bbb72d, State: Initialized, Role: FOLLOWER
I20240503 02:14:41.543339 14758 consensus_queue.cc:260] T ac0fc766d4b04fb1a7c996c57d383349 P 27f29b79cf9b42f4a5b9838965bbb72d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 189, Last appended: 1.189, Last appended by leader: 189, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27f29b79cf9b42f4a5b9838965bbb72d" member_type: VOTER last_known_addr { host: "127.9.151.129" port: 33313 } }
I20240503 02:14:41.543479 14758 raft_consensus.cc:388] T ac0fc766d4b04fb1a7c996c57d383349 P 27f29b79cf9b42f4a5b9838965bbb72d [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20240503 02:14:41.543541 14758 raft_consensus.cc:482] T ac0fc766d4b04fb1a7c996c57d383349 P 27f29b79cf9b42f4a5b9838965bbb72d [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20240503 02:14:41.543603 14758 raft_consensus.cc:3037] T ac0fc766d4b04fb1a7c996c57d383349 P 27f29b79cf9b42f4a5b9838965bbb72d [term 1 FOLLOWER]: Advancing to term 2
I20240503 02:14:41.545426 14758 raft_consensus.cc:504] T ac0fc766d4b04fb1a7c996c57d383349 P 27f29b79cf9b42f4a5b9838965bbb72d [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27f29b79cf9b42f4a5b9838965bbb72d" member_type: VOTER last_known_addr { host: "127.9.151.129" port: 33313 } }
I20240503 02:14:41.545642 14758 leader_election.cc:302] T ac0fc766d4b04fb1a7c996c57d383349 P 27f29b79cf9b42f4a5b9838965bbb72d [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 27f29b79cf9b42f4a5b9838965bbb72d; no voters:
I20240503 02:14:41.545833 14758 leader_election.cc:288] T ac0fc766d4b04fb1a7c996c57d383349 P 27f29b79cf9b42f4a5b9838965bbb72d [CANDIDATE]: Term 2 election: Requested vote from peers
I20240503 02:14:41.545923 14777 raft_consensus.cc:2781] T ac0fc766d4b04fb1a7c996c57d383349 P 27f29b79cf9b42f4a5b9838965bbb72d [term 2 FOLLOWER]: Leader election won for term 2
I20240503 02:14:41.546329 14758 ts_tablet_manager.cc:1398] T ac0fc766d4b04fb1a7c996c57d383349 P 27f29b79cf9b42f4a5b9838965bbb72d: Time spent starting tablet: real 0.004s user 0.004s sys 0.000s
I20240503 02:14:41.546527 14777 raft_consensus.cc:686] T ac0fc766d4b04fb1a7c996c57d383349 P 27f29b79cf9b42f4a5b9838965bbb72d [term 2 LEADER]: Becoming Leader. State: Replica: 27f29b79cf9b42f4a5b9838965bbb72d, State: Running, Role: LEADER
I20240503 02:14:41.546633 14758 tablet_bootstrap.cc:492] T 02b52f763d8a412788ceabba8278d71f P 27f29b79cf9b42f4a5b9838965bbb72d: Bootstrap starting.
I20240503 02:14:41.546867 14777 consensus_queue.cc:237] T ac0fc766d4b04fb1a7c996c57d383349 P 27f29b79cf9b42f4a5b9838965bbb72d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 189, Committed index: 189, Last appended: 1.189, Last appended by leader: 189, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27f29b79cf9b42f4a5b9838965bbb72d" member_type: VOTER last_known_addr { host: "127.9.151.129" port: 33313 } }
I20240503 02:14:41.549818 14438 catalog_manager.cc:5484] T ac0fc766d4b04fb1a7c996c57d383349 P 27f29b79cf9b42f4a5b9838965bbb72d reported cstate change: term changed from 1 to 2. New cstate: current_term: 2 leader_uuid: "27f29b79cf9b42f4a5b9838965bbb72d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27f29b79cf9b42f4a5b9838965bbb72d" member_type: VOTER last_known_addr { host: "127.9.151.129" port: 33313 } health_report { overall_health: HEALTHY } } }
I20240503 02:14:41.733748 14758 tablet_bootstrap.cc:492] T 02b52f763d8a412788ceabba8278d71f P 27f29b79cf9b42f4a5b9838965bbb72d: Bootstrap replayed 1/1 log segments. Stats: ops{read=192 overwritten=0 applied=192 ignored=0} inserts{seen=534 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240503 02:14:41.734727 14758 tablet_bootstrap.cc:492] T 02b52f763d8a412788ceabba8278d71f P 27f29b79cf9b42f4a5b9838965bbb72d: Bootstrap complete.
I20240503 02:14:41.735806 14758 ts_tablet_manager.cc:1367] T 02b52f763d8a412788ceabba8278d71f P 27f29b79cf9b42f4a5b9838965bbb72d: Time spent bootstrapping tablet: real 0.189s user 0.160s sys 0.028s
I20240503 02:14:41.736562 14758 raft_consensus.cc:348] T 02b52f763d8a412788ceabba8278d71f P 27f29b79cf9b42f4a5b9838965bbb72d [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27f29b79cf9b42f4a5b9838965bbb72d" member_type: VOTER last_known_addr { host: "127.9.151.129" port: 33313 } }
I20240503 02:14:41.736785 14758 raft_consensus.cc:729] T 02b52f763d8a412788ceabba8278d71f P 27f29b79cf9b42f4a5b9838965bbb72d [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 27f29b79cf9b42f4a5b9838965bbb72d, State: Initialized, Role: FOLLOWER
I20240503 02:14:41.737025 14758 consensus_queue.cc:260] T 02b52f763d8a412788ceabba8278d71f P 27f29b79cf9b42f4a5b9838965bbb72d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 192, Last appended: 1.192, Last appended by leader: 192, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27f29b79cf9b42f4a5b9838965bbb72d" member_type: VOTER last_known_addr { host: "127.9.151.129" port: 33313 } }
I20240503 02:14:41.737219 14758 raft_consensus.cc:388] T 02b52f763d8a412788ceabba8278d71f P 27f29b79cf9b42f4a5b9838965bbb72d [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20240503 02:14:41.737272 14758 raft_consensus.cc:482] T 02b52f763d8a412788ceabba8278d71f P 27f29b79cf9b42f4a5b9838965bbb72d [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20240503 02:14:41.737356 14758 raft_consensus.cc:3037] T 02b52f763d8a412788ceabba8278d71f P 27f29b79cf9b42f4a5b9838965bbb72d [term 1 FOLLOWER]: Advancing to term 2
I20240503 02:14:41.739223 14758 raft_consensus.cc:504] T 02b52f763d8a412788ceabba8278d71f P 27f29b79cf9b42f4a5b9838965bbb72d [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27f29b79cf9b42f4a5b9838965bbb72d" member_type: VOTER last_known_addr { host: "127.9.151.129" port: 33313 } }
I20240503 02:14:41.739440 14758 leader_election.cc:302] T 02b52f763d8a412788ceabba8278d71f P 27f29b79cf9b42f4a5b9838965bbb72d [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 27f29b79cf9b42f4a5b9838965bbb72d; no voters:
I20240503 02:14:41.739615 14758 leader_election.cc:288] T 02b52f763d8a412788ceabba8278d71f P 27f29b79cf9b42f4a5b9838965bbb72d [CANDIDATE]: Term 2 election: Requested vote from peers
I20240503 02:14:41.739988 14777 raft_consensus.cc:2781] T 02b52f763d8a412788ceabba8278d71f P 27f29b79cf9b42f4a5b9838965bbb72d [term 2 FOLLOWER]: Leader election won for term 2
I20240503 02:14:41.740186 14758 ts_tablet_manager.cc:1398] T 02b52f763d8a412788ceabba8278d71f P 27f29b79cf9b42f4a5b9838965bbb72d: Time spent starting tablet: real 0.004s user 0.004s sys 0.000s
I20240503 02:14:41.740227 14777 raft_consensus.cc:686] T 02b52f763d8a412788ceabba8278d71f P 27f29b79cf9b42f4a5b9838965bbb72d [term 2 LEADER]: Becoming Leader. State: Replica: 27f29b79cf9b42f4a5b9838965bbb72d, State: Running, Role: LEADER
I20240503 02:14:41.740458 14758 tablet_bootstrap.cc:492] T 4ff002252c4d46d2af2a4956acd90b3f P 27f29b79cf9b42f4a5b9838965bbb72d: Bootstrap starting.
I20240503 02:14:41.740640 14777 consensus_queue.cc:237] T 02b52f763d8a412788ceabba8278d71f P 27f29b79cf9b42f4a5b9838965bbb72d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 192, Committed index: 192, Last appended: 1.192, Last appended by leader: 192, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27f29b79cf9b42f4a5b9838965bbb72d" member_type: VOTER last_known_addr { host: "127.9.151.129" port: 33313 } }
I20240503 02:14:41.743414 14438 catalog_manager.cc:5484] T 02b52f763d8a412788ceabba8278d71f P 27f29b79cf9b42f4a5b9838965bbb72d reported cstate change: term changed from 1 to 2. New cstate: current_term: 2 leader_uuid: "27f29b79cf9b42f4a5b9838965bbb72d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27f29b79cf9b42f4a5b9838965bbb72d" member_type: VOTER last_known_addr { host: "127.9.151.129" port: 33313 } health_report { overall_health: HEALTHY } } }
I20240503 02:14:41.960994 14758 tablet_bootstrap.cc:492] T 4ff002252c4d46d2af2a4956acd90b3f P 27f29b79cf9b42f4a5b9838965bbb72d: Bootstrap replayed 1/1 log segments. Stats: ops{read=192 overwritten=0 applied=192 ignored=0} inserts{seen=500 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240503 02:14:41.961514 14758 tablet_bootstrap.cc:492] T 4ff002252c4d46d2af2a4956acd90b3f P 27f29b79cf9b42f4a5b9838965bbb72d: Bootstrap complete.
I20240503 02:14:41.962499 14758 ts_tablet_manager.cc:1367] T 4ff002252c4d46d2af2a4956acd90b3f P 27f29b79cf9b42f4a5b9838965bbb72d: Time spent bootstrapping tablet: real 0.222s user 0.174s sys 0.012s
I20240503 02:14:41.963222 14758 raft_consensus.cc:348] T 4ff002252c4d46d2af2a4956acd90b3f P 27f29b79cf9b42f4a5b9838965bbb72d [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27f29b79cf9b42f4a5b9838965bbb72d" member_type: VOTER last_known_addr { host: "127.9.151.129" port: 33313 } }
I20240503 02:14:41.963376 14758 raft_consensus.cc:729] T 4ff002252c4d46d2af2a4956acd90b3f P 27f29b79cf9b42f4a5b9838965bbb72d [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 27f29b79cf9b42f4a5b9838965bbb72d, State: Initialized, Role: FOLLOWER
I20240503 02:14:41.963608 14758 consensus_queue.cc:260] T 4ff002252c4d46d2af2a4956acd90b3f P 27f29b79cf9b42f4a5b9838965bbb72d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 192, Last appended: 1.192, Last appended by leader: 192, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27f29b79cf9b42f4a5b9838965bbb72d" member_type: VOTER last_known_addr { host: "127.9.151.129" port: 33313 } }
I20240503 02:14:41.963738 14758 raft_consensus.cc:388] T 4ff002252c4d46d2af2a4956acd90b3f P 27f29b79cf9b42f4a5b9838965bbb72d [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20240503 02:14:41.963807 14758 raft_consensus.cc:482] T 4ff002252c4d46d2af2a4956acd90b3f P 27f29b79cf9b42f4a5b9838965bbb72d [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20240503 02:14:41.963909 14758 raft_consensus.cc:3037] T 4ff002252c4d46d2af2a4956acd90b3f P 27f29b79cf9b42f4a5b9838965bbb72d [term 1 FOLLOWER]: Advancing to term 2
I20240503 02:14:41.966039 14758 raft_consensus.cc:504] T 4ff002252c4d46d2af2a4956acd90b3f P 27f29b79cf9b42f4a5b9838965bbb72d [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27f29b79cf9b42f4a5b9838965bbb72d" member_type: VOTER last_known_addr { host: "127.9.151.129" port: 33313 } }
I20240503 02:14:41.966284 14758 leader_election.cc:302] T 4ff002252c4d46d2af2a4956acd90b3f P 27f29b79cf9b42f4a5b9838965bbb72d [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 27f29b79cf9b42f4a5b9838965bbb72d; no voters:
I20240503 02:14:41.966495 14758 leader_election.cc:288] T 4ff002252c4d46d2af2a4956acd90b3f P 27f29b79cf9b42f4a5b9838965bbb72d [CANDIDATE]: Term 2 election: Requested vote from peers
I20240503 02:14:41.966568 14777 raft_consensus.cc:2781] T 4ff002252c4d46d2af2a4956acd90b3f P 27f29b79cf9b42f4a5b9838965bbb72d [term 2 FOLLOWER]: Leader election won for term 2
I20240503 02:14:41.966814 14777 raft_consensus.cc:686] T 4ff002252c4d46d2af2a4956acd90b3f P 27f29b79cf9b42f4a5b9838965bbb72d [term 2 LEADER]: Becoming Leader. State: Replica: 27f29b79cf9b42f4a5b9838965bbb72d, State: Running, Role: LEADER
I20240503 02:14:41.967046 14758 ts_tablet_manager.cc:1398] T 4ff002252c4d46d2af2a4956acd90b3f P 27f29b79cf9b42f4a5b9838965bbb72d: Time spent starting tablet: real 0.004s user 0.008s sys 0.000s
I20240503 02:14:41.967087 14777 consensus_queue.cc:237] T 4ff002252c4d46d2af2a4956acd90b3f P 27f29b79cf9b42f4a5b9838965bbb72d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 192, Committed index: 192, Last appended: 1.192, Last appended by leader: 192, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27f29b79cf9b42f4a5b9838965bbb72d" member_type: VOTER last_known_addr { host: "127.9.151.129" port: 33313 } }
I20240503 02:14:41.967339 14758 tablet_bootstrap.cc:492] T b26fbd5ae3384b6cb444331bd8e75aef P 27f29b79cf9b42f4a5b9838965bbb72d: Bootstrap starting.
I20240503 02:14:41.970109 14438 catalog_manager.cc:5484] T 4ff002252c4d46d2af2a4956acd90b3f P 27f29b79cf9b42f4a5b9838965bbb72d reported cstate change: term changed from 1 to 2. New cstate: current_term: 2 leader_uuid: "27f29b79cf9b42f4a5b9838965bbb72d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27f29b79cf9b42f4a5b9838965bbb72d" member_type: VOTER last_known_addr { host: "127.9.151.129" port: 33313 } health_report { overall_health: HEALTHY } } }
I20240503 02:14:42.147603 14758 tablet_bootstrap.cc:492] T b26fbd5ae3384b6cb444331bd8e75aef P 27f29b79cf9b42f4a5b9838965bbb72d: Bootstrap replayed 1/1 log segments. Stats: ops{read=186 overwritten=0 applied=186 ignored=0} inserts{seen=502 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240503 02:14:42.148087 14758 tablet_bootstrap.cc:492] T b26fbd5ae3384b6cb444331bd8e75aef P 27f29b79cf9b42f4a5b9838965bbb72d: Bootstrap complete.
I20240503 02:14:42.149032 14758 ts_tablet_manager.cc:1367] T b26fbd5ae3384b6cb444331bd8e75aef P 27f29b79cf9b42f4a5b9838965bbb72d: Time spent bootstrapping tablet: real 0.182s user 0.155s sys 0.024s
I20240503 02:14:42.149669 14758 raft_consensus.cc:348] T b26fbd5ae3384b6cb444331bd8e75aef P 27f29b79cf9b42f4a5b9838965bbb72d [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27f29b79cf9b42f4a5b9838965bbb72d" member_type: VOTER last_known_addr { host: "127.9.151.129" port: 33313 } }
I20240503 02:14:42.149838 14758 raft_consensus.cc:729] T b26fbd5ae3384b6cb444331bd8e75aef P 27f29b79cf9b42f4a5b9838965bbb72d [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 27f29b79cf9b42f4a5b9838965bbb72d, State: Initialized, Role: FOLLOWER
I20240503 02:14:42.150053 14758 consensus_queue.cc:260] T b26fbd5ae3384b6cb444331bd8e75aef P 27f29b79cf9b42f4a5b9838965bbb72d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 186, Last appended: 1.186, Last appended by leader: 186, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27f29b79cf9b42f4a5b9838965bbb72d" member_type: VOTER last_known_addr { host: "127.9.151.129" port: 33313 } }
I20240503 02:14:42.150243 14758 raft_consensus.cc:388] T b26fbd5ae3384b6cb444331bd8e75aef P 27f29b79cf9b42f4a5b9838965bbb72d [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20240503 02:14:42.150305 14758 raft_consensus.cc:482] T b26fbd5ae3384b6cb444331bd8e75aef P 27f29b79cf9b42f4a5b9838965bbb72d [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20240503 02:14:42.150386 14758 raft_consensus.cc:3037] T b26fbd5ae3384b6cb444331bd8e75aef P 27f29b79cf9b42f4a5b9838965bbb72d [term 1 FOLLOWER]: Advancing to term 2
I20240503 02:14:42.152429 14758 raft_consensus.cc:504] T b26fbd5ae3384b6cb444331bd8e75aef P 27f29b79cf9b42f4a5b9838965bbb72d [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27f29b79cf9b42f4a5b9838965bbb72d" member_type: VOTER last_known_addr { host: "127.9.151.129" port: 33313 } }
I20240503 02:14:42.152689 14758 leader_election.cc:302] T b26fbd5ae3384b6cb444331bd8e75aef P 27f29b79cf9b42f4a5b9838965bbb72d [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 27f29b79cf9b42f4a5b9838965bbb72d; no voters:
I20240503 02:14:42.152906 14758 leader_election.cc:288] T b26fbd5ae3384b6cb444331bd8e75aef P 27f29b79cf9b42f4a5b9838965bbb72d [CANDIDATE]: Term 2 election: Requested vote from peers
I20240503 02:14:42.152987 14777 raft_consensus.cc:2781] T b26fbd5ae3384b6cb444331bd8e75aef P 27f29b79cf9b42f4a5b9838965bbb72d [term 2 FOLLOWER]: Leader election won for term 2
I20240503 02:14:42.153143 14777 raft_consensus.cc:686] T b26fbd5ae3384b6cb444331bd8e75aef P 27f29b79cf9b42f4a5b9838965bbb72d [term 2 LEADER]: Becoming Leader. State: Replica: 27f29b79cf9b42f4a5b9838965bbb72d, State: Running, Role: LEADER
I20240503 02:14:42.153466 14758 ts_tablet_manager.cc:1398] T b26fbd5ae3384b6cb444331bd8e75aef P 27f29b79cf9b42f4a5b9838965bbb72d: Time spent starting tablet: real 0.004s user 0.004s sys 0.000s
I20240503 02:14:42.153465 14777 consensus_queue.cc:237] T b26fbd5ae3384b6cb444331bd8e75aef P 27f29b79cf9b42f4a5b9838965bbb72d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 186, Committed index: 186, Last appended: 1.186, Last appended by leader: 186, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27f29b79cf9b42f4a5b9838965bbb72d" member_type: VOTER last_known_addr { host: "127.9.151.129" port: 33313 } }
I20240503 02:14:42.153858 14758 tablet_bootstrap.cc:492] T 9d03aee673644af6a0b56ffdc5fb1e99 P 27f29b79cf9b42f4a5b9838965bbb72d: Bootstrap starting.
I20240503 02:14:42.156391 14438 catalog_manager.cc:5484] T b26fbd5ae3384b6cb444331bd8e75aef P 27f29b79cf9b42f4a5b9838965bbb72d reported cstate change: term changed from 1 to 2. New cstate: current_term: 2 leader_uuid: "27f29b79cf9b42f4a5b9838965bbb72d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27f29b79cf9b42f4a5b9838965bbb72d" member_type: VOTER last_known_addr { host: "127.9.151.129" port: 33313 } health_report { overall_health: HEALTHY } } }
I20240503 02:14:42.340150 14758 tablet_bootstrap.cc:492] T 9d03aee673644af6a0b56ffdc5fb1e99 P 27f29b79cf9b42f4a5b9838965bbb72d: Bootstrap replayed 1/1 log segments. Stats: ops{read=195 overwritten=0 applied=195 ignored=0} inserts{seen=525 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240503 02:14:42.340709 14758 tablet_bootstrap.cc:492] T 9d03aee673644af6a0b56ffdc5fb1e99 P 27f29b79cf9b42f4a5b9838965bbb72d: Bootstrap complete.
I20240503 02:14:42.341676 14758 ts_tablet_manager.cc:1367] T 9d03aee673644af6a0b56ffdc5fb1e99 P 27f29b79cf9b42f4a5b9838965bbb72d: Time spent bootstrapping tablet: real 0.188s user 0.165s sys 0.020s
I20240503 02:14:42.342341 14758 raft_consensus.cc:348] T 9d03aee673644af6a0b56ffdc5fb1e99 P 27f29b79cf9b42f4a5b9838965bbb72d [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27f29b79cf9b42f4a5b9838965bbb72d" member_type: VOTER last_known_addr { host: "127.9.151.129" port: 33313 } }
I20240503 02:14:42.342522 14758 raft_consensus.cc:729] T 9d03aee673644af6a0b56ffdc5fb1e99 P 27f29b79cf9b42f4a5b9838965bbb72d [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 27f29b79cf9b42f4a5b9838965bbb72d, State: Initialized, Role: FOLLOWER
I20240503 02:14:42.342821 14758 consensus_queue.cc:260] T 9d03aee673644af6a0b56ffdc5fb1e99 P 27f29b79cf9b42f4a5b9838965bbb72d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 195, Last appended: 1.195, Last appended by leader: 195, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27f29b79cf9b42f4a5b9838965bbb72d" member_type: VOTER last_known_addr { host: "127.9.151.129" port: 33313 } }
I20240503 02:14:42.342962 14758 raft_consensus.cc:388] T 9d03aee673644af6a0b56ffdc5fb1e99 P 27f29b79cf9b42f4a5b9838965bbb72d [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20240503 02:14:42.343060 14758 raft_consensus.cc:482] T 9d03aee673644af6a0b56ffdc5fb1e99 P 27f29b79cf9b42f4a5b9838965bbb72d [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20240503 02:14:42.343152 14758 raft_consensus.cc:3037] T 9d03aee673644af6a0b56ffdc5fb1e99 P 27f29b79cf9b42f4a5b9838965bbb72d [term 1 FOLLOWER]: Advancing to term 2
I20240503 02:14:42.345016 14758 raft_consensus.cc:504] T 9d03aee673644af6a0b56ffdc5fb1e99 P 27f29b79cf9b42f4a5b9838965bbb72d [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27f29b79cf9b42f4a5b9838965bbb72d" member_type: VOTER last_known_addr { host: "127.9.151.129" port: 33313 } }
I20240503 02:14:42.345263 14758 leader_election.cc:302] T 9d03aee673644af6a0b56ffdc5fb1e99 P 27f29b79cf9b42f4a5b9838965bbb72d [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 27f29b79cf9b42f4a5b9838965bbb72d; no voters:
I20240503 02:14:42.345681 14777 raft_consensus.cc:2781] T 9d03aee673644af6a0b56ffdc5fb1e99 P 27f29b79cf9b42f4a5b9838965bbb72d [term 2 FOLLOWER]: Leader election won for term 2
I20240503 02:14:42.345669 14758 leader_election.cc:288] T 9d03aee673644af6a0b56ffdc5fb1e99 P 27f29b79cf9b42f4a5b9838965bbb72d [CANDIDATE]: Term 2 election: Requested vote from peers
I20240503 02:14:42.345934 14777 raft_consensus.cc:686] T 9d03aee673644af6a0b56ffdc5fb1e99 P 27f29b79cf9b42f4a5b9838965bbb72d [term 2 LEADER]: Becoming Leader. State: Replica: 27f29b79cf9b42f4a5b9838965bbb72d, State: Running, Role: LEADER
I20240503 02:14:42.346186 14777 consensus_queue.cc:237] T 9d03aee673644af6a0b56ffdc5fb1e99 P 27f29b79cf9b42f4a5b9838965bbb72d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 195, Committed index: 195, Last appended: 1.195, Last appended by leader: 195, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27f29b79cf9b42f4a5b9838965bbb72d" member_type: VOTER last_known_addr { host: "127.9.151.129" port: 33313 } }
I20240503 02:14:42.346304 14758 ts_tablet_manager.cc:1398] T 9d03aee673644af6a0b56ffdc5fb1e99 P 27f29b79cf9b42f4a5b9838965bbb72d: Time spent starting tablet: real 0.005s user 0.007s sys 0.000s
I20240503 02:14:42.346544 14758 tablet_bootstrap.cc:492] T ef909312fe9545b18bf9f918f52928c2 P 27f29b79cf9b42f4a5b9838965bbb72d: Bootstrap starting.
I20240503 02:14:42.348793 14438 catalog_manager.cc:5484] T 9d03aee673644af6a0b56ffdc5fb1e99 P 27f29b79cf9b42f4a5b9838965bbb72d reported cstate change: term changed from 1 to 2. New cstate: current_term: 2 leader_uuid: "27f29b79cf9b42f4a5b9838965bbb72d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27f29b79cf9b42f4a5b9838965bbb72d" member_type: VOTER last_known_addr { host: "127.9.151.129" port: 33313 } health_report { overall_health: HEALTHY } } }
I20240503 02:14:42.545291 14758 tablet_bootstrap.cc:492] T ef909312fe9545b18bf9f918f52928c2 P 27f29b79cf9b42f4a5b9838965bbb72d: Bootstrap replayed 1/1 log segments. Stats: ops{read=193 overwritten=0 applied=193 ignored=0} inserts{seen=507 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240503 02:14:42.545794 14758 tablet_bootstrap.cc:492] T ef909312fe9545b18bf9f918f52928c2 P 27f29b79cf9b42f4a5b9838965bbb72d: Bootstrap complete.
I20240503 02:14:42.546808 14758 ts_tablet_manager.cc:1367] T ef909312fe9545b18bf9f918f52928c2 P 27f29b79cf9b42f4a5b9838965bbb72d: Time spent bootstrapping tablet: real 0.200s user 0.180s sys 0.020s
I20240503 02:14:42.547420 14758 raft_consensus.cc:348] T ef909312fe9545b18bf9f918f52928c2 P 27f29b79cf9b42f4a5b9838965bbb72d [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27f29b79cf9b42f4a5b9838965bbb72d" member_type: VOTER last_known_addr { host: "127.9.151.129" port: 33313 } }
I20240503 02:14:42.547605 14758 raft_consensus.cc:729] T ef909312fe9545b18bf9f918f52928c2 P 27f29b79cf9b42f4a5b9838965bbb72d [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 27f29b79cf9b42f4a5b9838965bbb72d, State: Initialized, Role: FOLLOWER
I20240503 02:14:42.547858 14758 consensus_queue.cc:260] T ef909312fe9545b18bf9f918f52928c2 P 27f29b79cf9b42f4a5b9838965bbb72d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 193, Last appended: 1.193, Last appended by leader: 193, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27f29b79cf9b42f4a5b9838965bbb72d" member_type: VOTER last_known_addr { host: "127.9.151.129" port: 33313 } }
I20240503 02:14:42.548029 14758 raft_consensus.cc:388] T ef909312fe9545b18bf9f918f52928c2 P 27f29b79cf9b42f4a5b9838965bbb72d [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20240503 02:14:42.548115 14758 raft_consensus.cc:482] T ef909312fe9545b18bf9f918f52928c2 P 27f29b79cf9b42f4a5b9838965bbb72d [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20240503 02:14:42.548224 14758 raft_consensus.cc:3037] T ef909312fe9545b18bf9f918f52928c2 P 27f29b79cf9b42f4a5b9838965bbb72d [term 1 FOLLOWER]: Advancing to term 2
I20240503 02:14:42.550377 14758 raft_consensus.cc:504] T ef909312fe9545b18bf9f918f52928c2 P 27f29b79cf9b42f4a5b9838965bbb72d [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27f29b79cf9b42f4a5b9838965bbb72d" member_type: VOTER last_known_addr { host: "127.9.151.129" port: 33313 } }
I20240503 02:14:42.550642 14758 leader_election.cc:302] T ef909312fe9545b18bf9f918f52928c2 P 27f29b79cf9b42f4a5b9838965bbb72d [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 27f29b79cf9b42f4a5b9838965bbb72d; no voters:
I20240503 02:14:42.550859 14758 leader_election.cc:288] T ef909312fe9545b18bf9f918f52928c2 P 27f29b79cf9b42f4a5b9838965bbb72d [CANDIDATE]: Term 2 election: Requested vote from peers
I20240503 02:14:42.550992 14777 raft_consensus.cc:2781] T ef909312fe9545b18bf9f918f52928c2 P 27f29b79cf9b42f4a5b9838965bbb72d [term 2 FOLLOWER]: Leader election won for term 2
I20240503 02:14:42.551213 14777 raft_consensus.cc:686] T ef909312fe9545b18bf9f918f52928c2 P 27f29b79cf9b42f4a5b9838965bbb72d [term 2 LEADER]: Becoming Leader. State: Replica: 27f29b79cf9b42f4a5b9838965bbb72d, State: Running, Role: LEADER
I20240503 02:14:42.551352 14758 ts_tablet_manager.cc:1398] T ef909312fe9545b18bf9f918f52928c2 P 27f29b79cf9b42f4a5b9838965bbb72d: Time spent starting tablet: real 0.004s user 0.004s sys 0.000s
I20240503 02:14:42.551492 14777 consensus_queue.cc:237] T ef909312fe9545b18bf9f918f52928c2 P 27f29b79cf9b42f4a5b9838965bbb72d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 193, Committed index: 193, Last appended: 1.193, Last appended by leader: 193, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27f29b79cf9b42f4a5b9838965bbb72d" member_type: VOTER last_known_addr { host: "127.9.151.129" port: 33313 } }
I20240503 02:14:42.551617 14758 tablet_bootstrap.cc:492] T be3ab5a5dd774bdba45e6d7061019195 P 27f29b79cf9b42f4a5b9838965bbb72d: Bootstrap starting.
I20240503 02:14:42.554457 14438 catalog_manager.cc:5484] T ef909312fe9545b18bf9f918f52928c2 P 27f29b79cf9b42f4a5b9838965bbb72d reported cstate change: term changed from 1 to 2. New cstate: current_term: 2 leader_uuid: "27f29b79cf9b42f4a5b9838965bbb72d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27f29b79cf9b42f4a5b9838965bbb72d" member_type: VOTER last_known_addr { host: "127.9.151.129" port: 33313 } health_report { overall_health: HEALTHY } } }
I20240503 02:14:42.734256 14758 tablet_bootstrap.cc:492] T be3ab5a5dd774bdba45e6d7061019195 P 27f29b79cf9b42f4a5b9838965bbb72d: Bootstrap replayed 1/1 log segments. Stats: ops{read=193 overwritten=0 applied=193 ignored=0} inserts{seen=506 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240503 02:14:42.734938 14758 tablet_bootstrap.cc:492] T be3ab5a5dd774bdba45e6d7061019195 P 27f29b79cf9b42f4a5b9838965bbb72d: Bootstrap complete.
I20240503 02:14:42.735919 14758 ts_tablet_manager.cc:1367] T be3ab5a5dd774bdba45e6d7061019195 P 27f29b79cf9b42f4a5b9838965bbb72d: Time spent bootstrapping tablet: real 0.184s user 0.151s sys 0.032s
I20240503 02:14:42.736629 14758 raft_consensus.cc:348] T be3ab5a5dd774bdba45e6d7061019195 P 27f29b79cf9b42f4a5b9838965bbb72d [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27f29b79cf9b42f4a5b9838965bbb72d" member_type: VOTER last_known_addr { host: "127.9.151.129" port: 33313 } }
I20240503 02:14:42.736837 14758 raft_consensus.cc:729] T be3ab5a5dd774bdba45e6d7061019195 P 27f29b79cf9b42f4a5b9838965bbb72d [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 27f29b79cf9b42f4a5b9838965bbb72d, State: Initialized, Role: FOLLOWER
I20240503 02:14:42.737141 14758 consensus_queue.cc:260] T be3ab5a5dd774bdba45e6d7061019195 P 27f29b79cf9b42f4a5b9838965bbb72d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 193, Last appended: 1.193, Last appended by leader: 193, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27f29b79cf9b42f4a5b9838965bbb72d" member_type: VOTER last_known_addr { host: "127.9.151.129" port: 33313 } }
I20240503 02:14:42.737308 14758 raft_consensus.cc:388] T be3ab5a5dd774bdba45e6d7061019195 P 27f29b79cf9b42f4a5b9838965bbb72d [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20240503 02:14:42.737409 14758 raft_consensus.cc:482] T be3ab5a5dd774bdba45e6d7061019195 P 27f29b79cf9b42f4a5b9838965bbb72d [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20240503 02:14:42.737548 14758 raft_consensus.cc:3037] T be3ab5a5dd774bdba45e6d7061019195 P 27f29b79cf9b42f4a5b9838965bbb72d [term 1 FOLLOWER]: Advancing to term 2
I20240503 02:14:42.739774 14758 raft_consensus.cc:504] T be3ab5a5dd774bdba45e6d7061019195 P 27f29b79cf9b42f4a5b9838965bbb72d [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27f29b79cf9b42f4a5b9838965bbb72d" member_type: VOTER last_known_addr { host: "127.9.151.129" port: 33313 } }
I20240503 02:14:42.740042 14758 leader_election.cc:302] T be3ab5a5dd774bdba45e6d7061019195 P 27f29b79cf9b42f4a5b9838965bbb72d [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 27f29b79cf9b42f4a5b9838965bbb72d; no voters:
I20240503 02:14:42.740249 14758 leader_election.cc:288] T be3ab5a5dd774bdba45e6d7061019195 P 27f29b79cf9b42f4a5b9838965bbb72d [CANDIDATE]: Term 2 election: Requested vote from peers
I20240503 02:14:42.740345 14777 raft_consensus.cc:2781] T be3ab5a5dd774bdba45e6d7061019195 P 27f29b79cf9b42f4a5b9838965bbb72d [term 2 FOLLOWER]: Leader election won for term 2
I20240503 02:14:42.740536 14777 raft_consensus.cc:686] T be3ab5a5dd774bdba45e6d7061019195 P 27f29b79cf9b42f4a5b9838965bbb72d [term 2 LEADER]: Becoming Leader. State: Replica: 27f29b79cf9b42f4a5b9838965bbb72d, State: Running, Role: LEADER
I20240503 02:14:42.740689 14758 ts_tablet_manager.cc:1398] T be3ab5a5dd774bdba45e6d7061019195 P 27f29b79cf9b42f4a5b9838965bbb72d: Time spent starting tablet: real 0.005s user 0.004s sys 0.000s
I20240503 02:14:42.740795 14777 consensus_queue.cc:237] T be3ab5a5dd774bdba45e6d7061019195 P 27f29b79cf9b42f4a5b9838965bbb72d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 193, Committed index: 193, Last appended: 1.193, Last appended by leader: 193, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27f29b79cf9b42f4a5b9838965bbb72d" member_type: VOTER last_known_addr { host: "127.9.151.129" port: 33313 } }
I20240503 02:14:42.743211 14438 catalog_manager.cc:5484] T be3ab5a5dd774bdba45e6d7061019195 P 27f29b79cf9b42f4a5b9838965bbb72d reported cstate change: term changed from 1 to 2. New cstate: current_term: 2 leader_uuid: "27f29b79cf9b42f4a5b9838965bbb72d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27f29b79cf9b42f4a5b9838965bbb72d" member_type: VOTER last_known_addr { host: "127.9.151.129" port: 33313 } health_report { overall_health: HEALTHY } } }
I20240503 02:14:42.782256 14854 ts_tablet_manager.cc:902] T 9d424d86fde04f2dbdd9434ee5755150 P 5f665335495c43dcbc53e072f6b1f44f: Initiating tablet copy from peer 27f29b79cf9b42f4a5b9838965bbb72d (127.9.151.129:33313)
I20240503 02:14:42.891079 14856 ts_tablet_manager.cc:902] T e50f84673efd4108859e8c1360cad36f P 5f665335495c43dcbc53e072f6b1f44f: Initiating tablet copy from peer 27f29b79cf9b42f4a5b9838965bbb72d (127.9.151.129:33313)
I20240503 02:14:42.927978 14855 ts_tablet_manager.cc:902] T 6069a864c5ff43348881ff477218578d P 5f665335495c43dcbc53e072f6b1f44f: Initiating tablet copy from peer 27f29b79cf9b42f4a5b9838965bbb72d (127.9.151.129:33313)
I20240503 02:14:42.928606 14853 ts_tablet_manager.cc:902] T 9d03aee673644af6a0b56ffdc5fb1e99 P 5f665335495c43dcbc53e072f6b1f44f: Initiating tablet copy from peer 27f29b79cf9b42f4a5b9838965bbb72d (127.9.151.129:33313)
I20240503 02:14:42.928138 14852 ts_tablet_manager.cc:902] T be3ab5a5dd774bdba45e6d7061019195 P 5f665335495c43dcbc53e072f6b1f44f: Initiating tablet copy from peer 27f29b79cf9b42f4a5b9838965bbb72d (127.9.151.129:33313)
I20240503 02:14:42.929667 14857 ts_tablet_manager.cc:902] T f884dde21ea246a792634883aed3cb82 P 5f665335495c43dcbc53e072f6b1f44f: Initiating tablet copy from peer 27f29b79cf9b42f4a5b9838965bbb72d (127.9.151.129:33313)
I20240503 02:14:42.929982 14860 ts_tablet_manager.cc:902] T fa681a37b30c4a0b879d3f3b184405fc P 5f665335495c43dcbc53e072f6b1f44f: Initiating tablet copy from peer 27f29b79cf9b42f4a5b9838965bbb72d (127.9.151.129:33313)
I20240503 02:14:42.932094 14864 ts_tablet_manager.cc:902] T a40f48ddab6e48d5ba3707ef78923ee5 P 5f665335495c43dcbc53e072f6b1f44f: Initiating tablet copy from peer 27f29b79cf9b42f4a5b9838965bbb72d (127.9.151.129:33313)
I20240503 02:14:42.932467 14851 ts_tablet_manager.cc:902] T ef909312fe9545b18bf9f918f52928c2 P 5f665335495c43dcbc53e072f6b1f44f: Initiating tablet copy from peer 27f29b79cf9b42f4a5b9838965bbb72d (127.9.151.129:33313)
I20240503 02:14:42.932713 14859 ts_tablet_manager.cc:902] T b26fbd5ae3384b6cb444331bd8e75aef P 5f665335495c43dcbc53e072f6b1f44f: Initiating tablet copy from peer 27f29b79cf9b42f4a5b9838965bbb72d (127.9.151.129:33313)
I20240503 02:14:42.936764 14869 ts_tablet_manager.cc:902] T 4ff002252c4d46d2af2a4956acd90b3f P 5f665335495c43dcbc53e072f6b1f44f: Initiating tablet copy from peer 27f29b79cf9b42f4a5b9838965bbb72d (127.9.151.129:33313)
I20240503 02:14:42.933202 14858 ts_tablet_manager.cc:902] T cefdd1e872814ddbb522f4013d55b26c P 5f665335495c43dcbc53e072f6b1f44f: Initiating tablet copy from peer 27f29b79cf9b42f4a5b9838965bbb72d (127.9.151.129:33313)
I20240503 02:14:42.933676 14861 ts_tablet_manager.cc:902] T b0df8ae119f948eaa5461faa42007713 P 5f665335495c43dcbc53e072f6b1f44f: Initiating tablet copy from peer 27f29b79cf9b42f4a5b9838965bbb72d (127.9.151.129:33313)
I20240503 02:14:42.934132 14862 ts_tablet_manager.cc:902] T ac0fc766d4b04fb1a7c996c57d383349 P 5f665335495c43dcbc53e072f6b1f44f: Initiating tablet copy from peer 27f29b79cf9b42f4a5b9838965bbb72d (127.9.151.129:33313)
I20240503 02:14:42.934564 14863 ts_tablet_manager.cc:902] T 97c56e26ef4f47f4acf50d0b8d8116dc P 5f665335495c43dcbc53e072f6b1f44f: Initiating tablet copy from peer 27f29b79cf9b42f4a5b9838965bbb72d (127.9.151.129:33313)
I20240503 02:14:42.934819 14865 ts_tablet_manager.cc:902] T fc8f5043623a48418283e5ecd50d58e2 P 5f665335495c43dcbc53e072f6b1f44f: Initiating tablet copy from peer 27f29b79cf9b42f4a5b9838965bbb72d (127.9.151.129:33313)
I20240503 02:14:42.935045 14866 ts_tablet_manager.cc:902] T 92cd1f3c9cdc43e0b9d56e3375488f50 P 5f665335495c43dcbc53e072f6b1f44f: Initiating tablet copy from peer 27f29b79cf9b42f4a5b9838965bbb72d (127.9.151.129:33313)
F20240503 02:14:42.940948 14866 trace_metrics.cc:68] Check failed: g_intern_map->size() < 100 (100 vs. 100) Too many interned strings: (MaintenanceMgr.queue_time_us, MaintenanceMgr.queue_time_us) (MaintenanceMgr.run_cpu_time_us, MaintenanceMgr.run_cpu_time_us) (MaintenanceMgr.run_wall_time_us, MaintenanceMgr.run_wall_time_us) (apply.queue_time_us, apply.queue_time_us) (apply.run_cpu_time_us, apply.run_cpu_time_us) (apply.run_wall_time_us, apply.run_wall_time_us) (client-negotiator.queue_time_us, client-negotiator.queue_time_us) (client-negotiator.run_cpu_time_us, client-negotiator.run_cpu_time_us) (client-negotiator.run_wall_time_us, client-negotiator.run_wall_time_us) (compiler_manager_pool.queue_time_us, compiler_manager_pool.queue_time_us) (compiler_manager_pool.run_cpu_time_us, compiler_manager_pool.run_cpu_time_us) (compiler_manager_pool.run_wall_time_us, compiler_manager_pool.run_wall_time_us) (dirs.queue_time_us, dirs.queue_time_us) (dirs.run_cpu_time_us, dirs.run_cpu_time_us) (dirs.run_wall_time_us, dirs.run_wall_time_us) (dns-resolver.queue_time_us, dns-resolver.queue_time_us) (dns-resolver.run_cpu_time_us, dns-resolver.run_cpu_time_us) (dns-resolver.run_wall_time_us, dns-resolver.run_wall_time_us) (prepare.queue_time_us, prepare.queue_time_us) (prepare.run_cpu_time_us, prepare.run_cpu_time_us) (prepare.run_wall_time_us, prepare.run_wall_time_us) (raft.queue_time_us, raft.queue_time_us) (raft.run_cpu_time_us, raft.run_cpu_time_us) (raft.run_wall_time_us, raft.run_wall_time_us) (repair_pool.queue_time_us, repair_pool.queue_time_us) (repair_pool.run_cpu_time_us, repair_pool.run_cpu_time_us) (repair_pool.run_wall_time_us, repair_pool.run_wall_time_us) (server-negotiator.queue_time_us, server-negotiator.queue_time_us) (server-negotiator.run_cpu_time_us, server-negotiator.run_cpu_time_us) (server-negotiator.run_wall_time_us, server-negotiator.run_wall_time_us) (tablet-copy.queue_time_us, tablet-copy.queue_time_us) (tablet-copy.run_cpu_time_us, tablet-copy.run_cpu_time_us) (tablet-copy.run_wall_time_us, tablet-copy.run_wall_time_us) (tablet-delete.queue_time_us, tablet-delete.queue_time_us) (tablet-delete.run_cpu_time_us, tablet-delete.run_cpu_time_us) (tablet-delete.run_wall_time_us, tablet-delete.run_wall_time_us) (tablet-download-pool-4ff002252c4d46d2af2a4956acd90b3f.queue_time_us, tablet-download-pool-4ff002252c4d46d2af2a4956acd90b3f.queue_time_us) (tablet-download-pool-4ff002252c4d46d2af2a4956acd90b3f.run_cpu_time_us, tablet-download-pool-4ff002252c4d46d2af2a4956acd90b3f.run_cpu_time_us) (tablet-download-pool-4ff002252c4d46d2af2a4956acd90b3f.run_wall_time_us, tablet-download-pool-4ff002252c4d46d2af2a4956acd90b3f.run_wall_time_us) (tablet-download-pool-6069a864c5ff43348881ff477218578d.queue_time_us, tablet-download-pool-6069a864c5ff43348881ff477218578d.queue_time_us) (tablet-download-pool-6069a864c5ff43348881ff477218578d.run_cpu_time_us, tablet-download-pool-6069a864c5ff43348881ff477218578d.run_cpu_time_us) (tablet-download-pool-6069a864c5ff43348881ff477218578d.run_wall_time_us, tablet-download-pool-6069a864c5ff43348881ff477218578d.run_wall_time_us) (tablet-download-pool-92cd1f3c9cdc43e0b9d56e3375488f50.queue_time_us, tablet-download-pool-92cd1f3c9cdc43e0b9d56e3375488f50.queue_time_us) (tablet-download-pool-97c56e26ef4f47f4acf50d0b8d8116dc.queue_time_us, tablet-download-pool-97c56e26ef4f47f4acf50d0b8d8116dc.queue_time_us) (tablet-download-pool-97c56e26ef4f47f4acf50d0b8d8116dc.run_cpu_time_us, tablet-download-pool-97c56e26ef4f47f4acf50d0b8d8116dc.run_cpu_time_us) (tablet-download-pool-97c56e26ef4f47f4acf50d0b8d8116dc.run_wall_time_us, tablet-download-pool-97c56e26ef4f47f4acf50d0b8d8116dc.run_wall_time_us) (tablet-download-pool-9d03aee673644af6a0b56ffdc5fb1e99.queue_time_us, tablet-download-pool-9d03aee673644af6a0b56ffdc5fb1e99.queue_time_us) (tablet-download-pool-9d03aee673644af6a0b56ffdc5fb1e99.run_cpu_time_us, tablet-download-pool-9d03aee673644af6a0b56ffdc5fb1e99.run_cpu_time_us) (tablet-download-pool-9d03aee673644af6a0b56ffdc5fb1e99.run_wall_time_us, tablet-download-pool-9d03aee673644af6a0b56ffdc5fb1e99.run_wall_time_us) (tablet-download-pool-9d424d86fde04f2dbdd9434ee5755150.queue_time_us, tablet-download-pool-9d424d86fde04f2dbdd9434ee5755150.queue_time_us) (tablet-download-pool-9d424d86fde04f2dbdd9434ee5755150.run_cpu_time_us, tablet-download-pool-9d424d86fde04f2dbdd9434ee5755150.run_cpu_time_us) (tablet-download-pool-9d424d86fde04f2dbdd9434ee5755150.run_wall_time_us, tablet-download-pool-9d424d86fde04f2dbdd9434ee5755150.run_wall_time_us) (tablet-download-pool-a40f48ddab6e48d5ba3707ef78923ee5.queue_time_us, tablet-download-pool-a40f48ddab6e48d5ba3707ef78923ee5.queue_time_us) (tablet-download-pool-a40f48ddab6e48d5ba3707ef78923ee5.run_cpu_time_us, tablet-download-pool-a40f48ddab6e48d5ba3707ef78923ee5.run_cpu_time_us) (tablet-download-pool-a40f48ddab6e48d5ba3707ef78923ee5.run_wall_time_us, tablet-download-pool-a40f48ddab6e48d5ba3707ef78923ee5.run_wall_time_us) (tablet-download-pool-ac0fc766d4b04fb1a7c996c57d383349.queue_time_us, tablet-download-pool-ac0fc766d4b04fb1a7c996c57d383349.queue_time_us) (tablet-download-pool-ac0fc766d4b04fb1a7c996c57d383349.run_cpu_time_us, tablet-download-pool-ac0fc766d4b04fb1a7c996c57d383349.run_cpu_time_us) (tablet-download-pool-ac0fc766d4b04fb1a7c996c57d383349.run_wall_time_us, tablet-download-pool-ac0fc766d4b04fb1a7c996c57d383349.run_wall_time_us) (tablet-download-pool-b0df8ae119f948eaa5461faa42007713.queue_time_us, tablet-download-pool-b0df8ae119f948eaa5461faa42007713.queue_time_us) (tablet-download-pool-b0df8ae119f948eaa5461faa42007713.run_cpu_time_us, tablet-download-pool-b0df8ae119f948eaa5461faa42007713.run_cpu_time_us) (tablet-download-pool-b0df8ae119f948eaa5461faa42007713.run_wall_time_us, tablet-download-pool-b0df8ae119f948eaa5461faa42007713.run_wall_time_us) (tablet-download-pool-b26fbd5ae3384b6cb444331bd8e75aef.queue_time_us, tablet-download-pool-b26fbd5ae3384b6cb444331bd8e75aef.queue_time_us) (tablet-download-pool-b26fbd5ae3384b6cb444331bd8e75aef.run_cpu_time_us, tablet-download-pool-b26fbd5ae3384b6cb444331bd8e75aef.run_cpu_time_us) (tablet-download-pool-b26fbd5ae3384b6cb444331bd8e75aef.run_wall_time_us, tablet-download-pool-b26fbd5ae3384b6cb444331bd8e75aef.run_wall_time_us) (tablet-download-pool-be3ab5a5dd774bdba45e6d7061019195.queue_time_us, tablet-download-pool-be3ab5a5dd774bdba45e6d7061019195.queue_time_us) (tablet-download-pool-be3ab5a5dd774bdba45e6d7061019195.run_cpu_time_us, tablet-download-pool-be3ab5a5dd774bdba45e6d7061019195.run_cpu_time_us) (tablet-download-pool-be3ab5a5dd774bdba45e6d7061019195.run_wall_time_us, tablet-download-pool-be3ab5a5dd774bdba45e6d7061019195.run_wall_time_us) (tablet-download-pool-cefdd1e872814ddbb522f4013d55b26c.queue_time_us, tablet-download-pool-cefdd1e872814ddbb522f4013d55b26c.queue_time_us) (tablet-download-pool-cefdd1e872814ddbb522f4013d55b26c.run_cpu_time_us, tablet-download-pool-cefdd1e872814ddbb522f4013d55b26c.run_cpu_time_us) (tablet-download-pool-cefdd1e872814ddbb522f4013d55b26c.run_wall_time_us, tablet-download-pool-cefdd1e872814ddbb522f4013d55b26c.run_wall_time_us) (tablet-download-pool-e50f84673efd4108859e8c1360cad36f.queue_time_us, tablet-download-pool-e50f84673efd4108859e8c1360cad36f.queue_time_us) (tablet-download-pool-e50f84673efd4108859e8c1360cad36f.run_cpu_time_us, tablet-download-pool-e50f84673efd4108859e8c1360cad36f.run_cpu_time_us) (tablet-download-pool-e50f84673efd4108859e8c1360cad36f.run_wall_time_us, tablet-download-pool-e50f84673efd4108859e8c1360cad36f.run_wall_time_us) (tablet-download-pool-ef909312fe9545b18bf9f918f52928c2.queue_time_us, tablet-download-pool-ef909312fe9545b18bf9f918f52928c2.queue_time_us) (tablet-download-pool-ef909312fe9545b18bf9f918f52928c2.run_cpu_time_us, tablet-download-pool-ef909312fe9545b18bf9f918f52928c2.run_cpu_time_us) (tablet-download-pool-ef909312fe9545b18bf9f918f52928c2.run_wall_time_us, tablet-download-pool-ef909312fe9545b18bf9f918f52928c2.run_wall_time_us) (tablet-download-pool-f884dde21ea246a792634883aed3cb82.queue_time_us, tablet-download-pool-f884dde21ea246a792634883aed3cb82.queue_time_us) (tablet-download-pool-f884dde21ea246a792634883aed3cb82.run_cpu_time_us, tablet-download-pool-f884dde21ea246a792634883aed3cb82.run_cpu_time_us) (tablet-download-pool-f884dde21ea246a792634883aed3cb82.run_wall_time_us, tablet-download-pool-f884dde21ea246a792634883aed3cb82.run_wall_time_us) (tablet-download-pool-fa681a37b30c4a0b879d3f3b184405fc.queue_time_us, tablet-download-pool-fa681a37b30c4a0b879d3f3b184405fc.queue_time_us) (tablet-download-pool-fa681a37b30c4a0b879d3f3b184405fc.run_cpu_time_us, tablet-download-pool-fa681a37b30c4a0b879d3f3b184405fc.run_cpu_time_us) (tablet-download-pool-fa681a37b30c4a0b879d3f3b184405fc.run_wall_time_us, tablet-download-pool-fa681a37b30c4a0b879d3f3b184405fc.run_wall_time_us) (tablet-download-pool-fc8f5043623a48418283e5ecd50d58e2.queue_time_us, tablet-download-pool-fc8f5043623a48418283e5ecd50d58e2.queue_time_us) (tablet-download-pool-fc8f5043623a48418283e5ecd50d58e2.run_cpu_time_us, tablet-download-pool-fc8f5043623a48418283e5ecd50d58e2.run_cpu_time_us) (tablet-download-pool-fc8f5043623a48418283e5ecd50d58e2.run_wall_time_us, tablet-download-pool-fc8f5043623a48418283e5ecd50d58e2.run_wall_time_us) (tablet-open.queue_time_us, tablet-open.queue_time_us) (tablet-open.run_cpu_time_us, tablet-open.run_cpu_time_us) (tablet-open.run_wall_time_us, tablet-open.run_wall_time_us) (txn-commit.queue_time_us, txn-commit.queue_time_us) (txn-commit.run_cpu_time_us, txn-commit.run_cpu_time_us) (txn-commit.run_wall_time_us, txn-commit.run_wall_time_us) (txn-participant-registration.queue_time_us, txn-participant-registration.queue_time_us) (txn-participant-registration.run_cpu_time_us, txn-participant-registration.run_cpu_time_us) (txn-participant-registration.run_wall_time_us, txn-participant-registration.run_wall_time_us) (txn-status-manager.queue_time_us, txn-status-manager.queue_time_us) (txn-status-manager.run_cpu_time_us, txn-status-manager.run_cpu_time_us) (txn-status-manager.run_wall_time_us, txn-status-manager.run_wall_time_us) (txn-status-tablet-reload.queue_time_us, txn-status-tablet-reload.queue_time_us) (txn-status-tablet-reload.run_cpu_time_us, txn-status-tablet-reload.run_cpu_time_us) (txn-status-tablet-reload.run_wall_time_us, txn-status-tablet-reload.run_wall_time_us)
*** Check failure stack trace: ***
I20240503 02:14:42.948185 14850 ts_tablet_manager.cc:902] T fe8b26bb37ab4b948fd500b648bf6b5a P 5f665335495c43dcbc53e072f6b1f44f: Initiating tablet copy from peer 27f29b79cf9b42f4a5b9838965bbb72d (127.9.151.129:33313)
I20240503 02:14:42.950789 14848 ts_tablet_manager.cc:902] T 02b52f763d8a412788ceabba8278d71f P 5f665335495c43dcbc53e072f6b1f44f: Initiating tablet copy from peer 27f29b79cf9b42f4a5b9838965bbb72d (127.9.151.129:33313)
I20240503 02:14:42.951790 14849 ts_tablet_manager.cc:902] T 24e81fa4c7b249aca961df89f56a8557 P 5f665335495c43dcbc53e072f6b1f44f: Initiating tablet copy from peer 27f29b79cf9b42f4a5b9838965bbb72d (127.9.151.129:33313)
@ 0x7f1414602dcd google::LogMessage::Fail() at ??:0
@ 0x7f1414606b93 google::LogMessage::SendToLog() at ??:0
@ 0x7f14146027cc google::LogMessage::Flush() at ??:0
@ 0x7f1414603f59 google::LogMessageFatal::~LogMessageFatal() at ??:0
I20240503 02:14:43.000536 14857 tablet_copy_client.cc:308] T f884dde21ea246a792634883aed3cb82 P 5f665335495c43dcbc53e072f6b1f44f: tablet copy: Beginning tablet copy session from remote peer at address 127.9.151.129:33313
@ 0x7f1417343742 kudu::TraceMetrics::InternName() at ??:0
I20240503 02:14:43.026232 14853 tablet_copy_client.cc:308] T 9d03aee673644af6a0b56ffdc5fb1e99 P 5f665335495c43dcbc53e072f6b1f44f: tablet copy: Beginning tablet copy session from remote peer at address 127.9.151.129:33313
I20240503 02:14:43.028364 14855 tablet_copy_client.cc:308] T 6069a864c5ff43348881ff477218578d P 5f665335495c43dcbc53e072f6b1f44f: tablet copy: Beginning tablet copy session from remote peer at address 127.9.151.129:33313
I20240503 02:14:43.029846 14852 tablet_copy_client.cc:308] T be3ab5a5dd774bdba45e6d7061019195 P 5f665335495c43dcbc53e072f6b1f44f: tablet copy: Beginning tablet copy session from remote peer at address 127.9.151.129:33313
I20240503 02:14:43.030531 14856 tablet_copy_client.cc:308] T e50f84673efd4108859e8c1360cad36f P 5f665335495c43dcbc53e072f6b1f44f: tablet copy: Beginning tablet copy session from remote peer at address 127.9.151.129:33313
I20240503 02:14:43.030822 14854 tablet_copy_client.cc:308] T 9d424d86fde04f2dbdd9434ee5755150 P 5f665335495c43dcbc53e072f6b1f44f: tablet copy: Beginning tablet copy session from remote peer at address 127.9.151.129:33313
I20240503 02:14:43.036926 14860 tablet_copy_client.cc:308] T fa681a37b30c4a0b879d3f3b184405fc P 5f665335495c43dcbc53e072f6b1f44f: tablet copy: Beginning tablet copy session from remote peer at address 127.9.151.129:33313
I20240503 02:14:43.039902 14869 tablet_copy_client.cc:308] T 4ff002252c4d46d2af2a4956acd90b3f P 5f665335495c43dcbc53e072f6b1f44f: tablet copy: Beginning tablet copy session from remote peer at address 127.9.151.129:33313
I20240503 02:14:43.041818 14862 tablet_copy_client.cc:308] T ac0fc766d4b04fb1a7c996c57d383349 P 5f665335495c43dcbc53e072f6b1f44f: tablet copy: Beginning tablet copy session from remote peer at address 127.9.151.129:33313
I20240503 02:14:43.044202 14858 tablet_copy_client.cc:308] T cefdd1e872814ddbb522f4013d55b26c P 5f665335495c43dcbc53e072f6b1f44f: tablet copy: Beginning tablet copy session from remote peer at address 127.9.151.129:33313
@ 0x7f141730ff8b kudu::ThreadPool::ThreadPool() at ??:0
I20240503 02:14:43.045421 14864 tablet_copy_client.cc:308] T a40f48ddab6e48d5ba3707ef78923ee5 P 5f665335495c43dcbc53e072f6b1f44f: tablet copy: Beginning tablet copy session from remote peer at address 127.9.151.129:33313
I20240503 02:14:43.045554 14851 tablet_copy_client.cc:308] T ef909312fe9545b18bf9f918f52928c2 P 5f665335495c43dcbc53e072f6b1f44f: tablet copy: Beginning tablet copy session from remote peer at address 127.9.151.129:33313
I20240503 02:14:43.045902 14865 tablet_copy_client.cc:308] T fc8f5043623a48418283e5ecd50d58e2 P 5f665335495c43dcbc53e072f6b1f44f: tablet copy: Beginning tablet copy session from remote peer at address 127.9.151.129:33313
I20240503 02:14:43.046162 14863 tablet_copy_client.cc:308] T 97c56e26ef4f47f4acf50d0b8d8116dc P 5f665335495c43dcbc53e072f6b1f44f: tablet copy: Beginning tablet copy session from remote peer at address 127.9.151.129:33313
I20240503 02:14:43.046837 14861 tablet_copy_client.cc:308] T b0df8ae119f948eaa5461faa42007713 P 5f665335495c43dcbc53e072f6b1f44f: tablet copy: Beginning tablet copy session from remote peer at address 127.9.151.129:33313
@ 0x7f1417309cd6 kudu::ThreadPoolBuilder::Build() at ??:0
I20240503 02:14:43.056174 14859 tablet_copy_client.cc:308] T b26fbd5ae3384b6cb444331bd8e75aef P 5f665335495c43dcbc53e072f6b1f44f: tablet copy: Beginning tablet copy session from remote peer at address 127.9.151.129:33313
@ 0x7f142675d8de kudu::tserver::TabletCopyClient::TabletCopyClient() at ??:0
I20240503 02:14:43.060905 14764 tablet_copy_service.cc:140] P 27f29b79cf9b42f4a5b9838965bbb72d: Received BeginTabletCopySession request for tablet f884dde21ea246a792634883aed3cb82 from peer 5f665335495c43dcbc53e072f6b1f44f ({username='slave'} at 127.9.151.130:60831)
W20240503 02:14:43.061163 14753 service_pool.cc:135] BeginTabletCopySession request on kudu.tserver.TabletCopyService from 127.9.151.130:60831 dropped due to backpressure. The service queue is full; it has 1 items.
I20240503 02:14:43.061286 14764 tablet_copy_service.cc:161] P 27f29b79cf9b42f4a5b9838965bbb72d: Beginning new tablet copy session on tablet f884dde21ea246a792634883aed3cb82 from peer 5f665335495c43dcbc53e072f6b1f44f at {username='slave'} at 127.9.151.130:60831: session id = 5f665335495c43dcbc53e072f6b1f44f-f884dde21ea246a792634883aed3cb82
I20240503 02:14:43.061627 14753 service_pool.cc:138] BeginTabletCopySession request on kudu.tserver.TabletCopyService from 127.9.151.130:60831 dropped due to backpressure. The service queue is full; it has 1 items. Contents of service queue:
Call kudu.tserver.TabletCopyService.BeginTabletCopySession from 127.9.151.130:60831 (request call id 1)
I20240503 02:14:43.062352 14753 service_pool.cc:138] BeginTabletCopySession request on kudu.tserver.TabletCopyService from 127.9.151.130:60831 dropped due to backpressure. The service queue is full; it has 1 items. Contents of service queue:
Call kudu.tserver.TabletCopyService.BeginTabletCopySession from 127.9.151.130:60831 (request call id 1)
I20240503 02:14:43.062747 14753 service_pool.cc:138] BeginTabletCopySession request on kudu.tserver.TabletCopyService from 127.9.151.130:60831 dropped due to backpressure. The service queue is full; it has 1 items. Contents of service queue:
Call kudu.tserver.TabletCopyService.BeginTabletCopySession from 127.9.151.130:60831 (request call id 1)
I20240503 02:14:43.063066 14753 service_pool.cc:138] BeginTabletCopySession request on kudu.tserver.TabletCopyService from 127.9.151.130:60831 dropped due to backpressure. The service queue is full; it has 1 items. Contents of service queue:
Call kudu.tserver.TabletCopyService.BeginTabletCopySession from 127.9.151.130:60831 (request call id 1)
I20240503 02:14:43.063364 14753 service_pool.cc:138] BeginTabletCopySession request on kudu.tserver.TabletCopyService from 127.9.151.130:60831 dropped due to backpressure. The service queue is full; it has 1 items. Contents of service queue:
Call kudu.tserver.TabletCopyService.BeginTabletCopySession from 127.9.151.130:60831 (request call id 1)
@ 0x7f1426775daf kudu::tserver::RemoteTabletCopyClient::RemoteTabletCopyClient() at ??:0
@ 0x7f14268a0480 kudu::tserver::TSTabletManager::RunTabletCopy() at ??:0
I20240503 02:14:43.071344 14753 service_pool.cc:138] BeginTabletCopySession request on kudu.tserver.TabletCopyService from 127.9.151.130:60831 dropped due to backpressure. The service queue is full; it has 1 items. Contents of service queue:
Call kudu.tserver.TabletCopyService.BeginTabletCopySession from 127.9.151.130:60831 (request call id 1)
I20240503 02:14:43.071835 14753 service_pool.cc:138] BeginTabletCopySession request on kudu.tserver.TabletCopyService from 127.9.151.130:60831 dropped due to backpressure. The service queue is full; it has 1 items. Contents of service queue:
Call kudu.tserver.TabletCopyService.BeginTabletCopySession from 127.9.151.130:60831 (request call id 1)
I20240503 02:14:43.072163 14753 service_pool.cc:138] BeginTabletCopySession request on kudu.tserver.TabletCopyService from 127.9.151.130:60831 dropped due to backpressure. The service queue is full; it has 1 items. Contents of service queue:
Call kudu.tserver.TabletCopyService.BeginTabletCopySession from 127.9.151.130:60831 (request call id 1)
I20240503 02:14:43.072491 14753 service_pool.cc:138] BeginTabletCopySession request on kudu.tserver.TabletCopyService from 127.9.151.130:60831 dropped due to backpressure. The service queue is full; it has 1 items. Contents of service queue:
Call kudu.tserver.TabletCopyService.BeginTabletCopySession from 127.9.151.130:60831 (request call id 1)
I20240503 02:14:43.072595 14764 tablet_copy_source_session.cc:205] T f884dde21ea246a792634883aed3cb82 P 27f29b79cf9b42f4a5b9838965bbb72d: Tablet Copy: opened 0 blocks and 1 log segments
I20240503 02:14:43.072842 14753 service_pool.cc:138] BeginTabletCopySession request on kudu.tserver.TabletCopyService from 127.9.151.130:60831 dropped due to backpressure. The service queue is full; it has 1 items. Contents of service queue:
Call kudu.tserver.TabletCopyService.BeginTabletCopySession from 127.9.151.130:60831 (request call id 1)
I20240503 02:14:43.073698 14753 service_pool.cc:138] BeginTabletCopySession request on kudu.tserver.TabletCopyService from 127.9.151.130:60831 dropped due to backpressure. The service queue is full; it has 1 items. Contents of service queue:
Call kudu.tserver.TabletCopyService.BeginTabletCopySession from 127.9.151.130:60831 (request call id 1)
I20240503 02:14:43.074051 14764 tablet_copy_service.cc:140] P 27f29b79cf9b42f4a5b9838965bbb72d: Received BeginTabletCopySession request for tablet 9d03aee673644af6a0b56ffdc5fb1e99 from peer 5f665335495c43dcbc53e072f6b1f44f ({username='slave'} at 127.9.151.130:60831)
I20240503 02:14:43.074259 14764 tablet_copy_service.cc:161] P 27f29b79cf9b42f4a5b9838965bbb72d: Beginning new tablet copy session on tablet 9d03aee673644af6a0b56ffdc5fb1e99 from peer 5f665335495c43dcbc53e072f6b1f44f at {username='slave'} at 127.9.151.130:60831: session id = 5f665335495c43dcbc53e072f6b1f44f-9d03aee673644af6a0b56ffdc5fb1e99
I20240503 02:14:43.074410 14753 service_pool.cc:138] BeginTabletCopySession request on kudu.tserver.TabletCopyService from 127.9.151.130:60831 dropped due to backpressure. The service queue is full; it has 1 items. Contents of service queue:
Call kudu.tserver.TabletCopyService.BeginTabletCopySession from 127.9.151.130:60831 (request call id 13)
I20240503 02:14:43.074795 14753 service_pool.cc:138] BeginTabletCopySession request on kudu.tserver.TabletCopyService from 127.9.151.130:60831 dropped due to backpressure. The service queue is full; it has 1 items. Contents of service queue:
Call kudu.tserver.TabletCopyService.BeginTabletCopySession from 127.9.151.130:60831 (request call id 13)
@ 0x7f14268d8a8d kudu::tserver::TabletCopyRunnable::Run() at ??:0
@ 0x7f14268bb981 kudu::tserver::TSTabletManager::StartTabletCopy()::$_12::operator()() at ??:0
I20240503 02:14:43.082218 14753 service_pool.cc:138] BeginTabletCopySession request on kudu.tserver.TabletCopyService from 127.9.151.130:60831 dropped due to backpressure. The service queue is full; it has 1 items. Contents of service queue:
Call kudu.tserver.TabletCopyService.BeginTabletCopySession from 127.9.151.130:60831 (request call id 16)
I20240503 02:14:43.083885 14857 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f884dde21ea246a792634883aed3cb82. 1 dirs total, 0 dirs full, 0 dirs failed
I20240503 02:14:43.084772 14753 service_pool.cc:138] BeginTabletCopySession request on kudu.tserver.TabletCopyService from 127.9.151.130:60831 dropped due to backpressure. The service queue is full; it has 1 items. Contents of service queue:
Call kudu.tserver.TabletCopyService.BeginTabletCopySession from 127.9.151.130:60831 (request call id 17)
I20240503 02:14:43.085136 14753 service_pool.cc:138] BeginTabletCopySession request on kudu.tserver.TabletCopyService from 127.9.151.130:60831 dropped due to backpressure. The service queue is full; it has 1 items. Contents of service queue:
Call kudu.tserver.TabletCopyService.BeginTabletCopySession from 127.9.151.130:60831 (request call id 18)
I20240503 02:14:43.085441 14753 service_pool.cc:138] BeginTabletCopySession request on kudu.tserver.TabletCopyService from 127.9.151.130:60831 dropped due to backpressure. The service queue is full; it has 1 items. Contents of service queue:
Call kudu.tserver.TabletCopyService.BeginTabletCopySession from 127.9.151.130:60831 (request call id 18)
@ 0x7f14268bb70e std::_Function_handler<>::_M_invoke() at ??:0
I20240503 02:14:43.087219 14753 service_pool.cc:138] BeginTabletCopySession request on kudu.tserver.TabletCopyService from 127.9.151.130:60831 dropped due to backpressure. The service queue is full; it has 1 items. Contents of service queue:
Call kudu.tserver.TabletCopyService.BeginTabletCopySession from 127.9.151.130:60831 (request call id 18)
I20240503 02:14:43.089157 14753 service_pool.cc:138] BeginTabletCopySession request on kudu.tserver.TabletCopyService from 127.9.151.130:60831 dropped due to backpressure. The service queue is full; it has 1 items. Contents of service queue:
Call kudu.tserver.TabletCopyService.BeginTabletCopySession from 127.9.151.130:60831 (request call id 21)
@ 0xa60bcd std::function<>::operator()() at ??:?
I20240503 02:14:43.089591 14753 service_pool.cc:138] BeginTabletCopySession request on kudu.tserver.TabletCopyService from 127.9.151.130:60831 dropped due to backpressure. The service queue is full; it has 1 items. Contents of service queue:
Call kudu.tserver.TabletCopyService.BeginTabletCopySession from 127.9.151.130:60831 (request call id 21)
I20240503 02:14:43.091109 14753 service_pool.cc:138] BeginTabletCopySession request on kudu.tserver.TabletCopyService from 127.9.151.130:60831 dropped due to backpressure. The service queue is full; it has 1 items. Contents of service queue:
Call kudu.tserver.TabletCopyService.BeginTabletCopySession from 127.9.151.130:60831 (request call id 21)
I20240503 02:14:43.091553 14753 service_pool.cc:138] BeginTabletCopySession request on kudu.tserver.TabletCopyService from 127.9.151.130:60831 dropped due to backpressure. The service queue is full; it has 1 items. Contents of service queue:
Call kudu.tserver.TabletCopyService.BeginTabletCopySession from 127.9.151.130:60831 (request call id 21)
I20240503 02:14:43.091876 14753 service_pool.cc:138] BeginTabletCopySession request on kudu.tserver.TabletCopyService from 127.9.151.130:60831 dropped due to backpressure. The service queue is full; it has 1 items. Contents of service queue:
Call kudu.tserver.TabletCopyService.BeginTabletCopySession from 127.9.151.130:60831 (request call id 21)
I20240503 02:14:43.092298 14753 service_pool.cc:138] BeginTabletCopySession request on kudu.tserver.TabletCopyService from 127.9.151.130:60831 dropped due to backpressure. The service queue is full; it has 1 items. Contents of service queue:
Call kudu.tserver.TabletCopyService.BeginTabletCopySession from 127.9.151.130:60831 (request call id 26)
I20240503 02:14:43.092648 14753 service_pool.cc:138] BeginTabletCopySession request on kudu.tserver.TabletCopyService from 127.9.151.130:60831 dropped due to backpressure. The service queue is full; it has 1 items. Contents of service queue:
Call kudu.tserver.TabletCopyService.BeginTabletCopySession from 127.9.151.130:60831 (request call id 26)
I20240503 02:14:43.093116 14753 service_pool.cc:138] BeginTabletCopySession request on kudu.tserver.TabletCopyService from 127.9.151.130:60831 dropped due to backpressure. The service queue is full; it has 1 items. Contents of service queue:
Call kudu.tserver.TabletCopyService.BeginTabletCopySession from 127.9.151.130:60831 (request call id 26)
@ 0x7f1417313b75 kudu::ThreadPool::DispatchThread() at ??:0
I20240503 02:14:43.094326 14764 tablet_copy_source_session.cc:205] T 9d03aee673644af6a0b56ffdc5fb1e99 P 27f29b79cf9b42f4a5b9838965bbb72d: Tablet Copy: opened 0 blocks and 1 log segments
I20240503 02:14:43.094880 14764 tablet_copy_service.cc:140] P 27f29b79cf9b42f4a5b9838965bbb72d: Received BeginTabletCopySession request for tablet 6069a864c5ff43348881ff477218578d from peer 5f665335495c43dcbc53e072f6b1f44f ({username='slave'} at 127.9.151.130:60831)
I20240503 02:14:43.095060 14764 tablet_copy_service.cc:161] P 27f29b79cf9b42f4a5b9838965bbb72d: Beginning new tablet copy session on tablet 6069a864c5ff43348881ff477218578d from peer 5f665335495c43dcbc53e072f6b1f44f at {username='slave'} at 127.9.151.130:60831: session id = 5f665335495c43dcbc53e072f6b1f44f-6069a864c5ff43348881ff477218578d
I20240503 02:14:43.097904 14853 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9d03aee673644af6a0b56ffdc5fb1e99. 1 dirs total, 0 dirs full, 0 dirs failed
@ 0x7f141731719c kudu::ThreadPool::CreateThread()::$_2::operator()() at ??:0
@ 0x7f1417317019 std::_Function_handler<>::_M_invoke() at ??:0
I20240503 02:14:43.104210 14764 tablet_copy_source_session.cc:205] T 6069a864c5ff43348881ff477218578d P 27f29b79cf9b42f4a5b9838965bbb72d: Tablet Copy: opened 0 blocks and 1 log segments
@ 0xa60bcd std::function<>::operator()() at ??:?
I20240503 02:14:43.104704 14764 tablet_copy_service.cc:140] P 27f29b79cf9b42f4a5b9838965bbb72d: Received BeginTabletCopySession request for tablet b26fbd5ae3384b6cb444331bd8e75aef from peer 5f665335495c43dcbc53e072f6b1f44f ({username='slave'} at 127.9.151.130:60831)
I20240503 02:14:43.105412 14764 tablet_copy_service.cc:161] P 27f29b79cf9b42f4a5b9838965bbb72d: Beginning new tablet copy session on tablet b26fbd5ae3384b6cb444331bd8e75aef from peer 5f665335495c43dcbc53e072f6b1f44f at {username='slave'} at 127.9.151.130:60831: session id = 5f665335495c43dcbc53e072f6b1f44f-b26fbd5ae3384b6cb444331bd8e75aef
I20240503 02:14:43.105903 14855 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6069a864c5ff43348881ff477218578d. 1 dirs total, 0 dirs full, 0 dirs failed
@ 0x7f14172f201e kudu::Thread::SuperviseThread() at ??:0
I20240503 02:14:43.111161 14853 tablet_copy_client.cc:783] T 9d03aee673644af6a0b56ffdc5fb1e99 P 5f665335495c43dcbc53e072f6b1f44f: tablet copy: Starting download of 0 data blocks...
I20240503 02:14:43.113536 14853 tablet_copy_client.cc:647] T 9d03aee673644af6a0b56ffdc5fb1e99 P 5f665335495c43dcbc53e072f6b1f44f: tablet copy: Starting download of 1 WAL segments...
I20240503 02:14:43.114756 14857 tablet_copy_client.cc:783] T f884dde21ea246a792634883aed3cb82 P 5f665335495c43dcbc53e072f6b1f44f: tablet copy: Starting download of 0 data blocks...
I20240503 02:14:43.115054 14857 tablet_copy_client.cc:647] T f884dde21ea246a792634883aed3cb82 P 5f665335495c43dcbc53e072f6b1f44f: tablet copy: Starting download of 1 WAL segments...
I20240503 02:14:43.116446 14753 service_pool.cc:138] FetchData request on kudu.tserver.TabletCopyService from 127.9.151.130:60831 dropped due to backpressure. The service queue is full; it has 1 items. Contents of service queue:
Call kudu.tserver.TabletCopyService.FetchData from 127.9.151.130:60831 (request call id 30)
I20240503 02:14:43.116926 14753 service_pool.cc:138] FetchData request on kudu.tserver.TabletCopyService from 127.9.151.130:60831 dropped due to backpressure. The service queue is full; it has 1 items. Contents of service queue:
Call kudu.tserver.TabletCopyService.BeginTabletCopySession from 127.9.151.130:60831 (request call id 32)
I20240503 02:14:43.117272 14764 tablet_copy_source_session.cc:205] T b26fbd5ae3384b6cb444331bd8e75aef P 27f29b79cf9b42f4a5b9838965bbb72d: Tablet Copy: opened 0 blocks and 1 log segments
I20240503 02:14:43.117830 14855 tablet_copy_client.cc:783] T 6069a864c5ff43348881ff477218578d P 5f665335495c43dcbc53e072f6b1f44f: tablet copy: Starting download of 0 data blocks...
@ 0x7f141a6776db start_thread at ??:0
I20240503 02:14:43.118140 14855 tablet_copy_client.cc:647] T 6069a864c5ff43348881ff477218578d P 5f665335495c43dcbc53e072f6b1f44f: tablet copy: Starting download of 1 WAL segments...
I20240503 02:14:43.118131 14764 tablet_copy_service.cc:140] P 27f29b79cf9b42f4a5b9838965bbb72d: Received BeginTabletCopySession request for tablet a40f48ddab6e48d5ba3707ef78923ee5 from peer 5f665335495c43dcbc53e072f6b1f44f ({username='slave'} at 127.9.151.130:60831)
I20240503 02:14:43.118269 14764 tablet_copy_service.cc:161] P 27f29b79cf9b42f4a5b9838965bbb72d: Beginning new tablet copy session on tablet a40f48ddab6e48d5ba3707ef78923ee5 from peer 5f665335495c43dcbc53e072f6b1f44f at {username='slave'} at 127.9.151.130:60831: session id = 5f665335495c43dcbc53e072f6b1f44f-a40f48ddab6e48d5ba3707ef78923ee5
I20240503 02:14:43.118495 14859 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b26fbd5ae3384b6cb444331bd8e75aef. 1 dirs total, 0 dirs full, 0 dirs failed
@ 0x7f14134b571f clone at ??:0
I20240503 02:14:43.129834 14764 tablet_copy_source_session.cc:205] T a40f48ddab6e48d5ba3707ef78923ee5 P 27f29b79cf9b42f4a5b9838965bbb72d: Tablet Copy: opened 0 blocks and 1 log segments
W20240503 02:14:43.312528 14753 connection.cc:535] server connection from 127.9.151.130:60831 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20240503 02:14:43.312614 14042 connection.cc:535] client connection to 127.9.151.130:41429 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
F20240503 02:14:43.313174 14831 tablet_copy_client_session-itest.cc:314] Check failed: _s.ok() Bad status: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
F20240503 02:14:43.313366 14835 tablet_copy_client_session-itest.cc:314] Check failed: _s.ok() Bad status: F20240503 02:14:43.313688 14827 tablet_copy_client_session-itest.cc:314] Check failed: _s.ok() Bad status: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)F20240503 02:14:43.313864 14833 tablet_copy_client_session-itest.cc:314] Check failed: _s.ok() Bad status: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
*** Check failure stack trace: ***
*** Check failure stack trace: ***
*** Aborted at 1714702483 (unix time) try "date -d @1714702483" if you are using GNU date ***
F20240503 02:14:43.313366 14835 tablet_copy_client_session-itest.cc:314] Check failed: _s.ok() Bad status: F20240503 02:14:43.313688 14827 tablet_copy_client_session-itest.cc:314] Check failed: _s.ok() Bad status: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)F20240503 02:14:43.313864 14833 tablet_copy_client_session-itest.cc:314] Check failed: _s.ok() Bad status: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)F20240503 02:14:43.314018 14844 tablet_copy_client_session-itest.cc:314] Check failed: _s.ok() Bad status: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
*** Check failure stack trace: ***
F20240503 02:14:43.313366 14835 tablet_copy_client_session-itest.cc:314] Check failed: _s.ok() Bad status: F20240503 02:14:43.313688 14827 tablet_copy_client_session-itest.cc:314] Check failed: _s.ok() Bad status: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)F20240503 02:14:43.313864 14833 tablet_copy_client_session-itest.cc:314] Check failed: _s.ok() Bad status: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)F20240503 02:14:43.314018 14844 tablet_copy_client_session-itest.cc:314] Check failed: _s.ok() Bad status: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)F20240503 02:14:43.314378 14828 tablet_copy_client_session-itest.cc:314] Check failed: _s.ok() Bad status: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
*** Check failure stack trace: ***
F20240503 02:14:43.313366 14835 tablet_copy_client_session-itest.cc:314] Check failed: _s.ok() Bad status: F20240503 02:14:43.313688 14827 tablet_copy_client_session-itest.cc:314] Check failed: _s.ok() Bad status: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)F20240503 02:14:43.313864 14833 tablet_copy_client_session-itest.cc:314] Check failed: _s.ok() Bad status: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)F20240503 02:14:43.314018 14844 tablet_copy_client_session-itest.cc:314] Check failed: _s.ok() Bad status: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)F20240503 02:14:43.314378 14828 tablet_copy_client_session-itest.cc:314] Check failed: _s.ok() Bad status: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)F20240503 02:14:43.314548 14847 tablet_copy_client_session-itest.cc:314] Check failed: _s.ok() Bad status: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
*** Check failure stack trace: ***
F20240503 02:14:43.313366 14835 tablet_copy_client_session-itest.cc:314] Check failed: _s.ok() Bad status: F20240503 02:14:43.313688 14827 tablet_copy_client_session-itest.cc:314] Check failed: _s.ok() Bad status: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)F20240503 02:14:43.313864 14833 tablet_copy_client_session-itest.cc:314] Check failed: _s.ok() Bad status: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)F20240503 02:14:43.314018 14844 tablet_copy_client_session-itest.cc:314] Check failed: _s.ok() Bad status: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)F20240503 02:14:43.314378 14828 tablet_copy_client_session-itest.cc:314] Check failed: _s.ok() Bad status: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)F20240503 02:14:43.314548 14847 tablet_copy_client_session-itest.cc:314] Check failed: _s.ok() Bad status: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)F20240503 02:14:43.314812 14846 tablet_copy_client_session-itest.cc:314] Check failed: _s.ok() Bad status: F20240503 02:14:43.314858 14826 tablet_copy_client_session-itest.cc:314] Check failed: _s.ok() Bad status: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
*** Check failure stack trace: ***
F20240503 02:14:43.313366 14835 tablet_copy_client_session-itest.cc:314] Check failed: _s.ok() Bad status: F20240503 02:14:43.313688 14827 tablet_copy_client_session-itest.cc:314] Check failed: _s.ok() Bad status: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)F20240503 02:14:43.313864 14833 tablet_copy_client_session-itest.cc:314] Check failed: _s.ok() Bad status: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)F20240503 02:14:43.314018 14844 tablet_copy_client_session-itest.cc:314] Check failed: _s.ok() Bad status: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)F20240503 02:14:43.314378 14828 tablet_copy_client_session-itest.cc:314] Check failed: _s.ok() Bad status: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)F20240503 02:14:43.314548 14847 tablet_copy_client_session-itest.cc:314] Check failed: _s.ok() Bad status: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)F20240503 02:14:43.314812 14846 tablet_copy_client_session-itest.cc:314] Check failed: _s.ok() Bad status: F20240503 02:14:43.314858 14826 tablet_copy_client_session-itest.cc:314] Check failed: _s.ok() Bad status: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
*** Check failure stack trace: ***
*** Check failure stack trace: ***
*** Check failure stack trace: ***
F20240503 02:14:43.313366 14835 tablet_copy_client_session-itest.cc:314] Check failed: _s.ok() Bad status: F20240503 02:14:43.313688 14827 tablet_copy_client_session-itest.cc:314] Check failed: _s.ok() Bad status: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)F20240503 02:14:43.313864 14833 tablet_copy_client_session-itest.cc:314] Check failed: _s.ok() Bad status: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)F20240503 02:14:43.314018 14844 tablet_copy_client_session-itest.cc:314] Check failed: _s.ok() Bad status: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)F20240503 02:14:43.314378 14828 tablet_copy_client_session-itest.cc:314] Check failed: _s.ok() Bad status: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)F20240503 02:14:43.314548 14847 tablet_copy_client_session-itest.cc:314] Check failed: _s.ok() Bad status: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)F20240503 02:14:43.314812 14846 tablet_copy_client_session-itest.cc:314] Check failed: _s.ok() Bad status: F20240503 02:14:43.314858 14826 tablet_copy_client_session-itest.cc:314] Check failed: _s.ok() Bad status: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)F20240503 02:14:43.315109 14845 tablet_copy_client_session-itest.cc:314] Check failed: _s.ok() Bad status: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
*** Check failure stack trace: ***
F20240503 02:14:43.313366 14835 tablet_copy_client_session-itest.cc:314] Check failed: _s.ok() Bad status: F20240503 02:14:43.313688 14827 tablet_copy_client_session-itest.cc:314] Check failed: _s.ok() Bad status: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)F20240503 02:14:43.313864 14833 tablet_copy_client_session-itest.cc:314] Check failed: _s.ok() Bad status: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)F20240503 02:14:43.314018 14844 tablet_copy_client_session-itest.cc:314] Check failed: _s.ok() Bad status: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)F20240503 02:14:43.314378 14828 tablet_copy_client_session-itest.cc:314] Check failed: _s.ok() Bad status: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)F20240503 02:14:43.314548 14847 tablet_copy_client_session-itest.cc:314] Check failed: _s.ok() Bad status: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)F20240503 02:14:43.314812 14846 tablet_copy_client_session-itest.cc:314] Check failed: _s.ok() Bad status: F20240503 02:14:43.314858 14826 tablet_copy_client_session-itest.cc:314] Check failed: _s.ok() Bad status: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)F20240503 02:14:43.315109 14845 tablet_copy_client_session-itest.cc:314] Check failed: _s.ok() Bad status: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)F20240503 02:14:43.315434 14836 tablet_copy_client_session-itest.cc:314] Check failed: _s.ok() Bad status: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
*** Check failure stack trace: ***
F20240503 02:14:43.313366 14835 tablet_copy_client_session-itest.cc:314] Check failed: _s.ok() Bad status: F20240503 02:14:43.313688 14827 tablet_copy_client_session-itest.cc:314] Check failed: _s.ok() Bad status: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)F20240503 02:14:43.313864 14833 tablet_copy_client_session-itest.cc:314] Check failed: _s.ok() Bad status: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)F20240503 02:14:43.314018 14844 tablet_copy_client_session-itest.cc:314] Check failed: _s.ok() Bad status: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)F20240503 02:14:43.314378 14828 tablet_copy_client_session-itest.cc:314] Check failed: _s.ok() Bad status: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)F20240503 02:14:43.314548 14847 tablet_copy_client_session-itest.cc:314] Check failed: _s.ok() Bad status: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)F20240503 02:14:43.314812 14846 tablet_copy_client_session-itest.cc:314] Check failed: _s.ok() Bad status: F20240503 02:14:43.314858 14826 tablet_copy_client_session-itest.cc:314] Check failed: _s.ok() Bad status: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)F20240503 02:14:43.315109 14845 tablet_copy_client_session-itest.cc:314] Check failed: _s.ok() Bad status: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)F20240503 02:14:43.315434 14836 tablet_copy_client_session-itest.cc:314] Check failed: _s.ok() Bad status: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)F20240503 02:14:43.315686 14834 tablet_copy_client_session-itest.cc:314] Check failed: _s.ok() Bad status: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
*** Check failure stack trace: ***
F20240503 02:14:43.313366 14835 tablet_copy_client_session-itest.cc:314] Check failed: _s.ok() Bad status: F20240503 02:14:43.313688 14827 tablet_copy_client_session-itest.cc:314] Check failed: _s.ok() Bad status: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)F20240503 02:14:43.313864 14833 tablet_copy_client_session-itest.cc:314] Check failed: _s.ok() Bad status: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)F20240503 02:14:43.314018 14844 tablet_copy_client_session-itest.cc:314] Check failed: _s.ok() Bad status: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)F20240503 02:14:43.314378 14828 tablet_copy_client_session-itest.cc:314] Check failed: _s.ok() Bad status: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)F20240503 02:14:43.314548 14847 tablet_copy_client_session-itest.cc:314] Check failed: _s.ok() Bad status: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)F20240503 02:14:43.314812 14846 tablet_copy_client_session-itest.cc:314] Check failed: _s.ok() Bad status: F20240503 02:14:43.314858 14826 tablet_copy_client_session-itest.cc:314] Check failed: _s.ok() Bad status: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)F20240503 02:14:43.315109 14845 tablet_copy_client_session-itest.cc:314] Check failed: _s.ok() Bad status: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)F20240503 02:14:43.315434 14836 tablet_copy_client_session-itest.cc:314] Check failed: _s.ok() Bad status: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)F20240503 02:14:43.315686 14834 tablet_copy_client_session-itest.cc:314] Check failed: _s.ok() Bad status: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)F20240503 02:14:43.315822 14841 tablet_copy_client_session-itest.cc:314] Check failed: _s.ok() Bad status: F20240503 02:14:43.315721 14840 tablet_copy_client_session-itest.cc:314] Check failed: _s.ok() Bad status: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
*** Check failure stack trace: ***
F20240503 02:14:43.313366 14835 tablet_copy_client_session-itest.cc:314] Check failed: _s.ok() Bad status: F20240503 02:14:43.313688 14827 tablet_copy_client_session-itest.cc:314] Check failed: _s.ok() Bad status: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)F20240503 02:14:43.313864 14833 tablet_copy_client_session-itest.cc:314] Check failed: _s.ok() Bad status: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)F20240503 02:14:43.314018 14844 tablet_copy_client_session-itest.cc:314] Check failed: _s.ok() Bad status: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)F20240503 02:14:43.314378 14828 tablet_copy_client_session-itest.cc:314] Check failed: _s.ok() Bad status: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)F20240503 02:14:43.314548 14847 tablet_copy_client_session-itest.cc:314] Check failed: _s.ok() Bad status: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)F20240503 02:14:43.314812 14846 tablet_copy_client_session-itest.cc:314] Check failed: _s.ok() Bad status: F20240503 02:14:43.314858 14826 tablet_copy_client_session-itest.cc:314] Check failed: _s.ok() Bad status: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)F20240503 02:14:43.315109 14845 tablet_copy_client_session-itest.cc:314] Check failed: _s.ok() Bad status: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)F20240503 02:14:43.315434 14836 tablet_copy_client_session-itest.cc:314] Check failed: _s.ok() Bad status: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)F20240503 02:14:43.315686 14834 tablet_copy_client_session-itest.cc:314] Check failed: _s.ok() Bad status: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)F20240503 02:14:43.315822 14841 tablet_copy_client_session-itest.cc:314] Check failed: _s.ok() Bad status: F20240503 02:14:43.315721 14840 tablet_copy_client_session-itest.cc:314] Check failed: _s.ok() Bad status: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
*** Check failure stack trace: ***
F20240503 02:14:43.313366 14835 tablet_copy_client_session-itest.cc:314] Check failed: _s.ok() Bad status: F20240503 02:14:43.313688 14827 tablet_copy_client_session-itest.cc:314] Check failed: _s.ok() Bad status: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)F20240503 02:14:43.313864 14833 tablet_copy_client_session-itest.cc:314] Check failed: _s.ok() Bad status: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)F20240503 02:14:43.314018 14844 tablet_copy_client_session-itest.cc:314] Check failed: _s.ok() Bad status: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)F20240503 02:14:43.314378 14828 tablet_copy_client_session-itest.cc:314] Check failed: _s.ok() Bad status: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)F20240503 02:14:43.314548 14847 tablet_copy_client_session-itest.cc:314] Check failed: _s.ok() Bad status: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)F20240503 02:14:43.314812 14846 tablet_copy_client_session-itest.cc:314] Check failed: _s.ok() Bad status: F20240503 02:14:43.314858 14826 tablet_copy_client_session-itest.cc:314] Check failed: _s.ok() Bad status: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)F20240503 02:14:43.315109 14845 tablet_copy_client_session-itest.cc:314] Check failed: _s.ok() Bad status: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)F20240503 02:14:43.315434 14836 tablet_copy_client_session-itest.cc:314] Check failed: _s.ok() Bad status: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)F20240503 02:14:43.315686 14834 tablet_copy_client_session-itest.cc:314] Check failed: _s.ok() Bad status: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)F20240503 02:14:43.315822 14841 tablet_copy_client_session-itest.cc:314] Check failed: _s.ok() Bad status: F20240503 02:14:43.315721 14840 tablet_copy_client_session-itest.cc:314] Check failed: _s.ok() Bad status: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)F20240503 02:14:43.316190 14842 tablet_copy_client_session-itest.cc:314] Check failed: _s.ok() Bad status: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
*** Check failure stack trace: ***
F20240503 02:14:43.313366 14835 tablet_copy_client_session-itest.cc:314] Check failed: _s.ok() Bad status: F20240503 02:14:43.313688 14827 tablet_copy_client_session-itest.cc:314] Check failed: _s.ok() Bad status: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)F20240503 02:14:43.313864 14833 tablet_copy_client_session-itest.cc:314] Check failed: _s.ok() Bad status: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)F20240503 02:14:43.314018 14844 tablet_copy_client_session-itest.cc:314] Check failed: _s.ok() Bad status: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)F20240503 02:14:43.314378 14828 tablet_copy_client_session-itest.cc:314] Check failed: _s.ok() Bad status: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)F20240503 02:14:43.314548 14847 tablet_copy_client_session-itest.cc:314] Check failed: _s.ok() Bad status: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)F20240503 02:14:43.314812 14846 tablet_copy_client_session-itest.cc:314] Check failed: _s.ok() Bad status: F20240503 02:14:43.314858 14826 tablet_copy_client_session-itest.cc:314] Check failed: _s.ok() Bad status: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)F20240503 02:14:43.315109 14845 tablet_copy_client_session-itest.cc:314] Check failed: _s.ok() Bad status: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)F20240503 02:14:43.315434 14836 tablet_copy_client_session-itest.cc:314] Check failed: _s.ok() Bad status: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)F20240503 02:14:43.315686 14834 tablet_copy_client_session-itest.cc:314] Check failed: _s.ok() Bad status: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)F20240503 02:14:43.315822 14841 tablet_copy_client_session-itest.cc:314] Check failed: _s.ok() Bad status: F20240503 02:14:43.315721 14840 tablet_copy_client_session-itest.cc:314] Check failed: _s.ok() Bad status: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)F20240503 02:14:43.316190 14842 tablet_copy_client_session-itest.cc:314] Check failed: _s.ok() Bad status: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)F20240503 02:14:43.316706 14839 tablet_copy_client_session-itest.cc:314] Check failed: _s.ok() Bad status: F20240503 02:14:43.316761 14843 tablet_copy_client_session-itest.cc:314] Check failed: _s.ok() Bad status: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
*** Check failure stack trace: ***
F20240503 02:14:43.313366 14835 tablet_copy_client_session-itest.cc:314] Check failed: _s.ok() Bad status: F20240503 02:14:43.313688 14827 tablet_copy_client_session-itest.cc:314] Check failed: _s.ok() Bad status: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)F20240503 02:14:43.313864 14833 tablet_copy_client_session-itest.cc:314] Check failed: _s.ok() Bad status: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)F20240503 02:14:43.314018 14844 tablet_copy_client_session-itest.cc:314] Check failed: _s.ok() Bad status: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)F20240503 02:14:43.314378 14828 tablet_copy_client_session-itest.cc:314] Check failed: _s.ok() Bad status: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)F20240503 02:14:43.314548 14847 tablet_copy_client_session-itest.cc:314] Check failed: _s.ok() Bad status: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)F20240503 02:14:43.314812 14846 tablet_copy_client_session-itest.cc:314] Check failed: _s.ok() Bad status: F20240503 02:14:43.314858 14826 tablet_copy_client_session-itest.cc:314] Check failed: _s.ok() Bad status: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)F20240503 02:14:43.315109 14845 tablet_copy_client_session-itest.cc:314] Check failed: _s.ok() Bad status: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)F20240503 02:14:43.315434 14836 tablet_copy_client_session-itest.cc:314] Check failed: _s.ok() Bad status: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)F20240503 02:14:43.315686 14834 tablet_copy_client_session-itest.cc:314] Check failed: _s.ok() Bad status: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)F20240503 02:14:43.315822 14841 tablet_copy_client_session-itest.cc:314] Check failed: _s.ok() Bad status: F20240503 02:14:43.315721 14840 tablet_copy_client_session-itest.cc:314] Check failed: _s.ok() Bad status: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)F20240503 02:14:43.316190 14842 tablet_copy_client_session-itest.cc:314] Check failed: _s.ok() Bad status: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)F20240503 02:14:43.316706 14839 tablet_copy_client_session-itest.cc:314] Check failed: _s.ok() Bad status: F20240503 02:14:43.316761 14843 tablet_copy_client_session-itest.cc:314] Check failed: _s.ok() Bad status: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
*** Check failure stack trace: ***
PC: @ 0x0 (unknown)
*** SIGABRT (@0x3e80000265e) received by PID 9822 (TID 0x7fadf8db0700) from PID 9822; stack trace: ***
@ 0x7fae0d60a980 (unknown) at ??:0
@ 0x7fae066b5fb7 gsignal at ??:0
@ 0x7fae066b7921 abort at ??:0
@ 0x7fae093dedcd google::LogMessage::Fail() at ??:0
@ 0x7fae093e2b93 google::LogMessage::SendToLog() at ??:0
@ 0x7fae093de7cc google::LogMessage::Flush() at ??:0
@ 0x7fae093dff59 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x3b9874 kudu::TabletCopyClientSessionITest_TestTabletCopyWithBusySource_Test::TestBody()::$_3::operator()() at /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/integration-tests/tablet_copy_client_session-itest.cc:313
@ 0x3b926e std::__invoke_impl<>() at /usr/lib/gcc/x86_64-linux-gnu/7.5.0/../../../../include/c++/7.5.0/bits/invoke.h:60
@ 0x3b91ee std::__invoke<>() at /usr/lib/gcc/x86_64-linux-gnu/7.5.0/../../../../include/c++/7.5.0/bits/invoke.h:95
@ 0x3b91b3 _ZNSt6thread8_InvokerISt5tupleIJZN4kudu62TabletCopyClientSessionITest_TestTabletCopyWithBusySource_Test8TestBodyEvE3$_3EEE9_M_invokeIJLm0EEEEDTclsr3stdE8__invokespcl10_S_declvalIXT_EEEEESt12_Index_tupleIJXspT_EEE at /usr/lib/gcc/x86_64-linux-gnu/7.5.0/../../../../include/c++/7.5.0/thread:234
@ 0x3b917e std::thread::_Invoker<>::operator()() at /usr/lib/gcc/x86_64-linux-gnu/7.5.0/../../../../include/c++/7.5.0/thread:243
@ 0x3b9069 std::thread::_State_impl<>::_M_run() at /usr/lib/gcc/x86_64-linux-gnu/7.5.0/../../../../include/c++/7.5.0/thread:186
@ 0x7fae06d3f6df (unknown) at ??:0
@ 0x7fae0d5ff6db start_thread at ??:0
@ 0x7fae0679871f clone at ??:0