[==========] Running 56 tests from 4 test suites.
[----------] Global test environment set-up.
[----------] 30 tests from SecurityITest
[ RUN ] SecurityITest.TestAuthorizationOnListTablets
Loading random data
Initializing database '/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786336863388938-23197-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 04:41:03 dist-test-slave-bqcl krb5kdc[23212](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 04:41:03 dist-test-slave-bqcl krb5kdc[23212](info): set up 2 sockets
Aug 10 04:41:03 dist-test-slave-bqcl krb5kdc[23212](info): commencing operation
krb5kdc: starting...
WARNING: Logging before InitGoogleLogging() is written to STDERR
W20260810 04:41:04.635406 23197 mini_kdc.cc:121] Time spent starting KDC: real 1.229s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786336863388938-23197-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786336863388938-23197-0/krb5kdc/test-user.keytab.
Aug 10 04:41:04 dist-test-slave-bqcl krb5kdc[23212](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786336864, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T04:41:04Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T04:41:04Z Disabled control of system clock
WARNING: no policy specified for kudu/127.22.167.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.167.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.167.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786336863388938-23197-0/krb5kdc/kudu_127.22.167.126.keytab.
Entry for principal kudu/127.22.167.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786336863388938-23197-0/krb5kdc/kudu_127.22.167.126.keytab.
I20260810 04:41:04.752975 23197 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskemv_AM/build/release/bin/kudu
/tmp/dist-test-taskemv_AM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786336863388938-23197-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786336863388938-23197-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786336863388938-23197-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786336863388938-23197-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.167.126:46389
--webserver_interface=127.22.167.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786336863388938-23197-0/krb5kdc/kudu_127.22.167.126.keytab
--principal=kudu/127.22.167.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.167.84:45729
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.167.126:46389
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786336863388938-23197-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786336863388938-23197-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786336863388938-23197-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 04:41:04.835896 23234 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 04:41:04.836091 23234 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 04:41:04.836112 23234 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 04:41:04.837231 23234 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 04:41:04.837270 23234 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 04:41:04.837283 23234 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 04:41:04.837296 23234 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 04:41:04.837308 23234 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 04:41:04.839164 23234 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.167.84:45729
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786336863388938-23197-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786336863388938-23197-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.22.167.126:46389
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786336863388938-23197-0/krb5kdc/kudu_127.22.167.126.keytab
--principal=kudu/127.22.167.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.167.126:46389
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786336863388938-23197-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.167.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23234
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786336863388938-23197-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 03:54:01 UTC on ccycloud.quasar-icijmt.root.comops.site
build id 4577009
I20260810 04:41:04.839380 23234 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 04:41:04.839607 23234 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 04:41:04.844369 23240 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 04:41:04.844383 23243 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 04:41:04.844488 23241 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 04:41:04.844727 23234 server_base.cc:1034] running on GCE node
I20260810 04:41:04.844990 23234 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 04:41:04.845335 23234 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 04:41:04.846468 23234 hybrid_clock.cc:648] HybridClock initialized: now 1786336864846457 us; error 31 us; skew 500 ppm
Aug 10 04:41:04 dist-test-slave-bqcl krb5kdc[23212](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786336864, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.167.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 04:41:04.851316 23234 init.cc:377] Logged in from keytab as kudu/127.22.167.126@KRBTEST.COM (short username kudu)
I20260810 04:41:04.851913 23234 webserver.cc:460] Webserver started at http://127.22.167.126:39861/ using document root <none> and password file <none>
I20260810 04:41:04.852154 23234 fs_manager.cc:359] Metadata directory not provided
I20260810 04:41:04.852205 23234 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 04:41:04.852344 23234 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 04:41:04.853394 23234 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786336863388938-23197-0/minicluster-data/master-0/data/instance:
uuid: "84b4c450220743d1b68a8dcea1dfba59"
format_stamp: "Formatted at 2026-08-10 04:41:04 on dist-test-slave-bqcl"
server_key: "911b6db925fd59b55d1992eba7422ba3"
server_key_iv: "319df0a3c7b9381ec5c3fd290d838409"
server_key_version: "encryptionkey@0"
I20260810 04:41:04.853760 23234 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786336863388938-23197-0/minicluster-data/master-0/wal/instance:
uuid: "84b4c450220743d1b68a8dcea1dfba59"
format_stamp: "Formatted at 2026-08-10 04:41:04 on dist-test-slave-bqcl"
server_key: "911b6db925fd59b55d1992eba7422ba3"
server_key_iv: "319df0a3c7b9381ec5c3fd290d838409"
server_key_version: "encryptionkey@0"
I20260810 04:41:04.855104 23234 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260810 04:41:04.855818 23254 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 04:41:04.856009 23234 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260810 04:41:04.856098 23234 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786336863388938-23197-0/minicluster-data/master-0/data,/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786336863388938-23197-0/minicluster-data/master-0/wal
uuid: "84b4c450220743d1b68a8dcea1dfba59"
format_stamp: "Formatted at 2026-08-10 04:41:04 on dist-test-slave-bqcl"
server_key: "911b6db925fd59b55d1992eba7422ba3"
server_key_iv: "319df0a3c7b9381ec5c3fd290d838409"
server_key_version: "encryptionkey@0"
I20260810 04:41:04.856161 23234 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786336863388938-23197-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786336863388938-23197-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786336863388938-23197-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 04:41:04.875393 23234 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 04:41:04.877475 23234 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 04:41:04.877627 23234 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 04:41:04.885825 23234 rpc_server.cc:307] RPC server started. Bound to: 127.22.167.126:46389
I20260810 04:41:04.885867 23346 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.167.126:46389 every 8 connection(s)
I20260810 04:41:04.886250 23234 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786336863388938-23197-0/minicluster-data/master-0/data/info.pb
I20260810 04:41:04.887115 23347 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 04:41:04.887557 23197 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskemv_AM/build/release/bin/kudu as pid 23234
I20260810 04:41:04.887640 23197 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786336863388938-23197-0/minicluster-data/master-0/wal/instance
I20260810 04:41:04.888403 23197 external_mini_cluster.cc:1428] Setting key bb3147930fd7739f7733b8c18d680189
I20260810 04:41:04.889683 23347 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 84b4c450220743d1b68a8dcea1dfba59: Bootstrap starting.
I20260810 04:41:04.890303 23347 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 84b4c450220743d1b68a8dcea1dfba59: Neither blocks nor log segments found. Creating new log.
I20260810 04:41:04.890568 23347 log.cc:824] T 00000000000000000000000000000000 P 84b4c450220743d1b68a8dcea1dfba59: Log is configured to *not* fsync() on all Append() calls
I20260810 04:41:04.891202 23347 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 84b4c450220743d1b68a8dcea1dfba59: No bootstrap required, opened a new log
I20260810 04:41:04.893196 23347 raft_consensus.cc:348] T 00000000000000000000000000000000 P 84b4c450220743d1b68a8dcea1dfba59 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "84b4c450220743d1b68a8dcea1dfba59" member_type: VOTER last_known_addr { host: "127.22.167.126" port: 46389 } }
I20260810 04:41:04.893329 23347 raft_consensus.cc:374] T 00000000000000000000000000000000 P 84b4c450220743d1b68a8dcea1dfba59 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 04:41:04.893361 23347 raft_consensus.cc:729] T 00000000000000000000000000000000 P 84b4c450220743d1b68a8dcea1dfba59 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84b4c450220743d1b68a8dcea1dfba59, State: Initialized, Role: FOLLOWER
I20260810 04:41:04.893483 23347 consensus_queue.cc:260] T 00000000000000000000000000000000 P 84b4c450220743d1b68a8dcea1dfba59 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "84b4c450220743d1b68a8dcea1dfba59" member_type: VOTER last_known_addr { host: "127.22.167.126" port: 46389 } }
I20260810 04:41:04.893600 23347 raft_consensus.cc:388] T 00000000000000000000000000000000 P 84b4c450220743d1b68a8dcea1dfba59 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 04:41:04.893646 23347 raft_consensus.cc:482] T 00000000000000000000000000000000 P 84b4c450220743d1b68a8dcea1dfba59 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 04:41:04.893694 23347 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 84b4c450220743d1b68a8dcea1dfba59 [term 0 FOLLOWER]: Advancing to term 1
I20260810 04:41:04.894559 23347 raft_consensus.cc:504] T 00000000000000000000000000000000 P 84b4c450220743d1b68a8dcea1dfba59 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "84b4c450220743d1b68a8dcea1dfba59" member_type: VOTER last_known_addr { host: "127.22.167.126" port: 46389 } }
I20260810 04:41:04.894661 23347 leader_election.cc:302] T 00000000000000000000000000000000 P 84b4c450220743d1b68a8dcea1dfba59 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 84b4c450220743d1b68a8dcea1dfba59; no voters:
I20260810 04:41:04.894824 23347 leader_election.cc:288] T 00000000000000000000000000000000 P 84b4c450220743d1b68a8dcea1dfba59 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 04:41:04.894860 23354 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 84b4c450220743d1b68a8dcea1dfba59 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 04:41:04.894968 23354 raft_consensus.cc:686] T 00000000000000000000000000000000 P 84b4c450220743d1b68a8dcea1dfba59 [term 1 LEADER]: Becoming Leader. State: Replica: 84b4c450220743d1b68a8dcea1dfba59, State: Running, Role: LEADER
I20260810 04:41:04.895077 23347 sys_catalog.cc:564] T 00000000000000000000000000000000 P 84b4c450220743d1b68a8dcea1dfba59 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 04:41:04.895113 23354 consensus_queue.cc:237] T 00000000000000000000000000000000 P 84b4c450220743d1b68a8dcea1dfba59 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "84b4c450220743d1b68a8dcea1dfba59" member_type: VOTER last_known_addr { host: "127.22.167.126" port: 46389 } }
I20260810 04:41:04.895481 23354 sys_catalog.cc:455] T 00000000000000000000000000000000 P 84b4c450220743d1b68a8dcea1dfba59 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 84b4c450220743d1b68a8dcea1dfba59. Latest consensus state: current_term: 1 leader_uuid: "84b4c450220743d1b68a8dcea1dfba59" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "84b4c450220743d1b68a8dcea1dfba59" member_type: VOTER last_known_addr { host: "127.22.167.126" port: 46389 } } }
I20260810 04:41:04.895550 23356 sys_catalog.cc:455] T 00000000000000000000000000000000 P 84b4c450220743d1b68a8dcea1dfba59 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "84b4c450220743d1b68a8dcea1dfba59" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "84b4c450220743d1b68a8dcea1dfba59" member_type: VOTER last_known_addr { host: "127.22.167.126" port: 46389 } } }
I20260810 04:41:04.895644 23354 sys_catalog.cc:458] T 00000000000000000000000000000000 P 84b4c450220743d1b68a8dcea1dfba59 [sys.catalog]: This master's current role is: LEADER
I20260810 04:41:04.895941 23366 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 04:41:04.895654 23356 sys_catalog.cc:458] T 00000000000000000000000000000000 P 84b4c450220743d1b68a8dcea1dfba59 [sys.catalog]: This master's current role is: LEADER
I20260810 04:41:04.896432 23366 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Aug 10 04:41:04 dist-test-slave-bqcl krb5kdc[23212](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786336864, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.167.126@KRBTEST.COM
I20260810 04:41:04.904181 23350 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:41:04.889363 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48282 (local address 127.22.167.126:46389)
0810 04:41:04.889650 (+ 287us) server_negotiation.cc:206] Beginning negotiation
0810 04:41:04.889659 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 04:41:04.889781 (+ 122us) server_negotiation.cc:407] Connection header received
0810 04:41:04.892580 (+ 2799us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:41:04.892588 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:41:04.892637 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:41:04.892762 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0810 04:41:04.894412 (+ 1650us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:41:04.895639 (+ 1227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:41:04.897117 (+ 1478us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:41:04.897600 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:41:04.897697 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:41:04.900879 (+ 3182us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 04:41:04.900883 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 04:41:04.900883 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 04:41:04.900896 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0810 04:41:04.902471 (+ 1575us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 04:41:04.902855 (+ 384us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 04:41:04.902858 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 04:41:04.902858 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 04:41:04.902937 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 04:41:04.903165 (+ 228us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 04:41:04.903166 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 04:41:04.903166 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 04:41:04.903385 (+ 219us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 04:41:04.903539 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0810 04:41:04.903803 (+ 264us) server_negotiation.cc:299] Negotiation successful
0810 04:41:04.903926 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":243,"thread_start_us":118,"threads_started":1}
I20260810 04:41:04.905211 23366 catalog_manager.cc:1347] Generated new cluster ID: 74a1a853011941898fe504445c43b79c
I20260810 04:41:04.905267 23366 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 04:41:04.916517 23366 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 04:41:04.917232 23366 catalog_manager.cc:1504] Loading token signing keys...
I20260810 04:41:04.927361 23366 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 84b4c450220743d1b68a8dcea1dfba59: Generated new TSK 0
I20260810 04:41:04.927546 23366 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.22.167.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.167.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.167.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786336863388938-23197-0/krb5kdc/kudu_127.22.167.65.keytab.
Entry for principal kudu/127.22.167.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786336863388938-23197-0/krb5kdc/kudu_127.22.167.65.keytab.
I20260810 04:41:04.956557 23197 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskemv_AM/build/release/bin/kudu
/tmp/dist-test-taskemv_AM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786336863388938-23197-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786336863388938-23197-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786336863388938-23197-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786336863388938-23197-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.167.65:0
--local_ip_for_outbound_sockets=127.22.167.65
--webserver_interface=127.22.167.65
--webserver_port=0
--tserver_master_addrs=127.22.167.126:46389
--keytab_file=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786336863388938-23197-0/krb5kdc/kudu_127.22.167.65.keytab
--principal=kudu/127.22.167.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.167.84:45729
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786336863388938-23197-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786336863388938-23197-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786336863388938-23197-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 04:41:05.049572 23381 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 04:41:05.049827 23381 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 04:41:05.049862 23381 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 04:41:05.051249 23381 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 04:41:05.051295 23381 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 04:41:05.051333 23381 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.167.65
I20260810 04:41:05.053041 23381 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.167.84:45729
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786336863388938-23197-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786336863388938-23197-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786336863388938-23197-0/krb5kdc/kudu_127.22.167.65.keytab
--principal=kudu/127.22.167.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.167.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786336863388938-23197-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.22.167.65
--webserver_port=0
--tserver_master_addrs=127.22.167.126:46389
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23381
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.167.65
--log_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786336863388938-23197-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 03:54:01 UTC on ccycloud.quasar-icijmt.root.comops.site
build id 4577009
I20260810 04:41:05.053310 23381 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 04:41:05.053524 23381 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 04:41:05.058362 23392 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 04:41:05.058463 23388 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 04:41:05.058490 23387 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 04:41:05.058745 23381 server_base.cc:1034] running on GCE node
I20260810 04:41:05.058929 23381 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 04:41:05.059125 23381 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 04:41:05.060251 23381 hybrid_clock.cc:648] HybridClock initialized: now 1786336865060240 us; error 26 us; skew 500 ppm
Aug 10 04:41:05 dist-test-slave-bqcl krb5kdc[23212](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786336865, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.167.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 04:41:05.062624 23381 init.cc:377] Logged in from keytab as kudu/127.22.167.65@KRBTEST.COM (short username kudu)
I20260810 04:41:05.063054 23381 webserver.cc:460] Webserver started at http://127.22.167.65:32873/ using document root <none> and password file <none>
I20260810 04:41:05.063261 23381 fs_manager.cc:359] Metadata directory not provided
I20260810 04:41:05.063309 23381 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 04:41:05.063434 23381 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 04:41:05.064528 23381 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786336863388938-23197-0/minicluster-data/ts-0/data/instance:
uuid: "2414418ebadb40cba7c089eaf143360e"
format_stamp: "Formatted at 2026-08-10 04:41:05 on dist-test-slave-bqcl"
server_key: "dc07a5dd3aa204e5d9035f3e973353c9"
server_key_iv: "11d3b4b59119d28866830bbbfb8fca88"
server_key_version: "encryptionkey@0"
I20260810 04:41:05.064910 23381 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786336863388938-23197-0/minicluster-data/ts-0/wal/instance:
uuid: "2414418ebadb40cba7c089eaf143360e"
format_stamp: "Formatted at 2026-08-10 04:41:05 on dist-test-slave-bqcl"
server_key: "dc07a5dd3aa204e5d9035f3e973353c9"
server_key_iv: "11d3b4b59119d28866830bbbfb8fca88"
server_key_version: "encryptionkey@0"
I20260810 04:41:05.066489 23381 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260810 04:41:05.067255 23401 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 04:41:05.067420 23381 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 04:41:05.067503 23381 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786336863388938-23197-0/minicluster-data/ts-0/data,/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786336863388938-23197-0/minicluster-data/ts-0/wal
uuid: "2414418ebadb40cba7c089eaf143360e"
format_stamp: "Formatted at 2026-08-10 04:41:05 on dist-test-slave-bqcl"
server_key: "dc07a5dd3aa204e5d9035f3e973353c9"
server_key_iv: "11d3b4b59119d28866830bbbfb8fca88"
server_key_version: "encryptionkey@0"
I20260810 04:41:05.067564 23381 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786336863388938-23197-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786336863388938-23197-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786336863388938-23197-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 04:41:05.081699 23381 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 04:41:05.083734 23381 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 04:41:05.083925 23381 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 04:41:05.084302 23381 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 04:41:05.084658 23381 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 04:41:05.084700 23381 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 04:41:05.084728 23381 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 04:41:05.084744 23381 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 04:41:05.100507 23381 rpc_server.cc:307] RPC server started. Bound to: 127.22.167.65:36581
I20260810 04:41:05.100661 23582 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.167.65:36581 every 8 connection(s)
I20260810 04:41:05.100904 23381 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786336863388938-23197-0/minicluster-data/ts-0/data/info.pb
I20260810 04:41:05.101550 23197 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskemv_AM/build/release/bin/kudu as pid 23381
I20260810 04:41:05.101644 23197 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786336863388938-23197-0/minicluster-data/ts-0/wal/instance
I20260810 04:41:05.101799 23197 external_mini_cluster.cc:1428] Setting key f62d8ff710882ecff3297514bd1979e3
Aug 10 04:41:05 dist-test-slave-bqcl krb5kdc[23212](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786336865, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.167.65@KRBTEST.COM for kudu/127.22.167.126@KRBTEST.COM
I20260810 04:41:05.111509 23350 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:41:05.101653 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.167.65:58863 (local address 127.22.167.126:46389)
0810 04:41:05.101700 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0810 04:41:05.101701 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0810 04:41:05.102040 (+ 339us) server_negotiation.cc:407] Connection header received
0810 04:41:05.102638 (+ 598us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:41:05.102639 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:41:05.102651 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:41:05.102724 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0810 04:41:05.103660 (+ 936us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:41:05.104410 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:41:05.105853 (+ 1443us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:41:05.106161 (+ 308us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:41:05.106228 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:41:05.108692 (+ 2464us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 04:41:05.108693 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 04:41:05.108693 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 04:41:05.108701 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0810 04:41:05.110069 (+ 1368us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 04:41:05.110587 (+ 518us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 04:41:05.110588 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 04:41:05.110588 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 04:41:05.110647 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 04:41:05.110908 (+ 261us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 04:41:05.110908 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 04:41:05.110908 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 04:41:05.111009 (+ 101us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 04:41:05.111058 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0810 04:41:05.111372 (+ 314us) server_negotiation.cc:299] Negotiation successful
0810 04:41:05.111440 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260810 04:41:05.111829 23585 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:41:05.101635 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.126:46389 (local address 127.22.167.65:58863)
0810 04:41:05.101901 (+ 266us) negotiation.cc:107] Waiting for socket to connect
0810 04:41:05.101943 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0810 04:41:05.102543 (+ 600us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 04:41:05.102790 (+ 247us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 04:41:05.102793 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 04:41:05.103370 (+ 577us) client_negotiation.cc:189] Negotiated authn=SASL
0810 04:41:05.103563 (+ 193us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:41:05.103568 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:41:05.104516 (+ 948us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:41:05.104517 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:41:05.105771 (+ 1254us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:41:05.105776 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:41:05.106238 (+ 462us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:41:05.106239 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:41:05.106348 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:41:05.106862 (+ 514us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 04:41:05.106870 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0810 04:41:05.108576 (+ 1706us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 04:41:05.110172 (+ 1596us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 04:41:05.110180 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 04:41:05.110182 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 04:41:05.110502 (+ 320us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 04:41:05.110711 (+ 209us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 04:41:05.110712 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 04:41:05.110712 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 04:41:05.110852 (+ 140us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 04:41:05.111068 (+ 216us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 04:41:05.111075 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 04:41:05.111241 (+ 166us) client_negotiation.cc:769] Sending connection context
0810 04:41:05.111364 (+ 123us) client_negotiation.cc:240] Negotiation successful
0810 04:41:05.111540 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":225,"thread_start_us":192,"threads_started":1}
I20260810 04:41:05.112365 23583 heartbeater.cc:347] Connected to a master server at 127.22.167.126:46389
I20260810 04:41:05.112458 23583 heartbeater.cc:464] Registering TS with master...
I20260810 04:41:05.112618 23583 heartbeater.cc:511] Master 127.22.167.126:46389 requested a full tablet report, sending...
I20260810 04:41:05.113204 23275 ts_manager.cc:194] Registered new tserver with Master: 2414418ebadb40cba7c089eaf143360e (127.22.167.65:36581)
WARNING: no policy specified for kudu/127.22.167.66@KRBTEST.COM; defaulting to no policy
I20260810 04:41:05.114229 23275 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.167.65@KRBTEST.COM'} at 127.22.167.65:58863
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.167.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.167.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786336863388938-23197-0/krb5kdc/kudu_127.22.167.66.keytab.
Entry for principal kudu/127.22.167.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786336863388938-23197-0/krb5kdc/kudu_127.22.167.66.keytab.
I20260810 04:41:05.130616 23197 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskemv_AM/build/release/bin/kudu
/tmp/dist-test-taskemv_AM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786336863388938-23197-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786336863388938-23197-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786336863388938-23197-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786336863388938-23197-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.167.66:0
--local_ip_for_outbound_sockets=127.22.167.66
--webserver_interface=127.22.167.66
--webserver_port=0
--tserver_master_addrs=127.22.167.126:46389
--keytab_file=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786336863388938-23197-0/krb5kdc/kudu_127.22.167.66.keytab
--principal=kudu/127.22.167.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.167.84:45729
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786336863388938-23197-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786336863388938-23197-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786336863388938-23197-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 04:41:05.256642 23589 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 04:41:05.256862 23589 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 04:41:05.256888 23589 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 04:41:05.258033 23589 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 04:41:05.258075 23589 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 04:41:05.258105 23589 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.167.66
I20260810 04:41:05.259882 23589 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.167.84:45729
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786336863388938-23197-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786336863388938-23197-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786336863388938-23197-0/krb5kdc/kudu_127.22.167.66.keytab
--principal=kudu/127.22.167.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.167.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786336863388938-23197-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.22.167.66
--webserver_port=0
--tserver_master_addrs=127.22.167.126:46389
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23589
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.167.66
--log_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786336863388938-23197-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 03:54:01 UTC on ccycloud.quasar-icijmt.root.comops.site
build id 4577009
I20260810 04:41:05.260123 23589 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 04:41:05.260358 23589 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 04:41:05.265911 23602 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 04:41:05.266003 23589 server_base.cc:1034] running on GCE node
W20260810 04:41:05.265947 23605 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 04:41:05.265959 23600 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 04:41:05.266333 23589 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 04:41:05.266569 23589 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 04:41:05.267709 23589 hybrid_clock.cc:648] HybridClock initialized: now 1786336865267694 us; error 28 us; skew 500 ppm
Aug 10 04:41:05 dist-test-slave-bqcl krb5kdc[23212](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786336865, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.167.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 04:41:05.271102 23589 init.cc:377] Logged in from keytab as kudu/127.22.167.66@KRBTEST.COM (short username kudu)
I20260810 04:41:05.271543 23589 webserver.cc:460] Webserver started at http://127.22.167.66:39741/ using document root <none> and password file <none>
I20260810 04:41:05.271747 23589 fs_manager.cc:359] Metadata directory not provided
I20260810 04:41:05.271787 23589 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 04:41:05.271914 23589 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 04:41:05.273090 23589 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786336863388938-23197-0/minicluster-data/ts-1/data/instance:
uuid: "9942d98916de492f95a92901c69d4831"
format_stamp: "Formatted at 2026-08-10 04:41:05 on dist-test-slave-bqcl"
server_key: "b9777d6449518ecff0c3fd98e98ecf42"
server_key_iv: "9d1ec7d8c8518eb21b23aba98c1d9a09"
server_key_version: "encryptionkey@0"
I20260810 04:41:05.273432 23589 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786336863388938-23197-0/minicluster-data/ts-1/wal/instance:
uuid: "9942d98916de492f95a92901c69d4831"
format_stamp: "Formatted at 2026-08-10 04:41:05 on dist-test-slave-bqcl"
server_key: "b9777d6449518ecff0c3fd98e98ecf42"
server_key_iv: "9d1ec7d8c8518eb21b23aba98c1d9a09"
server_key_version: "encryptionkey@0"
I20260810 04:41:05.274956 23589 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260810 04:41:05.275663 23620 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 04:41:05.275812 23589 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260810 04:41:05.275895 23589 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786336863388938-23197-0/minicluster-data/ts-1/data,/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786336863388938-23197-0/minicluster-data/ts-1/wal
uuid: "9942d98916de492f95a92901c69d4831"
format_stamp: "Formatted at 2026-08-10 04:41:05 on dist-test-slave-bqcl"
server_key: "b9777d6449518ecff0c3fd98e98ecf42"
server_key_iv: "9d1ec7d8c8518eb21b23aba98c1d9a09"
server_key_version: "encryptionkey@0"
I20260810 04:41:05.275955 23589 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786336863388938-23197-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786336863388938-23197-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786336863388938-23197-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 04:41:05.325591 23589 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 04:41:05.327858 23589 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 04:41:05.328059 23589 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 04:41:05.328480 23589 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 04:41:05.328855 23589 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 04:41:05.328894 23589 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 04:41:05.328925 23589 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 04:41:05.328943 23589 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 04:41:05.344786 23589 rpc_server.cc:307] RPC server started. Bound to: 127.22.167.66:33557
I20260810 04:41:05.344959 23796 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.167.66:33557 every 8 connection(s)
I20260810 04:41:05.345229 23589 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786336863388938-23197-0/minicluster-data/ts-1/data/info.pb
I20260810 04:41:05.345816 23197 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskemv_AM/build/release/bin/kudu as pid 23589
I20260810 04:41:05.345909 23197 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786336863388938-23197-0/minicluster-data/ts-1/wal/instance
I20260810 04:41:05.346079 23197 external_mini_cluster.cc:1428] Setting key 935d574e637ba4e5dae9d7b2c3a4e568
Aug 10 04:41:05 dist-test-slave-bqcl krb5kdc[23212](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786336865, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.167.66@KRBTEST.COM for kudu/127.22.167.126@KRBTEST.COM
I20260810 04:41:05.357980 23350 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:41:05.346351 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.167.66:53653 (local address 127.22.167.126:46389)
0810 04:41:05.346407 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0810 04:41:05.346409 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 04:41:05.346559 (+ 150us) server_negotiation.cc:407] Connection header received
0810 04:41:05.347636 (+ 1077us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:41:05.347637 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:41:05.347647 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:41:05.347699 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0810 04:41:05.349223 (+ 1524us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:41:05.349943 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:41:05.351519 (+ 1576us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:41:05.351807 (+ 288us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:41:05.351856 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:41:05.354623 (+ 2767us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 04:41:05.354625 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 04:41:05.354625 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 04:41:05.354633 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0810 04:41:05.356185 (+ 1552us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 04:41:05.356747 (+ 562us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 04:41:05.356748 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 04:41:05.356749 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 04:41:05.356816 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 04:41:05.357172 (+ 356us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 04:41:05.357174 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 04:41:05.357174 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 04:41:05.357336 (+ 162us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 04:41:05.357405 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0810 04:41:05.357788 (+ 383us) server_negotiation.cc:299] Negotiation successful
0810 04:41:05.357876 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260810 04:41:05.358237 23799 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:41:05.346040 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.126:46389 (local address 127.22.167.66:53653)
0810 04:41:05.346396 (+ 356us) negotiation.cc:107] Waiting for socket to connect
0810 04:41:05.346446 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0810 04:41:05.347522 (+ 1076us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 04:41:05.347776 (+ 254us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 04:41:05.347781 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 04:41:05.348640 (+ 859us) client_negotiation.cc:189] Negotiated authn=SASL
0810 04:41:05.349105 (+ 465us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:41:05.349113 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:41:05.350046 (+ 933us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:41:05.350047 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:41:05.351408 (+ 1361us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:41:05.351414 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:41:05.351959 (+ 545us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:41:05.351960 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:41:05.352198 (+ 238us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:41:05.352810 (+ 612us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 04:41:05.352819 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0810 04:41:05.354520 (+ 1701us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 04:41:05.356280 (+ 1760us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 04:41:05.356288 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 04:41:05.356291 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 04:41:05.356643 (+ 352us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 04:41:05.356891 (+ 248us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 04:41:05.356909 (+ 18us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 04:41:05.356909 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 04:41:05.357069 (+ 160us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 04:41:05.357410 (+ 341us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 04:41:05.357418 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 04:41:05.357590 (+ 172us) client_negotiation.cc:769] Sending connection context
0810 04:41:05.357708 (+ 118us) client_negotiation.cc:240] Negotiation successful
0810 04:41:05.357880 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":281,"thread_start_us":195,"threads_started":1}
WARNING: no policy specified for kudu/127.22.167.67@KRBTEST.COM; defaulting to no policy
I20260810 04:41:05.364674 23797 heartbeater.cc:347] Connected to a master server at 127.22.167.126:46389
I20260810 04:41:05.364805 23797 heartbeater.cc:464] Registering TS with master...
I20260810 04:41:05.365046 23797 heartbeater.cc:511] Master 127.22.167.126:46389 requested a full tablet report, sending...
I20260810 04:41:05.365476 23275 ts_manager.cc:194] Registered new tserver with Master: 9942d98916de492f95a92901c69d4831 (127.22.167.66:33557)
I20260810 04:41:05.365912 23275 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.167.66@KRBTEST.COM'} at 127.22.167.66:53653
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.167.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.167.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786336863388938-23197-0/krb5kdc/kudu_127.22.167.67.keytab.
Entry for principal kudu/127.22.167.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786336863388938-23197-0/krb5kdc/kudu_127.22.167.67.keytab.
I20260810 04:41:05.382912 23197 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskemv_AM/build/release/bin/kudu
/tmp/dist-test-taskemv_AM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786336863388938-23197-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786336863388938-23197-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786336863388938-23197-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786336863388938-23197-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.167.67:0
--local_ip_for_outbound_sockets=127.22.167.67
--webserver_interface=127.22.167.67
--webserver_port=0
--tserver_master_addrs=127.22.167.126:46389
--keytab_file=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786336863388938-23197-0/krb5kdc/kudu_127.22.167.67.keytab
--principal=kudu/127.22.167.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.167.84:45729
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786336863388938-23197-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786336863388938-23197-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786336863388938-23197-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 04:41:05.479394 23804 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 04:41:05.479610 23804 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 04:41:05.479642 23804 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 04:41:05.480820 23804 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 04:41:05.480863 23804 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 04:41:05.480897 23804 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.167.67
I20260810 04:41:05.482556 23804 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.167.84:45729
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786336863388938-23197-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786336863388938-23197-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786336863388938-23197-0/krb5kdc/kudu_127.22.167.67.keytab
--principal=kudu/127.22.167.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.167.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786336863388938-23197-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.22.167.67
--webserver_port=0
--tserver_master_addrs=127.22.167.126:46389
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23804
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.167.67
--log_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786336863388938-23197-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 03:54:01 UTC on ccycloud.quasar-icijmt.root.comops.site
build id 4577009
I20260810 04:41:05.482810 23804 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 04:41:05.483024 23804 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 04:41:05.488546 23811 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 04:41:05.488639 23812 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 04:41:05.488557 23814 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 04:41:05.488591 23804 server_base.cc:1034] running on GCE node
I20260810 04:41:05.488902 23804 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 04:41:05.489154 23804 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 04:41:05.490315 23804 hybrid_clock.cc:648] HybridClock initialized: now 1786336865490297 us; error 27 us; skew 500 ppm
Aug 10 04:41:05 dist-test-slave-bqcl krb5kdc[23212](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786336865, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.167.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 04:41:05.493238 23804 init.cc:377] Logged in from keytab as kudu/127.22.167.67@KRBTEST.COM (short username kudu)
I20260810 04:41:05.493661 23804 webserver.cc:460] Webserver started at http://127.22.167.67:34205/ using document root <none> and password file <none>
I20260810 04:41:05.493854 23804 fs_manager.cc:359] Metadata directory not provided
I20260810 04:41:05.493901 23804 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 04:41:05.494027 23804 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 04:41:05.495074 23804 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786336863388938-23197-0/minicluster-data/ts-2/data/instance:
uuid: "de82a7a66a3e47d2b09f623042d00cda"
format_stamp: "Formatted at 2026-08-10 04:41:05 on dist-test-slave-bqcl"
server_key: "bc7598e3b1ab89504e29ef400d869c62"
server_key_iv: "5e3a8cec7ba52ad8323da0d19d1f1b1d"
server_key_version: "encryptionkey@0"
I20260810 04:41:05.495497 23804 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786336863388938-23197-0/minicluster-data/ts-2/wal/instance:
uuid: "de82a7a66a3e47d2b09f623042d00cda"
format_stamp: "Formatted at 2026-08-10 04:41:05 on dist-test-slave-bqcl"
server_key: "bc7598e3b1ab89504e29ef400d869c62"
server_key_iv: "5e3a8cec7ba52ad8323da0d19d1f1b1d"
server_key_version: "encryptionkey@0"
I20260810 04:41:05.496795 23804 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260810 04:41:05.497510 23821 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 04:41:05.497679 23804 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260810 04:41:05.497766 23804 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786336863388938-23197-0/minicluster-data/ts-2/data,/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786336863388938-23197-0/minicluster-data/ts-2/wal
uuid: "de82a7a66a3e47d2b09f623042d00cda"
format_stamp: "Formatted at 2026-08-10 04:41:05 on dist-test-slave-bqcl"
server_key: "bc7598e3b1ab89504e29ef400d869c62"
server_key_iv: "5e3a8cec7ba52ad8323da0d19d1f1b1d"
server_key_version: "encryptionkey@0"
I20260810 04:41:05.497829 23804 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786336863388938-23197-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786336863388938-23197-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786336863388938-23197-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 04:41:05.535156 23804 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 04:41:05.536908 23804 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 04:41:05.537062 23804 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 04:41:05.537288 23804 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 04:41:05.537595 23804 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 04:41:05.537631 23804 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 04:41:05.537659 23804 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 04:41:05.537676 23804 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 04:41:05.546986 23804 rpc_server.cc:307] RPC server started. Bound to: 127.22.167.67:36103
I20260810 04:41:05.547049 24003 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.167.67:36103 every 8 connection(s)
I20260810 04:41:05.547340 23804 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786336863388938-23197-0/minicluster-data/ts-2/data/info.pb
I20260810 04:41:05.547927 23197 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskemv_AM/build/release/bin/kudu as pid 23804
I20260810 04:41:05.548005 23197 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786336863388938-23197-0/minicluster-data/ts-2/wal/instance
I20260810 04:41:05.548164 23197 external_mini_cluster.cc:1428] Setting key 965fb2c99b81a37a6403c56a27acb648
Aug 10 04:41:05 dist-test-slave-bqcl krb5kdc[23212](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786336865, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.167.67@KRBTEST.COM for kudu/127.22.167.126@KRBTEST.COM
I20260810 04:41:05.555614 23350 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:41:05.548096 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.167.67:48175 (local address 127.22.167.126:46389)
0810 04:41:05.548161 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0810 04:41:05.548164 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 04:41:05.548330 (+ 166us) server_negotiation.cc:407] Connection header received
0810 04:41:05.548890 (+ 560us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:41:05.548891 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:41:05.548902 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:41:05.548955 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0810 04:41:05.549933 (+ 978us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:41:05.550478 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:41:05.551394 (+ 916us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:41:05.551574 (+ 180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:41:05.551620 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:41:05.553503 (+ 1883us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 04:41:05.553504 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 04:41:05.553504 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 04:41:05.553509 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0810 04:41:05.554610 (+ 1101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 04:41:05.554948 (+ 338us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 04:41:05.554948 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 04:41:05.554949 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 04:41:05.554992 (+ 43us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 04:41:05.555160 (+ 168us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 04:41:05.555161 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 04:41:05.555161 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 04:41:05.555251 (+ 90us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 04:41:05.555293 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0810 04:41:05.555495 (+ 202us) server_negotiation.cc:299] Negotiation successful
0810 04:41:05.555550 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260810 04:41:05.555784 24007 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:41:05.548023 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.126:46389 (local address 127.22.167.67:48175)
0810 04:41:05.548194 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0810 04:41:05.548236 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0810 04:41:05.548790 (+ 554us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 04:41:05.549028 (+ 238us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 04:41:05.549033 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 04:41:05.549654 (+ 621us) client_negotiation.cc:189] Negotiated authn=SASL
0810 04:41:05.549850 (+ 196us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:41:05.549855 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:41:05.550540 (+ 685us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:41:05.550541 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:41:05.551332 (+ 791us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:41:05.551334 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:41:05.551626 (+ 292us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:41:05.551627 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:41:05.551730 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:41:05.552159 (+ 429us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 04:41:05.552169 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0810 04:41:05.553434 (+ 1265us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 04:41:05.554679 (+ 1245us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 04:41:05.554685 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 04:41:05.554686 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 04:41:05.554890 (+ 204us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 04:41:05.555034 (+ 144us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 04:41:05.555034 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 04:41:05.555035 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 04:41:05.555120 (+ 85us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 04:41:05.555292 (+ 172us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 04:41:05.555295 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 04:41:05.555400 (+ 105us) client_negotiation.cc:769] Sending connection context
0810 04:41:05.555475 (+ 75us) client_negotiation.cc:240] Negotiation successful
0810 04:41:05.555586 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":83,"threads_started":1}
I20260810 04:41:05.556358 24004 heartbeater.cc:347] Connected to a master server at 127.22.167.126:46389
I20260810 04:41:05.556442 24004 heartbeater.cc:464] Registering TS with master...
I20260810 04:41:05.556578 24004 heartbeater.cc:511] Master 127.22.167.126:46389 requested a full tablet report, sending...
I20260810 04:41:05.556917 23275 ts_manager.cc:194] Registered new tserver with Master: de82a7a66a3e47d2b09f623042d00cda (127.22.167.67:36103)
I20260810 04:41:05.557253 23275 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.167.67@KRBTEST.COM'} at 127.22.167.67:48175
I20260810 04:41:05.559934 23197 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 10 04:41:05 dist-test-slave-bqcl krb5kdc[23212](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786336865, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 10 04:41:05 dist-test-slave-bqcl krb5kdc[23212](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786336865, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.22.167.65@KRBTEST.COM
I20260810 04:41:05.582981 24020 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:41:05.576347 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43172 (local address 127.22.167.65:36581)
0810 04:41:05.576483 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0810 04:41:05.576488 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 04:41:05.576498 (+ 10us) server_negotiation.cc:407] Connection header received
0810 04:41:05.576522 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:41:05.576524 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:41:05.576538 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:41:05.576603 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0810 04:41:05.577085 (+ 482us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:41:05.577766 (+ 681us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:41:05.578465 (+ 699us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:41:05.578777 (+ 312us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:41:05.578829 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:41:05.580794 (+ 1965us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 04:41:05.580802 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 04:41:05.580802 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 04:41:05.580808 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0810 04:41:05.582038 (+ 1230us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 04:41:05.582354 (+ 316us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 04:41:05.582357 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 04:41:05.582357 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 04:41:05.582428 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 04:41:05.582560 (+ 132us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 04:41:05.582560 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 04:41:05.582560 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 04:41:05.582675 (+ 115us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 04:41:05.582771 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0810 04:41:05.582824 (+ 53us) server_negotiation.cc:299] Negotiation successful
0810 04:41:05.582884 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":109,"thread_start_us":73,"threads_started":1}
W20260810 04:41:05.583297 23445 server_base.cc:1116] Unauthorized access attempt to method kudu.tserver.TabletServerService.ListTablets from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:43172
Aug 10 04:41:05 dist-test-slave-bqcl krb5kdc[23212](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786336865, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 10 04:41:05 dist-test-slave-bqcl krb5kdc[23212](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786336865, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.167.65@KRBTEST.COM
I20260810 04:41:05.605206 24020 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:41:05.599033 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43188 (local address 127.22.167.65:36581)
0810 04:41:05.599074 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0810 04:41:05.599076 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 04:41:05.599191 (+ 115us) server_negotiation.cc:407] Connection header received
0810 04:41:05.599256 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:41:05.599257 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:41:05.599268 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:41:05.599312 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0810 04:41:05.599830 (+ 518us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:41:05.600321 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:41:05.601041 (+ 720us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:41:05.601241 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:41:05.601293 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:41:05.603217 (+ 1924us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 04:41:05.603218 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 04:41:05.603219 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 04:41:05.603226 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 04:41:05.604262 (+ 1036us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 04:41:05.604544 (+ 282us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 04:41:05.604545 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 04:41:05.604545 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 04:41:05.604594 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 04:41:05.604763 (+ 169us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 04:41:05.604764 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 04:41:05.604764 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 04:41:05.604850 (+ 86us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 04:41:05.604954 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0810 04:41:05.605035 (+ 81us) server_negotiation.cc:299] Negotiation successful
0810 04:41:05.605109 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260810 04:41:05.605672 23197 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskemv_AM/build/release/bin/kudu with pid 23381
I20260810 04:41:05.621894 23197 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskemv_AM/build/release/bin/kudu with pid 23589
I20260810 04:41:05.629202 23197 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskemv_AM/build/release/bin/kudu with pid 23804
I20260810 04:41:05.638149 23197 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskemv_AM/build/release/bin/kudu with pid 23234
2026-08-10T04:41:05Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnListTablets (2261 ms)
[ RUN ] SecurityITest.TestAuthorizationOnChecksum
Loading random data
Initializing database '/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786336863388938-23197-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 04:41:05 dist-test-slave-bqcl krb5kdc[24032](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 04:41:05 dist-test-slave-bqcl krb5kdc[24032](info): set up 2 sockets
Aug 10 04:41:05 dist-test-slave-bqcl krb5kdc[24032](info): commencing operation
krb5kdc: starting...
W20260810 04:41:06.856472 23197 mini_kdc.cc:121] Time spent starting KDC: real 1.194s user 0.003s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786336863388938-23197-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786336863388938-23197-0/krb5kdc/test-user.keytab.
Aug 10 04:41:06 dist-test-slave-bqcl krb5kdc[24032](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786336866, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T04:41:06Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T04:41:06Z Disabled control of system clock
WARNING: no policy specified for kudu/127.22.167.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.167.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.167.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786336863388938-23197-0/krb5kdc/kudu_127.22.167.126.keytab.
Entry for principal kudu/127.22.167.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786336863388938-23197-0/krb5kdc/kudu_127.22.167.126.keytab.
I20260810 04:41:06.970443 23197 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskemv_AM/build/release/bin/kudu
/tmp/dist-test-taskemv_AM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786336863388938-23197-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786336863388938-23197-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786336863388938-23197-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786336863388938-23197-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.167.126:41843
--webserver_interface=127.22.167.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786336863388938-23197-0/krb5kdc/kudu_127.22.167.126.keytab
--principal=kudu/127.22.167.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.167.84:46537
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.167.126:41843
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786336863388938-23197-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786336863388938-23197-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786336863388938-23197-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 04:41:07.050338 24052 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 04:41:07.050534 24052 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 04:41:07.050563 24052 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 04:41:07.051657 24052 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 04:41:07.051705 24052 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 04:41:07.051724 24052 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 04:41:07.051741 24052 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 04:41:07.051756 24052 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 04:41:07.053177 24052 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.167.84:46537
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786336863388938-23197-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786336863388938-23197-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.22.167.126:41843
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786336863388938-23197-0/krb5kdc/kudu_127.22.167.126.keytab
--principal=kudu/127.22.167.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.167.126:41843
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786336863388938-23197-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.167.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24052
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786336863388938-23197-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 03:54:01 UTC on ccycloud.quasar-icijmt.root.comops.site
build id 4577009
I20260810 04:41:07.053413 24052 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 04:41:07.053623 24052 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 04:41:07.058947 24063 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 04:41:07.058954 24066 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 04:41:07.058954 24062 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 04:41:07.059115 24052 server_base.cc:1034] running on GCE node
I20260810 04:41:07.059417 24052 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 04:41:07.059695 24052 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 04:41:07.060824 24052 hybrid_clock.cc:648] HybridClock initialized: now 1786336867060808 us; error 31 us; skew 500 ppm
Aug 10 04:41:07 dist-test-slave-bqcl krb5kdc[24032](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786336867, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.167.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 04:41:07.063222 24052 init.cc:377] Logged in from keytab as kudu/127.22.167.126@KRBTEST.COM (short username kudu)
I20260810 04:41:07.063644 24052 webserver.cc:460] Webserver started at http://127.22.167.126:33503/ using document root <none> and password file <none>
I20260810 04:41:07.063858 24052 fs_manager.cc:359] Metadata directory not provided
I20260810 04:41:07.063910 24052 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 04:41:07.064038 24052 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 04:41:07.065029 24052 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786336863388938-23197-0/minicluster-data/master-0/data/instance:
uuid: "4cca811c99f849c1847ed703acb3134b"
format_stamp: "Formatted at 2026-08-10 04:41:07 on dist-test-slave-bqcl"
server_key: "dbadb113aae586e19b9dd329a5a0eea2"
server_key_iv: "c6d17d0c2d5a49d5a5d88a057612a790"
server_key_version: "encryptionkey@0"
I20260810 04:41:07.065356 24052 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786336863388938-23197-0/minicluster-data/master-0/wal/instance:
uuid: "4cca811c99f849c1847ed703acb3134b"
format_stamp: "Formatted at 2026-08-10 04:41:07 on dist-test-slave-bqcl"
server_key: "dbadb113aae586e19b9dd329a5a0eea2"
server_key_iv: "c6d17d0c2d5a49d5a5d88a057612a790"
server_key_version: "encryptionkey@0"
I20260810 04:41:07.066624 24052 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260810 04:41:07.067333 24075 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 04:41:07.067498 24052 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260810 04:41:07.067581 24052 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786336863388938-23197-0/minicluster-data/master-0/data,/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786336863388938-23197-0/minicluster-data/master-0/wal
uuid: "4cca811c99f849c1847ed703acb3134b"
format_stamp: "Formatted at 2026-08-10 04:41:07 on dist-test-slave-bqcl"
server_key: "dbadb113aae586e19b9dd329a5a0eea2"
server_key_iv: "c6d17d0c2d5a49d5a5d88a057612a790"
server_key_version: "encryptionkey@0"
I20260810 04:41:07.067639 24052 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786336863388938-23197-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786336863388938-23197-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786336863388938-23197-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 04:41:07.081686 24052 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 04:41:07.083803 24052 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 04:41:07.083945 24052 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 04:41:07.087842 24052 rpc_server.cc:307] RPC server started. Bound to: 127.22.167.126:41843
I20260810 04:41:07.087884 24164 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.167.126:41843 every 8 connection(s)
I20260810 04:41:07.088201 24052 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786336863388938-23197-0/minicluster-data/master-0/data/info.pb
I20260810 04:41:07.089030 24165 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 04:41:07.091287 24165 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4cca811c99f849c1847ed703acb3134b: Bootstrap starting.
I20260810 04:41:07.091854 24165 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4cca811c99f849c1847ed703acb3134b: Neither blocks nor log segments found. Creating new log.
I20260810 04:41:07.092111 24165 log.cc:824] T 00000000000000000000000000000000 P 4cca811c99f849c1847ed703acb3134b: Log is configured to *not* fsync() on all Append() calls
I20260810 04:41:07.092757 24165 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4cca811c99f849c1847ed703acb3134b: No bootstrap required, opened a new log
I20260810 04:41:07.094434 24165 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4cca811c99f849c1847ed703acb3134b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4cca811c99f849c1847ed703acb3134b" member_type: VOTER last_known_addr { host: "127.22.167.126" port: 41843 } }
I20260810 04:41:07.094573 24165 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4cca811c99f849c1847ed703acb3134b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 04:41:07.094611 24165 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4cca811c99f849c1847ed703acb3134b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4cca811c99f849c1847ed703acb3134b, State: Initialized, Role: FOLLOWER
I20260810 04:41:07.094774 24165 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4cca811c99f849c1847ed703acb3134b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4cca811c99f849c1847ed703acb3134b" member_type: VOTER last_known_addr { host: "127.22.167.126" port: 41843 } }
I20260810 04:41:07.094852 24165 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4cca811c99f849c1847ed703acb3134b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 04:41:07.094837 23197 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskemv_AM/build/release/bin/kudu as pid 24052
I20260810 04:41:07.094877 24165 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4cca811c99f849c1847ed703acb3134b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 04:41:07.094918 24165 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4cca811c99f849c1847ed703acb3134b [term 0 FOLLOWER]: Advancing to term 1
I20260810 04:41:07.094929 23197 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786336863388938-23197-0/minicluster-data/master-0/wal/instance
I20260810 04:41:07.095059 23197 external_mini_cluster.cc:1428] Setting key f1879b3980cfaccbb1b7f9038f8ac488
I20260810 04:41:07.095530 24165 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4cca811c99f849c1847ed703acb3134b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4cca811c99f849c1847ed703acb3134b" member_type: VOTER last_known_addr { host: "127.22.167.126" port: 41843 } }
I20260810 04:41:07.095721 24165 leader_election.cc:302] T 00000000000000000000000000000000 P 4cca811c99f849c1847ed703acb3134b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4cca811c99f849c1847ed703acb3134b; no voters:
I20260810 04:41:07.095887 24165 leader_election.cc:288] T 00000000000000000000000000000000 P 4cca811c99f849c1847ed703acb3134b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 04:41:07.095928 24178 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4cca811c99f849c1847ed703acb3134b [term 1 FOLLOWER]: Leader election won for term 1
I20260810 04:41:07.096082 24178 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4cca811c99f849c1847ed703acb3134b [term 1 LEADER]: Becoming Leader. State: Replica: 4cca811c99f849c1847ed703acb3134b, State: Running, Role: LEADER
I20260810 04:41:07.096122 24165 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4cca811c99f849c1847ed703acb3134b [sys.catalog]: configured and running, proceeding with master startup.
I20260810 04:41:07.097143 24178 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4cca811c99f849c1847ed703acb3134b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4cca811c99f849c1847ed703acb3134b" member_type: VOTER last_known_addr { host: "127.22.167.126" port: 41843 } }
I20260810 04:41:07.097842 24179 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4cca811c99f849c1847ed703acb3134b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4cca811c99f849c1847ed703acb3134b. Latest consensus state: current_term: 1 leader_uuid: "4cca811c99f849c1847ed703acb3134b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4cca811c99f849c1847ed703acb3134b" member_type: VOTER last_known_addr { host: "127.22.167.126" port: 41843 } } }
I20260810 04:41:07.097862 24182 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4cca811c99f849c1847ed703acb3134b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4cca811c99f849c1847ed703acb3134b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4cca811c99f849c1847ed703acb3134b" member_type: VOTER last_known_addr { host: "127.22.167.126" port: 41843 } } }
I20260810 04:41:07.097932 24179 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4cca811c99f849c1847ed703acb3134b [sys.catalog]: This master's current role is: LEADER
I20260810 04:41:07.097950 24182 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4cca811c99f849c1847ed703acb3134b [sys.catalog]: This master's current role is: LEADER
I20260810 04:41:07.098234 24199 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 04:41:07.098698 24199 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 04:41:07.100114 24199 catalog_manager.cc:1347] Generated new cluster ID: ebd690ee8300463c9891acdf341a16fb
I20260810 04:41:07.100162 24199 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 10 04:41:07 dist-test-slave-bqcl krb5kdc[24032](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786336866, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.167.126@KRBTEST.COM
I20260810 04:41:07.105997 24177 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:41:07.095494 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53436 (local address 127.22.167.126:41843)
0810 04:41:07.095684 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0810 04:41:07.095694 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 04:41:07.095729 (+ 35us) server_negotiation.cc:407] Connection header received
0810 04:41:07.095844 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:41:07.095851 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:41:07.095907 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:41:07.096044 (+ 137us) server_negotiation.cc:226] Negotiated authn=SASL
0810 04:41:07.096825 (+ 781us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:41:07.098216 (+ 1391us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:41:07.099229 (+ 1013us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:41:07.099774 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:41:07.099881 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:41:07.102418 (+ 2537us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 04:41:07.102422 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 04:41:07.102422 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 04:41:07.102433 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0810 04:41:07.104229 (+ 1796us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 04:41:07.104586 (+ 357us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 04:41:07.104589 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 04:41:07.104589 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 04:41:07.104701 (+ 112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 04:41:07.104877 (+ 176us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 04:41:07.104878 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 04:41:07.104878 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 04:41:07.105149 (+ 271us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 04:41:07.105613 (+ 464us) server_negotiation.cc:1035] Waiting for connection context
0810 04:41:07.105677 (+ 64us) server_negotiation.cc:299] Negotiation successful
0810 04:41:07.105799 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":137,"thread_start_us":99,"threads_started":1}
I20260810 04:41:07.108493 24199 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 04:41:07.108949 24199 catalog_manager.cc:1504] Loading token signing keys...
I20260810 04:41:07.111562 24199 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4cca811c99f849c1847ed703acb3134b: Generated new TSK 0
I20260810 04:41:07.111719 24199 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.22.167.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.167.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.167.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786336863388938-23197-0/krb5kdc/kudu_127.22.167.65.keytab.
Entry for principal kudu/127.22.167.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786336863388938-23197-0/krb5kdc/kudu_127.22.167.65.keytab.
I20260810 04:41:07.145053 23197 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskemv_AM/build/release/bin/kudu
/tmp/dist-test-taskemv_AM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786336863388938-23197-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786336863388938-23197-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786336863388938-23197-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786336863388938-23197-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.167.65:0
--local_ip_for_outbound_sockets=127.22.167.65
--webserver_interface=127.22.167.65
--webserver_port=0
--tserver_master_addrs=127.22.167.126:41843
--keytab_file=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786336863388938-23197-0/krb5kdc/kudu_127.22.167.65.keytab
--principal=kudu/127.22.167.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.167.84:46537
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786336863388938-23197-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786336863388938-23197-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786336863388938-23197-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 04:41:07.225458 24210 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 04:41:07.225664 24210 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 04:41:07.225687 24210 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 04:41:07.226831 24210 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 04:41:07.226873 24210 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 04:41:07.226897 24210 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.167.65
I20260810 04:41:07.228343 24210 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.167.84:46537
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786336863388938-23197-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786336863388938-23197-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786336863388938-23197-0/krb5kdc/kudu_127.22.167.65.keytab
--principal=kudu/127.22.167.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.167.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786336863388938-23197-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.22.167.65
--webserver_port=0
--tserver_master_addrs=127.22.167.126:41843
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24210
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.167.65
--log_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786336863388938-23197-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 03:54:01 UTC on ccycloud.quasar-icijmt.root.comops.site
build id 4577009
I20260810 04:41:07.228529 24210 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 04:41:07.228710 24210 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 04:41:07.233433 24218 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 04:41:07.233439 24216 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 04:41:07.233587 24210 server_base.cc:1034] running on GCE node
W20260810 04:41:07.233445 24215 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 04:41:07.233846 24210 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 04:41:07.234100 24210 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 04:41:07.235268 24210 hybrid_clock.cc:648] HybridClock initialized: now 1786336867235255 us; error 32 us; skew 500 ppm
Aug 10 04:41:07 dist-test-slave-bqcl krb5kdc[24032](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786336867, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.167.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 04:41:07.237717 24210 init.cc:377] Logged in from keytab as kudu/127.22.167.65@KRBTEST.COM (short username kudu)
I20260810 04:41:07.238108 24210 webserver.cc:460] Webserver started at http://127.22.167.65:33605/ using document root <none> and password file <none>
I20260810 04:41:07.238365 24210 fs_manager.cc:359] Metadata directory not provided
I20260810 04:41:07.238410 24210 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 04:41:07.238534 24210 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 04:41:07.239523 24210 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786336863388938-23197-0/minicluster-data/ts-0/data/instance:
uuid: "075bc7ddc396438f8df780c21cf9fc80"
format_stamp: "Formatted at 2026-08-10 04:41:07 on dist-test-slave-bqcl"
server_key: "e773cb7f8b543ce71fa0cb70b54bc6d5"
server_key_iv: "968e181c04c2e1f29df32d98ab9199c3"
server_key_version: "encryptionkey@0"
I20260810 04:41:07.239840 24210 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786336863388938-23197-0/minicluster-data/ts-0/wal/instance:
uuid: "075bc7ddc396438f8df780c21cf9fc80"
format_stamp: "Formatted at 2026-08-10 04:41:07 on dist-test-slave-bqcl"
server_key: "e773cb7f8b543ce71fa0cb70b54bc6d5"
server_key_iv: "968e181c04c2e1f29df32d98ab9199c3"
server_key_version: "encryptionkey@0"
I20260810 04:41:07.241086 24210 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260810 04:41:07.241720 24233 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 04:41:07.241858 24210 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260810 04:41:07.241941 24210 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786336863388938-23197-0/minicluster-data/ts-0/data,/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786336863388938-23197-0/minicluster-data/ts-0/wal
uuid: "075bc7ddc396438f8df780c21cf9fc80"
format_stamp: "Formatted at 2026-08-10 04:41:07 on dist-test-slave-bqcl"
server_key: "e773cb7f8b543ce71fa0cb70b54bc6d5"
server_key_iv: "968e181c04c2e1f29df32d98ab9199c3"
server_key_version: "encryptionkey@0"
I20260810 04:41:07.242002 24210 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786336863388938-23197-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786336863388938-23197-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786336863388938-23197-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 04:41:07.259423 24210 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 04:41:07.261548 24210 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 04:41:07.261687 24210 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 04:41:07.261906 24210 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 04:41:07.262279 24210 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 04:41:07.262316 24210 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 04:41:07.262347 24210 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 04:41:07.262367 24210 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 04:41:07.267798 24210 rpc_server.cc:307] RPC server started. Bound to: 127.22.167.65:39677
I20260810 04:41:07.267858 24411 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.167.65:39677 every 8 connection(s)
I20260810 04:41:07.268147 24210 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786336863388938-23197-0/minicluster-data/ts-0/data/info.pb
I20260810 04:41:07.269855 23197 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskemv_AM/build/release/bin/kudu as pid 24210
I20260810 04:41:07.269941 23197 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786336863388938-23197-0/minicluster-data/ts-0/wal/instance
I20260810 04:41:07.270143 23197 external_mini_cluster.cc:1428] Setting key cd59e155a17e16cd358ae15a9f61ecff
Aug 10 04:41:07 dist-test-slave-bqcl krb5kdc[24032](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786336867, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.167.65@KRBTEST.COM for kudu/127.22.167.126@KRBTEST.COM
I20260810 04:41:07.277668 24177 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:41:07.268813 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.167.65:37531 (local address 127.22.167.126:41843)
0810 04:41:07.268893 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0810 04:41:07.268895 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 04:41:07.269054 (+ 159us) server_negotiation.cc:407] Connection header received
0810 04:41:07.270146 (+ 1092us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:41:07.270148 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:41:07.270161 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:41:07.270225 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0810 04:41:07.271041 (+ 816us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:41:07.271574 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:41:07.272848 (+ 1274us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:41:07.273046 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:41:07.273108 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:41:07.275191 (+ 2083us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 04:41:07.275192 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 04:41:07.275192 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 04:41:07.275199 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 04:41:07.276399 (+ 1200us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 04:41:07.276846 (+ 447us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 04:41:07.276847 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 04:41:07.276847 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 04:41:07.276889 (+ 42us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 04:41:07.277110 (+ 221us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 04:41:07.277110 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 04:41:07.277110 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 04:41:07.277199 (+ 89us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 04:41:07.277246 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0810 04:41:07.277543 (+ 297us) server_negotiation.cc:299] Negotiation successful
0810 04:41:07.277606 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260810 04:41:07.277913 24415 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:41:07.268782 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.126:41843 (local address 127.22.167.65:37531)
0810 04:41:07.268937 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0810 04:41:07.268974 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0810 04:41:07.269991 (+ 1017us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 04:41:07.270363 (+ 372us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 04:41:07.270367 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 04:41:07.270682 (+ 315us) client_negotiation.cc:189] Negotiated authn=SASL
0810 04:41:07.270946 (+ 264us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:41:07.270953 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:41:07.271660 (+ 707us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:41:07.271661 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:41:07.272768 (+ 1107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:41:07.272770 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:41:07.273122 (+ 352us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:41:07.273123 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:41:07.273253 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:41:07.273679 (+ 426us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 04:41:07.273686 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0810 04:41:07.275097 (+ 1411us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 04:41:07.276475 (+ 1378us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 04:41:07.276483 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 04:41:07.276486 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 04:41:07.276775 (+ 289us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 04:41:07.276928 (+ 153us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 04:41:07.276929 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 04:41:07.276929 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 04:41:07.277059 (+ 130us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 04:41:07.277254 (+ 195us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 04:41:07.277260 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 04:41:07.277416 (+ 156us) client_negotiation.cc:769] Sending connection context
0810 04:41:07.277535 (+ 119us) client_negotiation.cc:240] Negotiation successful
0810 04:41:07.277695 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":79,"threads_started":1}
I20260810 04:41:07.278472 24413 heartbeater.cc:347] Connected to a master server at 127.22.167.126:41843
I20260810 04:41:07.278565 24413 heartbeater.cc:464] Registering TS with master...
I20260810 04:41:07.278733 24413 heartbeater.cc:511] Master 127.22.167.126:41843 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.22.167.66@KRBTEST.COM; defaulting to no policy
I20260810 04:41:07.279294 24108 ts_manager.cc:194] Registered new tserver with Master: 075bc7ddc396438f8df780c21cf9fc80 (127.22.167.65:39677)
I20260810 04:41:07.279907 24108 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.167.65@KRBTEST.COM'} at 127.22.167.65:37531
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.167.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.167.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786336863388938-23197-0/krb5kdc/kudu_127.22.167.66.keytab.
Entry for principal kudu/127.22.167.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786336863388938-23197-0/krb5kdc/kudu_127.22.167.66.keytab.
I20260810 04:41:07.294477 23197 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskemv_AM/build/release/bin/kudu
/tmp/dist-test-taskemv_AM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786336863388938-23197-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786336863388938-23197-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786336863388938-23197-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786336863388938-23197-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.167.66:0
--local_ip_for_outbound_sockets=127.22.167.66
--webserver_interface=127.22.167.66
--webserver_port=0
--tserver_master_addrs=127.22.167.126:41843
--keytab_file=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786336863388938-23197-0/krb5kdc/kudu_127.22.167.66.keytab
--principal=kudu/127.22.167.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.167.84:46537
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786336863388938-23197-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786336863388938-23197-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786336863388938-23197-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 04:41:07.376464 24422 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 04:41:07.376665 24422 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 04:41:07.376688 24422 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 04:41:07.377780 24422 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 04:41:07.377818 24422 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 04:41:07.377847 24422 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.167.66
I20260810 04:41:07.379351 24422 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.167.84:46537
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786336863388938-23197-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786336863388938-23197-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786336863388938-23197-0/krb5kdc/kudu_127.22.167.66.keytab
--principal=kudu/127.22.167.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.167.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786336863388938-23197-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.22.167.66
--webserver_port=0
--tserver_master_addrs=127.22.167.126:41843
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24422
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.167.66
--log_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786336863388938-23197-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 03:54:01 UTC on ccycloud.quasar-icijmt.root.comops.site
build id 4577009
I20260810 04:41:07.379566 24422 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 04:41:07.379768 24422 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 04:41:07.384466 24431 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 04:41:07.384513 24435 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 04:41:07.384474 24433 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 04:41:07.384688 24422 server_base.cc:1034] running on GCE node
I20260810 04:41:07.384919 24422 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 04:41:07.385134 24422 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 04:41:07.386265 24422 hybrid_clock.cc:648] HybridClock initialized: now 1786336867386259 us; error 36 us; skew 500 ppm
Aug 10 04:41:07 dist-test-slave-bqcl krb5kdc[24032](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786336867, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.167.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 04:41:07.388860 24422 init.cc:377] Logged in from keytab as kudu/127.22.167.66@KRBTEST.COM (short username kudu)
I20260810 04:41:07.389307 24422 webserver.cc:460] Webserver started at http://127.22.167.66:35765/ using document root <none> and password file <none>
I20260810 04:41:07.389575 24422 fs_manager.cc:359] Metadata directory not provided
I20260810 04:41:07.389633 24422 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 04:41:07.389773 24422 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 04:41:07.390829 24422 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786336863388938-23197-0/minicluster-data/ts-1/data/instance:
uuid: "8ede0cae26214b1bb27df91d3571f281"
format_stamp: "Formatted at 2026-08-10 04:41:07 on dist-test-slave-bqcl"
server_key: "862bb5d4bf6df5afa4c7989814269702"
server_key_iv: "7b825611364b870973c24b58c0a8c81d"
server_key_version: "encryptionkey@0"
I20260810 04:41:07.391160 24422 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786336863388938-23197-0/minicluster-data/ts-1/wal/instance:
uuid: "8ede0cae26214b1bb27df91d3571f281"
format_stamp: "Formatted at 2026-08-10 04:41:07 on dist-test-slave-bqcl"
server_key: "862bb5d4bf6df5afa4c7989814269702"
server_key_iv: "7b825611364b870973c24b58c0a8c81d"
server_key_version: "encryptionkey@0"
I20260810 04:41:07.392380 24422 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260810 04:41:07.393007 24445 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 04:41:07.393148 24422 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260810 04:41:07.393231 24422 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786336863388938-23197-0/minicluster-data/ts-1/data,/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786336863388938-23197-0/minicluster-data/ts-1/wal
uuid: "8ede0cae26214b1bb27df91d3571f281"
format_stamp: "Formatted at 2026-08-10 04:41:07 on dist-test-slave-bqcl"
server_key: "862bb5d4bf6df5afa4c7989814269702"
server_key_iv: "7b825611364b870973c24b58c0a8c81d"
server_key_version: "encryptionkey@0"
I20260810 04:41:07.393290 24422 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786336863388938-23197-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786336863388938-23197-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786336863388938-23197-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 04:41:07.412446 24422 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 04:41:07.414481 24422 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 04:41:07.414645 24422 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 04:41:07.414857 24422 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 04:41:07.415163 24422 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 04:41:07.415198 24422 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 04:41:07.415220 24422 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 04:41:07.415244 24422 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 04:41:07.420531 24422 rpc_server.cc:307] RPC server started. Bound to: 127.22.167.66:44505
I20260810 04:41:07.420605 24633 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.167.66:44505 every 8 connection(s)
I20260810 04:41:07.420892 24422 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786336863388938-23197-0/minicluster-data/ts-1/data/info.pb
Aug 10 04:41:07 dist-test-slave-bqcl krb5kdc[24032](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786336867, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.167.66@KRBTEST.COM for kudu/127.22.167.126@KRBTEST.COM
I20260810 04:41:07.429281 23197 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskemv_AM/build/release/bin/kudu as pid 24422
I20260810 04:41:07.429384 23197 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786336863388938-23197-0/minicluster-data/ts-1/wal/instance
I20260810 04:41:07.429612 23197 external_mini_cluster.cc:1428] Setting key ac019ffe9547df858eedb2b23e0cbd28
I20260810 04:41:07.430769 24177 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:41:07.421609 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.167.66:49347 (local address 127.22.167.126:41843)
0810 04:41:07.421690 (+ 81us) server_negotiation.cc:206] Beginning negotiation
0810 04:41:07.421692 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 04:41:07.421860 (+ 168us) server_negotiation.cc:407] Connection header received
0810 04:41:07.422826 (+ 966us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:41:07.422827 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:41:07.422839 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:41:07.422893 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0810 04:41:07.423543 (+ 650us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:41:07.424043 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:41:07.425037 (+ 994us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:41:07.425260 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:41:07.425319 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:41:07.427566 (+ 2247us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 04:41:07.427568 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 04:41:07.427569 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 04:41:07.427579 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0810 04:41:07.429023 (+ 1444us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 04:41:07.429540 (+ 517us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 04:41:07.429541 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 04:41:07.429541 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 04:41:07.429606 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 04:41:07.429954 (+ 348us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 04:41:07.429955 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 04:41:07.429956 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 04:41:07.430117 (+ 161us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 04:41:07.430199 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0810 04:41:07.430571 (+ 372us) server_negotiation.cc:299] Negotiation successful
0810 04:41:07.430668 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260810 04:41:07.431286 24639 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:41:07.421569 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.126:41843 (local address 127.22.167.66:49347)
0810 04:41:07.421741 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0810 04:41:07.421778 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0810 04:41:07.422709 (+ 931us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 04:41:07.422965 (+ 256us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 04:41:07.422969 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 04:41:07.423248 (+ 279us) client_negotiation.cc:189] Negotiated authn=SASL
0810 04:41:07.423448 (+ 200us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:41:07.423454 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:41:07.424131 (+ 677us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:41:07.424132 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:41:07.424951 (+ 819us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:41:07.424953 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:41:07.425334 (+ 381us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:41:07.425335 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:41:07.425456 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:41:07.425948 (+ 492us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 04:41:07.425956 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0810 04:41:07.427439 (+ 1483us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 04:41:07.429124 (+ 1685us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 04:41:07.429133 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 04:41:07.429135 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 04:41:07.429445 (+ 310us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 04:41:07.429677 (+ 232us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 04:41:07.429678 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 04:41:07.429678 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 04:41:07.429849 (+ 171us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 04:41:07.430227 (+ 378us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 04:41:07.430234 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 04:41:07.430406 (+ 172us) client_negotiation.cc:769] Sending connection context
0810 04:41:07.430904 (+ 498us) client_negotiation.cc:240] Negotiation successful
0810 04:41:07.431103 (+ 199us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":88,"threads_started":1}
I20260810 04:41:07.431805 24635 heartbeater.cc:347] Connected to a master server at 127.22.167.126:41843
I20260810 04:41:07.431903 24635 heartbeater.cc:464] Registering TS with master...
I20260810 04:41:07.432097 24635 heartbeater.cc:511] Master 127.22.167.126:41843 requested a full tablet report, sending...
I20260810 04:41:07.432480 24108 ts_manager.cc:194] Registered new tserver with Master: 8ede0cae26214b1bb27df91d3571f281 (127.22.167.66:44505)
I20260810 04:41:07.432857 24108 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.167.66@KRBTEST.COM'} at 127.22.167.66:49347
WARNING: no policy specified for kudu/127.22.167.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.167.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.167.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786336863388938-23197-0/krb5kdc/kudu_127.22.167.67.keytab.
Entry for principal kudu/127.22.167.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786336863388938-23197-0/krb5kdc/kudu_127.22.167.67.keytab.
I20260810 04:41:07.455070 23197 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskemv_AM/build/release/bin/kudu
/tmp/dist-test-taskemv_AM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786336863388938-23197-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786336863388938-23197-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786336863388938-23197-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786336863388938-23197-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.167.67:0
--local_ip_for_outbound_sockets=127.22.167.67
--webserver_interface=127.22.167.67
--webserver_port=0
--tserver_master_addrs=127.22.167.126:41843
--keytab_file=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786336863388938-23197-0/krb5kdc/kudu_127.22.167.67.keytab
--principal=kudu/127.22.167.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.167.84:46537
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786336863388938-23197-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786336863388938-23197-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786336863388938-23197-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 04:41:07.582763 24643 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 04:41:07.582983 24643 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 04:41:07.583010 24643 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 04:41:07.584278 24643 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 04:41:07.584317 24643 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 04:41:07.584347 24643 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.167.67
I20260810 04:41:07.586151 24643 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.167.84:46537
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786336863388938-23197-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786336863388938-23197-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786336863388938-23197-0/krb5kdc/kudu_127.22.167.67.keytab
--principal=kudu/127.22.167.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.167.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786336863388938-23197-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.22.167.67
--webserver_port=0
--tserver_master_addrs=127.22.167.126:41843
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24643
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.167.67
--log_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786336863388938-23197-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 03:54:01 UTC on ccycloud.quasar-icijmt.root.comops.site
build id 4577009
I20260810 04:41:07.586400 24643 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 04:41:07.586622 24643 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 04:41:07.592993 24654 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 04:41:07.593012 24651 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 04:41:07.593138 24643 server_base.cc:1034] running on GCE node
W20260810 04:41:07.593009 24660 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 04:41:07.593587 24643 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 04:41:07.593845 24643 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 04:41:07.594980 24643 hybrid_clock.cc:648] HybridClock initialized: now 1786336867594965 us; error 35 us; skew 500 ppm
Aug 10 04:41:07 dist-test-slave-bqcl krb5kdc[24032](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786336867, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.167.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 04:41:07.598742 24643 init.cc:377] Logged in from keytab as kudu/127.22.167.67@KRBTEST.COM (short username kudu)
I20260810 04:41:07.599141 24643 webserver.cc:460] Webserver started at http://127.22.167.67:45429/ using document root <none> and password file <none>
I20260810 04:41:07.599327 24643 fs_manager.cc:359] Metadata directory not provided
I20260810 04:41:07.599365 24643 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 04:41:07.599457 24643 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 04:41:07.600519 24643 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786336863388938-23197-0/minicluster-data/ts-2/data/instance:
uuid: "79deea90707b41efad9c9f731457dfa9"
format_stamp: "Formatted at 2026-08-10 04:41:07 on dist-test-slave-bqcl"
server_key: "2adb907ef7d1369efc925fcf6ba1577d"
server_key_iv: "582ff8813670c08611f68bb1c68b3a4f"
server_key_version: "encryptionkey@0"
I20260810 04:41:07.600855 24643 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786336863388938-23197-0/minicluster-data/ts-2/wal/instance:
uuid: "79deea90707b41efad9c9f731457dfa9"
format_stamp: "Formatted at 2026-08-10 04:41:07 on dist-test-slave-bqcl"
server_key: "2adb907ef7d1369efc925fcf6ba1577d"
server_key_iv: "582ff8813670c08611f68bb1c68b3a4f"
server_key_version: "encryptionkey@0"
I20260810 04:41:07.602420 24643 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.002s
I20260810 04:41:07.603149 24669 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 04:41:07.603312 24643 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260810 04:41:07.603397 24643 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786336863388938-23197-0/minicluster-data/ts-2/data,/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786336863388938-23197-0/minicluster-data/ts-2/wal
uuid: "79deea90707b41efad9c9f731457dfa9"
format_stamp: "Formatted at 2026-08-10 04:41:07 on dist-test-slave-bqcl"
server_key: "2adb907ef7d1369efc925fcf6ba1577d"
server_key_iv: "582ff8813670c08611f68bb1c68b3a4f"
server_key_version: "encryptionkey@0"
I20260810 04:41:07.603449 24643 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786336863388938-23197-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786336863388938-23197-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786336863388938-23197-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 04:41:07.622319 24643 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 04:41:07.624493 24643 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 04:41:07.624665 24643 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 04:41:07.625000 24643 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 04:41:07.625327 24643 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 04:41:07.625365 24643 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 04:41:07.625386 24643 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 04:41:07.625402 24643 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 04:41:07.641671 24643 rpc_server.cc:307] RPC server started. Bound to: 127.22.167.67:41323
I20260810 04:41:07.641805 24851 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.167.67:41323 every 8 connection(s)
I20260810 04:41:07.642067 24643 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786336863388938-23197-0/minicluster-data/ts-2/data/info.pb
Aug 10 04:41:07 dist-test-slave-bqcl krb5kdc[24032](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786336867, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.167.67@KRBTEST.COM for kudu/127.22.167.126@KRBTEST.COM
I20260810 04:41:07.651103 23197 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskemv_AM/build/release/bin/kudu as pid 24643
I20260810 04:41:07.651216 23197 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786336863388938-23197-0/minicluster-data/ts-2/wal/instance
I20260810 04:41:07.651387 23197 external_mini_cluster.cc:1428] Setting key 00f1ba54ddfb1cb4d6b875e5418b7d57
I20260810 04:41:07.652828 24177 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:41:07.642808 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.167.67:36575 (local address 127.22.167.126:41843)
0810 04:41:07.642889 (+ 81us) server_negotiation.cc:206] Beginning negotiation
0810 04:41:07.642892 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 04:41:07.643212 (+ 320us) server_negotiation.cc:407] Connection header received
0810 04:41:07.644525 (+ 1313us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:41:07.644526 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:41:07.644537 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:41:07.644579 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0810 04:41:07.645467 (+ 888us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:41:07.645967 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:41:07.647233 (+ 1266us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:41:07.647427 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:41:07.647477 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:41:07.649864 (+ 2387us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 04:41:07.649867 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 04:41:07.649867 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 04:41:07.649878 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0810 04:41:07.651319 (+ 1441us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 04:41:07.651752 (+ 433us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 04:41:07.651753 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 04:41:07.651753 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 04:41:07.651833 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 04:41:07.652124 (+ 291us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 04:41:07.652125 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 04:41:07.652125 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 04:41:07.652280 (+ 155us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 04:41:07.652350 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0810 04:41:07.652637 (+ 287us) server_negotiation.cc:299] Negotiation successful
0810 04:41:07.652728 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260810 04:41:07.653095 24856 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:41:07.642781 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.126:41843 (local address 127.22.167.67:36575)
0810 04:41:07.643067 (+ 286us) negotiation.cc:107] Waiting for socket to connect
0810 04:41:07.643103 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0810 04:41:07.644422 (+ 1319us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 04:41:07.644649 (+ 227us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 04:41:07.644652 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 04:41:07.645039 (+ 387us) client_negotiation.cc:189] Negotiated authn=SASL
0810 04:41:07.645391 (+ 352us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:41:07.645397 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:41:07.646031 (+ 634us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:41:07.646031 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:41:07.647154 (+ 1123us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:41:07.647157 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:41:07.647486 (+ 329us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:41:07.647487 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:41:07.647633 (+ 146us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:41:07.648180 (+ 547us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 04:41:07.648188 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0810 04:41:07.649750 (+ 1562us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 04:41:07.651398 (+ 1648us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 04:41:07.651404 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 04:41:07.651406 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 04:41:07.651657 (+ 251us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 04:41:07.651912 (+ 255us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 04:41:07.651913 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 04:41:07.651913 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 04:41:07.652058 (+ 145us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 04:41:07.652355 (+ 297us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 04:41:07.652359 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 04:41:07.652485 (+ 126us) client_negotiation.cc:769] Sending connection context
0810 04:41:07.652593 (+ 108us) client_negotiation.cc:240] Negotiation successful
0810 04:41:07.652767 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":223,"thread_start_us":136,"threads_started":1}
I20260810 04:41:07.654246 24853 heartbeater.cc:347] Connected to a master server at 127.22.167.126:41843
I20260810 04:41:07.654345 24853 heartbeater.cc:464] Registering TS with master...
I20260810 04:41:07.654524 24853 heartbeater.cc:511] Master 127.22.167.126:41843 requested a full tablet report, sending...
I20260810 04:41:07.654928 24108 ts_manager.cc:194] Registered new tserver with Master: 79deea90707b41efad9c9f731457dfa9 (127.22.167.67:41323)
I20260810 04:41:07.655272 24108 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.167.67@KRBTEST.COM'} at 127.22.167.67:36575
I20260810 04:41:07.662986 23197 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260810 04:41:07.670562 24177 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:41:07.664537 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53452 (local address 127.22.167.126:41843)
0810 04:41:07.664578 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0810 04:41:07.664580 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 04:41:07.664841 (+ 261us) server_negotiation.cc:407] Connection header received
0810 04:41:07.664893 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:41:07.664894 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:41:07.664904 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:41:07.664944 (+ 40us) server_negotiation.cc:226] Negotiated authn=SASL
0810 04:41:07.665574 (+ 630us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:41:07.666035 (+ 461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:41:07.666783 (+ 748us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:41:07.666967 (+ 184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:41:07.667021 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:41:07.668187 (+ 1166us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 04:41:07.668189 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 04:41:07.668189 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 04:41:07.668197 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0810 04:41:07.669434 (+ 1237us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 04:41:07.669768 (+ 334us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 04:41:07.669769 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 04:41:07.669770 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 04:41:07.669823 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 04:41:07.670023 (+ 200us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 04:41:07.670024 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 04:41:07.670025 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 04:41:07.670217 (+ 192us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 04:41:07.670302 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0810 04:41:07.670411 (+ 109us) server_negotiation.cc:299] Negotiation successful
0810 04:41:07.670478 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260810 04:41:07.672746 24108 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53452:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 04:41:07.674597 24108 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 04:41:07.681604 24875 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:41:07.676827 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.66:44505 (local address 127.0.0.1:58506)
0810 04:41:07.677107 (+ 280us) negotiation.cc:107] Waiting for socket to connect
0810 04:41:07.677141 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0810 04:41:07.677198 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 04:41:07.677961 (+ 763us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 04:41:07.677965 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 04:41:07.677967 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 04:41:07.678043 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:41:07.678045 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:41:07.679158 (+ 1113us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:41:07.679159 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:41:07.680239 (+ 1080us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:41:07.680242 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:41:07.681273 (+ 1031us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:41:07.681274 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:41:07.681393 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:41:07.681407 (+ 14us) client_negotiation.cc:769] Sending connection context
0810 04:41:07.681468 (+ 61us) client_negotiation.cc:240] Negotiation successful
0810 04:41:07.681532 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":184,"thread_start_us":50,"threads_started":1}
I20260810 04:41:07.681648 24880 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:41:07.677272 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58506 (local address 127.22.167.66:44505)
0810 04:41:07.677561 (+ 289us) server_negotiation.cc:206] Beginning negotiation
0810 04:41:07.677565 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 04:41:07.677574 (+ 9us) server_negotiation.cc:407] Connection header received
0810 04:41:07.677631 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:41:07.677633 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:41:07.677649 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:41:07.677730 (+ 81us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 04:41:07.678149 (+ 419us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:41:07.678978 (+ 829us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:41:07.680335 (+ 1357us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:41:07.681202 (+ 867us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:41:07.681263 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:41:07.681324 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0810 04:41:07.681498 (+ 174us) server_negotiation.cc:299] Negotiation successful
0810 04:41:07.681538 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":236,"thread_start_us":136,"threads_started":1}
I20260810 04:41:07.681908 24874 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:41:07.676829 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.65:39677 (local address 127.0.0.1:39940)
0810 04:41:07.677261 (+ 432us) negotiation.cc:107] Waiting for socket to connect
0810 04:41:07.677271 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0810 04:41:07.677312 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 04:41:07.678065 (+ 753us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 04:41:07.678065 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 04:41:07.678067 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 04:41:07.678099 (+ 32us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:41:07.678100 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:41:07.679739 (+ 1639us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:41:07.679740 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:41:07.680730 (+ 990us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:41:07.680732 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:41:07.681638 (+ 906us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:41:07.681639 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:41:07.681718 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:41:07.681724 (+ 6us) client_negotiation.cc:769] Sending connection context
0810 04:41:07.681757 (+ 33us) client_negotiation.cc:240] Negotiation successful
0810 04:41:07.681797 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":414,"mutex_wait_us":5,"thread_start_us":198,"threads_started":1}
I20260810 04:41:07.682379 24883 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:41:07.677555 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39940 (local address 127.22.167.65:39677)
0810 04:41:07.677820 (+ 265us) server_negotiation.cc:206] Beginning negotiation
0810 04:41:07.677825 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 04:41:07.677836 (+ 11us) server_negotiation.cc:407] Connection header received
0810 04:41:07.677906 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:41:07.677908 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:41:07.677926 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:41:07.678057 (+ 131us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 04:41:07.678268 (+ 211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:41:07.679537 (+ 1269us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:41:07.680806 (+ 1269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:41:07.681530 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:41:07.682097 (+ 567us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:41:07.682196 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0810 04:41:07.682223 (+ 27us) server_negotiation.cc:299] Negotiation successful
0810 04:41:07.682264 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":209,"thread_start_us":105,"threads_started":1}
I20260810 04:41:07.682475 24877 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:41:07.676997 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40410 (local address 127.22.167.67:41323)
0810 04:41:07.677925 (+ 928us) server_negotiation.cc:206] Beginning negotiation
0810 04:41:07.677929 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 04:41:07.677938 (+ 9us) server_negotiation.cc:407] Connection header received
0810 04:41:07.678004 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:41:07.678011 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:41:07.678042 (+ 31us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:41:07.678116 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 04:41:07.678542 (+ 426us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:41:07.679698 (+ 1156us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:41:07.681072 (+ 1374us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:41:07.682074 (+ 1002us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:41:07.682186 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:41:07.682239 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0810 04:41:07.682329 (+ 90us) server_negotiation.cc:299] Negotiation successful
0810 04:41:07.682370 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":869,"thread_start_us":158,"threads_started":1}
I20260810 04:41:07.682415 24876 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:41:07.676827 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.67:41323 (local address 127.0.0.1:40410)
0810 04:41:07.677079 (+ 252us) negotiation.cc:107] Waiting for socket to connect
0810 04:41:07.677124 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0810 04:41:07.677191 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 04:41:07.678171 (+ 980us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 04:41:07.678172 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 04:41:07.678174 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 04:41:07.678262 (+ 88us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:41:07.678264 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:41:07.679878 (+ 1614us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:41:07.679879 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:41:07.680984 (+ 1105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:41:07.680988 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:41:07.682184 (+ 1196us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:41:07.682185 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:41:07.682246 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:41:07.682250 (+ 4us) client_negotiation.cc:769] Sending connection context
0810 04:41:07.682311 (+ 61us) client_negotiation.cc:240] Negotiation successful
0810 04:41:07.682348 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":219,"mutex_wait_us":2,"thread_start_us":159,"threads_started":1}
I20260810 04:41:07.682757 24524 tablet_service.cc:1467] Processing CreateTablet for tablet 724ea50d5ea94092a602acce12174cd1 (DEFAULT_TABLE table=test-table [id=9d2c9f0b3a6f45168ede4c47be59832b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 04:41:07.683157 24524 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 724ea50d5ea94092a602acce12174cd1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 04:41:07.683596 24750 tablet_service.cc:1467] Processing CreateTablet for tablet 724ea50d5ea94092a602acce12174cd1 (DEFAULT_TABLE table=test-table [id=9d2c9f0b3a6f45168ede4c47be59832b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 04:41:07.683933 24750 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 724ea50d5ea94092a602acce12174cd1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 04:41:07.686559 24309 tablet_service.cc:1467] Processing CreateTablet for tablet 724ea50d5ea94092a602acce12174cd1 (DEFAULT_TABLE table=test-table [id=9d2c9f0b3a6f45168ede4c47be59832b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 04:41:07.686848 24309 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 724ea50d5ea94092a602acce12174cd1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 04:41:07.688369 24885 tablet_bootstrap.cc:492] T 724ea50d5ea94092a602acce12174cd1 P 8ede0cae26214b1bb27df91d3571f281: Bootstrap starting.
I20260810 04:41:07.689143 24887 tablet_bootstrap.cc:492] T 724ea50d5ea94092a602acce12174cd1 P 79deea90707b41efad9c9f731457dfa9: Bootstrap starting.
I20260810 04:41:07.690593 24885 tablet_bootstrap.cc:654] T 724ea50d5ea94092a602acce12174cd1 P 8ede0cae26214b1bb27df91d3571f281: Neither blocks nor log segments found. Creating new log.
I20260810 04:41:07.690609 24887 tablet_bootstrap.cc:654] T 724ea50d5ea94092a602acce12174cd1 P 79deea90707b41efad9c9f731457dfa9: Neither blocks nor log segments found. Creating new log.
I20260810 04:41:07.691598 24888 tablet_bootstrap.cc:492] T 724ea50d5ea94092a602acce12174cd1 P 075bc7ddc396438f8df780c21cf9fc80: Bootstrap starting.
I20260810 04:41:07.691810 24887 log.cc:824] T 724ea50d5ea94092a602acce12174cd1 P 79deea90707b41efad9c9f731457dfa9: Log is configured to *not* fsync() on all Append() calls
I20260810 04:41:07.692118 24885 log.cc:824] T 724ea50d5ea94092a602acce12174cd1 P 8ede0cae26214b1bb27df91d3571f281: Log is configured to *not* fsync() on all Append() calls
I20260810 04:41:07.692911 24887 tablet_bootstrap.cc:492] T 724ea50d5ea94092a602acce12174cd1 P 79deea90707b41efad9c9f731457dfa9: No bootstrap required, opened a new log
I20260810 04:41:07.692978 24887 ts_tablet_manager.cc:1397] T 724ea50d5ea94092a602acce12174cd1 P 79deea90707b41efad9c9f731457dfa9: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.003s
I20260810 04:41:07.693121 24885 tablet_bootstrap.cc:492] T 724ea50d5ea94092a602acce12174cd1 P 8ede0cae26214b1bb27df91d3571f281: No bootstrap required, opened a new log
I20260810 04:41:07.693178 24888 tablet_bootstrap.cc:654] T 724ea50d5ea94092a602acce12174cd1 P 075bc7ddc396438f8df780c21cf9fc80: Neither blocks nor log segments found. Creating new log.
I20260810 04:41:07.693555 24885 ts_tablet_manager.cc:1397] T 724ea50d5ea94092a602acce12174cd1 P 8ede0cae26214b1bb27df91d3571f281: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.004s
I20260810 04:41:07.694924 24888 log.cc:824] T 724ea50d5ea94092a602acce12174cd1 P 075bc7ddc396438f8df780c21cf9fc80: Log is configured to *not* fsync() on all Append() calls
I20260810 04:41:07.695940 24888 tablet_bootstrap.cc:492] T 724ea50d5ea94092a602acce12174cd1 P 075bc7ddc396438f8df780c21cf9fc80: No bootstrap required, opened a new log
I20260810 04:41:07.696017 24888 ts_tablet_manager.cc:1397] T 724ea50d5ea94092a602acce12174cd1 P 075bc7ddc396438f8df780c21cf9fc80: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.003s
I20260810 04:41:07.697060 24887 raft_consensus.cc:348] T 724ea50d5ea94092a602acce12174cd1 P 79deea90707b41efad9c9f731457dfa9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79deea90707b41efad9c9f731457dfa9" member_type: VOTER last_known_addr { host: "127.22.167.67" port: 41323 } } peers { permanent_uuid: "8ede0cae26214b1bb27df91d3571f281" member_type: VOTER last_known_addr { host: "127.22.167.66" port: 44505 } } peers { permanent_uuid: "075bc7ddc396438f8df780c21cf9fc80" member_type: VOTER last_known_addr { host: "127.22.167.65" port: 39677 } }
I20260810 04:41:07.697245 24887 raft_consensus.cc:374] T 724ea50d5ea94092a602acce12174cd1 P 79deea90707b41efad9c9f731457dfa9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 04:41:07.697288 24887 raft_consensus.cc:729] T 724ea50d5ea94092a602acce12174cd1 P 79deea90707b41efad9c9f731457dfa9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 79deea90707b41efad9c9f731457dfa9, State: Initialized, Role: FOLLOWER
I20260810 04:41:07.697424 24887 consensus_queue.cc:260] T 724ea50d5ea94092a602acce12174cd1 P 79deea90707b41efad9c9f731457dfa9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79deea90707b41efad9c9f731457dfa9" member_type: VOTER last_known_addr { host: "127.22.167.67" port: 41323 } } peers { permanent_uuid: "8ede0cae26214b1bb27df91d3571f281" member_type: VOTER last_known_addr { host: "127.22.167.66" port: 44505 } } peers { permanent_uuid: "075bc7ddc396438f8df780c21cf9fc80" member_type: VOTER last_known_addr { host: "127.22.167.65" port: 39677 } }
I20260810 04:41:07.697906 24853 heartbeater.cc:503] Master 127.22.167.126:41843 was elected leader, sending a full tablet report...
I20260810 04:41:07.697932 24887 ts_tablet_manager.cc:1428] T 724ea50d5ea94092a602acce12174cd1 P 79deea90707b41efad9c9f731457dfa9: Time spent starting tablet: real 0.005s user 0.000s sys 0.005s
I20260810 04:41:07.698159 24885 raft_consensus.cc:348] T 724ea50d5ea94092a602acce12174cd1 P 8ede0cae26214b1bb27df91d3571f281 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79deea90707b41efad9c9f731457dfa9" member_type: VOTER last_known_addr { host: "127.22.167.67" port: 41323 } } peers { permanent_uuid: "8ede0cae26214b1bb27df91d3571f281" member_type: VOTER last_known_addr { host: "127.22.167.66" port: 44505 } } peers { permanent_uuid: "075bc7ddc396438f8df780c21cf9fc80" member_type: VOTER last_known_addr { host: "127.22.167.65" port: 39677 } }
I20260810 04:41:07.698438 24885 raft_consensus.cc:374] T 724ea50d5ea94092a602acce12174cd1 P 8ede0cae26214b1bb27df91d3571f281 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 04:41:07.698482 24885 raft_consensus.cc:729] T 724ea50d5ea94092a602acce12174cd1 P 8ede0cae26214b1bb27df91d3571f281 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8ede0cae26214b1bb27df91d3571f281, State: Initialized, Role: FOLLOWER
I20260810 04:41:07.698592 24885 consensus_queue.cc:260] T 724ea50d5ea94092a602acce12174cd1 P 8ede0cae26214b1bb27df91d3571f281 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79deea90707b41efad9c9f731457dfa9" member_type: VOTER last_known_addr { host: "127.22.167.67" port: 41323 } } peers { permanent_uuid: "8ede0cae26214b1bb27df91d3571f281" member_type: VOTER last_known_addr { host: "127.22.167.66" port: 44505 } } peers { permanent_uuid: "075bc7ddc396438f8df780c21cf9fc80" member_type: VOTER last_known_addr { host: "127.22.167.65" port: 39677 } }
I20260810 04:41:07.699179 24635 heartbeater.cc:503] Master 127.22.167.126:41843 was elected leader, sending a full tablet report...
I20260810 04:41:07.699205 24885 ts_tablet_manager.cc:1428] T 724ea50d5ea94092a602acce12174cd1 P 8ede0cae26214b1bb27df91d3571f281: Time spent starting tablet: real 0.006s user 0.004s sys 0.001s
I20260810 04:41:07.700287 24888 raft_consensus.cc:348] T 724ea50d5ea94092a602acce12174cd1 P 075bc7ddc396438f8df780c21cf9fc80 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79deea90707b41efad9c9f731457dfa9" member_type: VOTER last_known_addr { host: "127.22.167.67" port: 41323 } } peers { permanent_uuid: "8ede0cae26214b1bb27df91d3571f281" member_type: VOTER last_known_addr { host: "127.22.167.66" port: 44505 } } peers { permanent_uuid: "075bc7ddc396438f8df780c21cf9fc80" member_type: VOTER last_known_addr { host: "127.22.167.65" port: 39677 } }
I20260810 04:41:07.700462 24888 raft_consensus.cc:374] T 724ea50d5ea94092a602acce12174cd1 P 075bc7ddc396438f8df780c21cf9fc80 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 04:41:07.700505 24888 raft_consensus.cc:729] T 724ea50d5ea94092a602acce12174cd1 P 075bc7ddc396438f8df780c21cf9fc80 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 075bc7ddc396438f8df780c21cf9fc80, State: Initialized, Role: FOLLOWER
I20260810 04:41:07.700618 24888 consensus_queue.cc:260] T 724ea50d5ea94092a602acce12174cd1 P 075bc7ddc396438f8df780c21cf9fc80 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79deea90707b41efad9c9f731457dfa9" member_type: VOTER last_known_addr { host: "127.22.167.67" port: 41323 } } peers { permanent_uuid: "8ede0cae26214b1bb27df91d3571f281" member_type: VOTER last_known_addr { host: "127.22.167.66" port: 44505 } } peers { permanent_uuid: "075bc7ddc396438f8df780c21cf9fc80" member_type: VOTER last_known_addr { host: "127.22.167.65" port: 39677 } }
I20260810 04:41:07.700991 24888 ts_tablet_manager.cc:1428] T 724ea50d5ea94092a602acce12174cd1 P 075bc7ddc396438f8df780c21cf9fc80: Time spent starting tablet: real 0.005s user 0.004s sys 0.001s
I20260810 04:41:07.701045 24413 heartbeater.cc:503] Master 127.22.167.126:41843 was elected leader, sending a full tablet report...
W20260810 04:41:07.768858 24414 tablet.cc:2369] T 724ea50d5ea94092a602acce12174cd1 P 075bc7ddc396438f8df780c21cf9fc80: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260810 04:41:07.892895 24854 tablet.cc:2369] T 724ea50d5ea94092a602acce12174cd1 P 79deea90707b41efad9c9f731457dfa9: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260810 04:41:07.921588 24638 tablet.cc:2369] T 724ea50d5ea94092a602acce12174cd1 P 8ede0cae26214b1bb27df91d3571f281: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 04:41:07.958277 24896 raft_consensus.cc:482] T 724ea50d5ea94092a602acce12174cd1 P 79deea90707b41efad9c9f731457dfa9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 04:41:07.958452 24896 raft_consensus.cc:504] T 724ea50d5ea94092a602acce12174cd1 P 79deea90707b41efad9c9f731457dfa9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79deea90707b41efad9c9f731457dfa9" member_type: VOTER last_known_addr { host: "127.22.167.67" port: 41323 } } peers { permanent_uuid: "8ede0cae26214b1bb27df91d3571f281" member_type: VOTER last_known_addr { host: "127.22.167.66" port: 44505 } } peers { permanent_uuid: "075bc7ddc396438f8df780c21cf9fc80" member_type: VOTER last_known_addr { host: "127.22.167.65" port: 39677 } }
I20260810 04:41:07.959026 24896 leader_election.cc:288] T 724ea50d5ea94092a602acce12174cd1 P 79deea90707b41efad9c9f731457dfa9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8ede0cae26214b1bb27df91d3571f281 (127.22.167.66:44505), 075bc7ddc396438f8df780c21cf9fc80 (127.22.167.65:39677)
I20260810 04:41:07.962855 24856 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:41:07.959299 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.66:44505 (local address 127.22.167.67:52719)
0810 04:41:07.959350 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0810 04:41:07.959361 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0810 04:41:07.959407 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 04:41:07.959631 (+ 224us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 04:41:07.959632 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 04:41:07.959635 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 04:41:07.959843 (+ 208us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:41:07.959846 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:41:07.960616 (+ 770us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:41:07.960617 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:41:07.961824 (+ 1207us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:41:07.961829 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:41:07.962543 (+ 714us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:41:07.962544 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:41:07.962681 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:41:07.962685 (+ 4us) client_negotiation.cc:769] Sending connection context
0810 04:41:07.962741 (+ 56us) client_negotiation.cc:240] Negotiation successful
0810 04:41:07.962780 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":35}
I20260810 04:41:07.962875 24880 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:41:07.959328 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.167.67:52719 (local address 127.22.167.66:44505)
0810 04:41:07.959375 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0810 04:41:07.959377 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 04:41:07.959391 (+ 14us) server_negotiation.cc:407] Connection header received
0810 04:41:07.959474 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:41:07.959476 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:41:07.959486 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:41:07.959581 (+ 95us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 04:41:07.959935 (+ 354us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:41:07.960539 (+ 604us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:41:07.961933 (+ 1394us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:41:07.962438 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:41:07.962527 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:41:07.962537 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0810 04:41:07.962761 (+ 224us) server_negotiation.cc:299] Negotiation successful
0810 04:41:07.962805 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260810 04:41:07.963244 24564 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "724ea50d5ea94092a602acce12174cd1" candidate_uuid: "79deea90707b41efad9c9f731457dfa9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8ede0cae26214b1bb27df91d3571f281" is_pre_election: true
I20260810 04:41:07.963433 24564 raft_consensus.cc:2454] T 724ea50d5ea94092a602acce12174cd1 P 8ede0cae26214b1bb27df91d3571f281 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 79deea90707b41efad9c9f731457dfa9 in term 0.
I20260810 04:41:07.963596 24883 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:41:07.959294 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.167.67:49669 (local address 127.22.167.65:39677)
0810 04:41:07.959340 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0810 04:41:07.959342 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 04:41:07.959831 (+ 489us) server_negotiation.cc:407] Connection header received
0810 04:41:07.959875 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:41:07.959876 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:41:07.959887 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:41:07.959948 (+ 61us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 04:41:07.960270 (+ 322us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:41:07.961053 (+ 783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:41:07.962512 (+ 1459us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:41:07.963035 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:41:07.963101 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:41:07.963110 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0810 04:41:07.963502 (+ 392us) server_negotiation.cc:299] Negotiation successful
0810 04:41:07.963545 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260810 04:41:07.963876 24899 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:41:07.959315 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.65:39677 (local address 127.22.167.67:49669)
0810 04:41:07.959732 (+ 417us) negotiation.cc:107] Waiting for socket to connect
0810 04:41:07.959743 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0810 04:41:07.959824 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 04:41:07.959958 (+ 134us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 04:41:07.959959 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 04:41:07.959961 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 04:41:07.960181 (+ 220us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:41:07.960184 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:41:07.961154 (+ 970us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:41:07.961155 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:41:07.962433 (+ 1278us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:41:07.962436 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:41:07.963250 (+ 814us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:41:07.963251 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:41:07.963373 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:41:07.963379 (+ 6us) client_negotiation.cc:769] Sending connection context
0810 04:41:07.963730 (+ 351us) client_negotiation.cc:240] Negotiation successful
0810 04:41:07.963772 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":357,"thread_start_us":258,"threads_started":1}
I20260810 04:41:07.963999 24679 leader_election.cc:302] T 724ea50d5ea94092a602acce12174cd1 P 79deea90707b41efad9c9f731457dfa9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 79deea90707b41efad9c9f731457dfa9, 8ede0cae26214b1bb27df91d3571f281; no voters:
I20260810 04:41:07.964152 24896 raft_consensus.cc:2781] T 724ea50d5ea94092a602acce12174cd1 P 79deea90707b41efad9c9f731457dfa9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 04:41:07.964236 24896 raft_consensus.cc:482] T 724ea50d5ea94092a602acce12174cd1 P 79deea90707b41efad9c9f731457dfa9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 04:41:07.964269 24896 raft_consensus.cc:3037] T 724ea50d5ea94092a602acce12174cd1 P 79deea90707b41efad9c9f731457dfa9 [term 0 FOLLOWER]: Advancing to term 1
I20260810 04:41:07.964453 24341 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "724ea50d5ea94092a602acce12174cd1" candidate_uuid: "79deea90707b41efad9c9f731457dfa9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "075bc7ddc396438f8df780c21cf9fc80" is_pre_election: true
I20260810 04:41:07.964675 24341 raft_consensus.cc:2454] T 724ea50d5ea94092a602acce12174cd1 P 075bc7ddc396438f8df780c21cf9fc80 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 79deea90707b41efad9c9f731457dfa9 in term 0.
I20260810 04:41:07.965376 24896 raft_consensus.cc:504] T 724ea50d5ea94092a602acce12174cd1 P 79deea90707b41efad9c9f731457dfa9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79deea90707b41efad9c9f731457dfa9" member_type: VOTER last_known_addr { host: "127.22.167.67" port: 41323 } } peers { permanent_uuid: "8ede0cae26214b1bb27df91d3571f281" member_type: VOTER last_known_addr { host: "127.22.167.66" port: 44505 } } peers { permanent_uuid: "075bc7ddc396438f8df780c21cf9fc80" member_type: VOTER last_known_addr { host: "127.22.167.65" port: 39677 } }
I20260810 04:41:07.965498 24896 leader_election.cc:288] T 724ea50d5ea94092a602acce12174cd1 P 79deea90707b41efad9c9f731457dfa9 [CANDIDATE]: Term 1 election: Requested vote from peers 8ede0cae26214b1bb27df91d3571f281 (127.22.167.66:44505), 075bc7ddc396438f8df780c21cf9fc80 (127.22.167.65:39677)
I20260810 04:41:07.965801 24564 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "724ea50d5ea94092a602acce12174cd1" candidate_uuid: "79deea90707b41efad9c9f731457dfa9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8ede0cae26214b1bb27df91d3571f281"
I20260810 04:41:07.965894 24564 raft_consensus.cc:3037] T 724ea50d5ea94092a602acce12174cd1 P 8ede0cae26214b1bb27df91d3571f281 [term 0 FOLLOWER]: Advancing to term 1
I20260810 04:41:07.967325 24564 raft_consensus.cc:2454] T 724ea50d5ea94092a602acce12174cd1 P 8ede0cae26214b1bb27df91d3571f281 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 79deea90707b41efad9c9f731457dfa9 in term 1.
I20260810 04:41:07.967463 24341 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "724ea50d5ea94092a602acce12174cd1" candidate_uuid: "79deea90707b41efad9c9f731457dfa9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "075bc7ddc396438f8df780c21cf9fc80"
I20260810 04:41:07.967554 24341 raft_consensus.cc:3037] T 724ea50d5ea94092a602acce12174cd1 P 075bc7ddc396438f8df780c21cf9fc80 [term 0 FOLLOWER]: Advancing to term 1
I20260810 04:41:07.968950 24341 raft_consensus.cc:2454] T 724ea50d5ea94092a602acce12174cd1 P 075bc7ddc396438f8df780c21cf9fc80 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 79deea90707b41efad9c9f731457dfa9 in term 1.
I20260810 04:41:07.969126 24679 leader_election.cc:302] T 724ea50d5ea94092a602acce12174cd1 P 79deea90707b41efad9c9f731457dfa9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 79deea90707b41efad9c9f731457dfa9, 8ede0cae26214b1bb27df91d3571f281; no voters:
I20260810 04:41:07.969223 24896 raft_consensus.cc:2781] T 724ea50d5ea94092a602acce12174cd1 P 79deea90707b41efad9c9f731457dfa9 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 04:41:07.969610 24896 raft_consensus.cc:686] T 724ea50d5ea94092a602acce12174cd1 P 79deea90707b41efad9c9f731457dfa9 [term 1 LEADER]: Becoming Leader. State: Replica: 79deea90707b41efad9c9f731457dfa9, State: Running, Role: LEADER
I20260810 04:41:07.969712 24896 consensus_queue.cc:237] T 724ea50d5ea94092a602acce12174cd1 P 79deea90707b41efad9c9f731457dfa9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79deea90707b41efad9c9f731457dfa9" member_type: VOTER last_known_addr { host: "127.22.167.67" port: 41323 } } peers { permanent_uuid: "8ede0cae26214b1bb27df91d3571f281" member_type: VOTER last_known_addr { host: "127.22.167.66" port: 44505 } } peers { permanent_uuid: "075bc7ddc396438f8df780c21cf9fc80" member_type: VOTER last_known_addr { host: "127.22.167.65" port: 39677 } }
I20260810 04:41:07.970640 24107 catalog_manager.cc:5495] T 724ea50d5ea94092a602acce12174cd1 P 79deea90707b41efad9c9f731457dfa9 reported cstate change: term changed from 0 to 1, leader changed from <none> to 79deea90707b41efad9c9f731457dfa9 (127.22.167.67). New cstate: current_term: 1 leader_uuid: "79deea90707b41efad9c9f731457dfa9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79deea90707b41efad9c9f731457dfa9" member_type: VOTER last_known_addr { host: "127.22.167.67" port: 41323 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8ede0cae26214b1bb27df91d3571f281" member_type: VOTER last_known_addr { host: "127.22.167.66" port: 44505 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "075bc7ddc396438f8df780c21cf9fc80" member_type: VOTER last_known_addr { host: "127.22.167.65" port: 39677 } health_report { overall_health: UNKNOWN } } }
Aug 10 04:41:08 dist-test-slave-bqcl krb5kdc[24032](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786336866, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.167.65@KRBTEST.COM
I20260810 04:41:08.036597 24883 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:41:08.029227 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39954 (local address 127.22.167.65:39677)
0810 04:41:08.029277 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0810 04:41:08.029279 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 04:41:08.029631 (+ 352us) server_negotiation.cc:407] Connection header received
0810 04:41:08.029677 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:41:08.029678 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:41:08.029689 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:41:08.029739 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0810 04:41:08.030426 (+ 687us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:41:08.030904 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:41:08.031568 (+ 664us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:41:08.031741 (+ 173us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:41:08.031783 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:41:08.034099 (+ 2316us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 04:41:08.034112 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 04:41:08.034113 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 04:41:08.034136 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0810 04:41:08.035675 (+ 1539us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 04:41:08.035968 (+ 293us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 04:41:08.035971 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 04:41:08.035971 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 04:41:08.036045 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 04:41:08.036194 (+ 149us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 04:41:08.036194 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 04:41:08.036194 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 04:41:08.036327 (+ 133us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 04:41:08.036427 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0810 04:41:08.036457 (+ 30us) server_negotiation.cc:299] Negotiation successful
0810 04:41:08.036515 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
Aug 10 04:41:08 dist-test-slave-bqcl krb5kdc[24032](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786336868, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 10 04:41:08 dist-test-slave-bqcl krb5kdc[24032](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786336868, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.22.167.65@KRBTEST.COM
I20260810 04:41:08.060483 24883 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:41:08.053437 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39960 (local address 127.22.167.65:39677)
0810 04:41:08.053483 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0810 04:41:08.053485 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 04:41:08.053570 (+ 85us) server_negotiation.cc:407] Connection header received
0810 04:41:08.053619 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:41:08.053620 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:41:08.053628 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:41:08.053694 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0810 04:41:08.054235 (+ 541us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:41:08.054706 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:41:08.055374 (+ 668us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:41:08.055551 (+ 177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:41:08.055597 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:41:08.057567 (+ 1970us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 04:41:08.057568 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 04:41:08.057568 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 04:41:08.057575 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 04:41:08.059165 (+ 1590us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 04:41:08.059551 (+ 386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 04:41:08.059552 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 04:41:08.059552 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 04:41:08.059622 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 04:41:08.059883 (+ 261us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 04:41:08.059884 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 04:41:08.059884 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 04:41:08.060030 (+ 146us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 04:41:08.060135 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0810 04:41:08.060303 (+ 168us) server_negotiation.cc:299] Negotiation successful
0810 04:41:08.060386 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
Aug 10 04:41:08 dist-test-slave-bqcl krb5kdc[24032](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786336868, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 10 04:41:08 dist-test-slave-bqcl krb5kdc[24032](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786336868, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.167.65@KRBTEST.COM
I20260810 04:41:08.083875 24883 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:41:08.077239 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39974 (local address 127.22.167.65:39677)
0810 04:41:08.077304 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0810 04:41:08.077306 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 04:41:08.077366 (+ 60us) server_negotiation.cc:407] Connection header received
0810 04:41:08.077430 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:41:08.077431 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:41:08.077441 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:41:08.077487 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0810 04:41:08.078008 (+ 521us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:41:08.078520 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:41:08.079265 (+ 745us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:41:08.079458 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:41:08.079511 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:41:08.081642 (+ 2131us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 04:41:08.081643 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 04:41:08.081644 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 04:41:08.081651 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 04:41:08.082899 (+ 1248us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 04:41:08.083202 (+ 303us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 04:41:08.083203 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 04:41:08.083203 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 04:41:08.083256 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 04:41:08.083416 (+ 160us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 04:41:08.083416 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 04:41:08.083417 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 04:41:08.083511 (+ 94us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 04:41:08.083599 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0810 04:41:08.083711 (+ 112us) server_negotiation.cc:299] Negotiation successful
0810 04:41:08.083782 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
W20260810 04:41:08.084452 24280 tablet_service.cc:2908] Rejecting scan request for tablet 724ea50d5ea94092a602acce12174cd1: Uninitialized: clean time has not yet been initialized
I20260810 04:41:08.085245 23197 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskemv_AM/build/release/bin/kudu with pid 24210
I20260810 04:41:08.096845 23197 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskemv_AM/build/release/bin/kudu with pid 24422
I20260810 04:41:08.108111 23197 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskemv_AM/build/release/bin/kudu with pid 24643
I20260810 04:41:08.116405 23197 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskemv_AM/build/release/bin/kudu with pid 24052
2026-08-10T04:41:08Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnChecksum (2479 ms)
[ RUN ] SecurityITest.SmokeTestAsAuthorizedUser
Loading random data
Initializing database '/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786336863388938-23197-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 04:41:08 dist-test-slave-bqcl krb5kdc[24924](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 04:41:08 dist-test-slave-bqcl krb5kdc[24924](info): set up 2 sockets
Aug 10 04:41:08 dist-test-slave-bqcl krb5kdc[24924](info): commencing operation
krb5kdc: starting...
W20260810 04:41:09.289081 23197 mini_kdc.cc:121] Time spent starting KDC: real 1.147s user 0.005s sys 0.000s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786336863388938-23197-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786336863388938-23197-0/krb5kdc/test-user.keytab.
Aug 10 04:41:09 dist-test-slave-bqcl krb5kdc[24924](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786336869, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T04:41:09Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T04:41:09Z Disabled control of system clock
WARNING: no policy specified for kudu/127.22.167.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.167.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.167.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786336863388938-23197-0/krb5kdc/kudu_127.22.167.126.keytab.
Entry for principal kudu/127.22.167.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786336863388938-23197-0/krb5kdc/kudu_127.22.167.126.keytab.
I20260810 04:41:09.416821 23197 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskemv_AM/build/release/bin/kudu
/tmp/dist-test-taskemv_AM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786336863388938-23197-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786336863388938-23197-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786336863388938-23197-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786336863388938-23197-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.167.126:37145
--webserver_interface=127.22.167.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786336863388938-23197-0/krb5kdc/kudu_127.22.167.126.keytab
--principal=kudu/127.22.167.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.167.84:37045
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.167.126:37145
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786336863388938-23197-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786336863388938-23197-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786336863388938-23197-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 04:41:09.497090 24946 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 04:41:09.497283 24946 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 04:41:09.497304 24946 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 04:41:09.498395 24946 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 04:41:09.498428 24946 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260810 04:41:09.498440 24946 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 04:41:09.498451 24946 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 04:41:09.498462 24946 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 04:41:09.498478 24946 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 04:41:09.499876 24946 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.167.84:37045
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786336863388938-23197-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786336863388938-23197-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.22.167.126:37145
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786336863388938-23197-0/krb5kdc/kudu_127.22.167.126.keytab
--principal=kudu/127.22.167.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.167.126:37145
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786336863388938-23197-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.167.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24946
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786336863388938-23197-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 03:54:01 UTC on ccycloud.quasar-icijmt.root.comops.site
build id 4577009
I20260810 04:41:09.500058 24946 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 04:41:09.500229 24946 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 04:41:09.504770 24956 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 04:41:09.504739 24960 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 04:41:09.504801 24957 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 04:41:09.504944 24946 server_base.cc:1034] running on GCE node
I20260810 04:41:09.505276 24946 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 04:41:09.505535 24946 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 04:41:09.506666 24946 hybrid_clock.cc:648] HybridClock initialized: now 1786336869506650 us; error 34 us; skew 500 ppm
Aug 10 04:41:09 dist-test-slave-bqcl krb5kdc[24924](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786336869, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.167.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 04:41:09.509003 24946 init.cc:377] Logged in from keytab as kudu/127.22.167.126@KRBTEST.COM (short username kudu)
I20260810 04:41:09.509387 24946 webserver.cc:460] Webserver started at http://127.22.167.126:41705/ using document root <none> and password file <none>
I20260810 04:41:09.509578 24946 fs_manager.cc:359] Metadata directory not provided
I20260810 04:41:09.509617 24946 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 04:41:09.509716 24946 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 04:41:09.510710 24946 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786336863388938-23197-0/minicluster-data/master-0/data/instance:
uuid: "09efc1c3f6d6489eb08c9e5e3287d110"
format_stamp: "Formatted at 2026-08-10 04:41:09 on dist-test-slave-bqcl"
server_key: "7404e03793df63de75f1595710daf8a0"
server_key_iv: "fc98ae37eca758c750851f544f80e631"
server_key_version: "encryptionkey@0"
I20260810 04:41:09.510991 24946 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786336863388938-23197-0/minicluster-data/master-0/wal/instance:
uuid: "09efc1c3f6d6489eb08c9e5e3287d110"
format_stamp: "Formatted at 2026-08-10 04:41:09 on dist-test-slave-bqcl"
server_key: "7404e03793df63de75f1595710daf8a0"
server_key_iv: "fc98ae37eca758c750851f544f80e631"
server_key_version: "encryptionkey@0"
I20260810 04:41:09.512128 24946 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260810 04:41:09.512750 24969 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 04:41:09.512890 24946 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260810 04:41:09.512971 24946 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786336863388938-23197-0/minicluster-data/master-0/data,/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786336863388938-23197-0/minicluster-data/master-0/wal
uuid: "09efc1c3f6d6489eb08c9e5e3287d110"
format_stamp: "Formatted at 2026-08-10 04:41:09 on dist-test-slave-bqcl"
server_key: "7404e03793df63de75f1595710daf8a0"
server_key_iv: "fc98ae37eca758c750851f544f80e631"
server_key_version: "encryptionkey@0"
I20260810 04:41:09.513024 24946 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786336863388938-23197-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786336863388938-23197-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786336863388938-23197-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 04:41:09.532228 24946 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 04:41:09.533978 24946 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 04:41:09.534096 24946 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 04:41:09.539479 24946 rpc_server.cc:307] RPC server started. Bound to: 127.22.167.126:37145
I20260810 04:41:09.539567 25070 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.167.126:37145 every 8 connection(s)
I20260810 04:41:09.539851 24946 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786336863388938-23197-0/minicluster-data/master-0/data/info.pb
I20260810 04:41:09.540501 25072 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 04:41:09.541389 23197 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskemv_AM/build/release/bin/kudu as pid 24946
I20260810 04:41:09.541473 23197 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786336863388938-23197-0/minicluster-data/master-0/wal/instance
I20260810 04:41:09.541632 23197 external_mini_cluster.cc:1428] Setting key 5e2eca1db9f549f45fdb737d3af0d28a
I20260810 04:41:09.543077 25072 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 09efc1c3f6d6489eb08c9e5e3287d110: Bootstrap starting.
I20260810 04:41:09.543838 25072 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 09efc1c3f6d6489eb08c9e5e3287d110: Neither blocks nor log segments found. Creating new log.
I20260810 04:41:09.544119 25072 log.cc:824] T 00000000000000000000000000000000 P 09efc1c3f6d6489eb08c9e5e3287d110: Log is configured to *not* fsync() on all Append() calls
I20260810 04:41:09.544785 25072 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 09efc1c3f6d6489eb08c9e5e3287d110: No bootstrap required, opened a new log
I20260810 04:41:09.547478 25072 raft_consensus.cc:348] T 00000000000000000000000000000000 P 09efc1c3f6d6489eb08c9e5e3287d110 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "09efc1c3f6d6489eb08c9e5e3287d110" member_type: VOTER last_known_addr { host: "127.22.167.126" port: 37145 } }
I20260810 04:41:09.547672 25072 raft_consensus.cc:374] T 00000000000000000000000000000000 P 09efc1c3f6d6489eb08c9e5e3287d110 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 04:41:09.547725 25072 raft_consensus.cc:729] T 00000000000000000000000000000000 P 09efc1c3f6d6489eb08c9e5e3287d110 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 09efc1c3f6d6489eb08c9e5e3287d110, State: Initialized, Role: FOLLOWER
I20260810 04:41:09.547869 25072 consensus_queue.cc:260] T 00000000000000000000000000000000 P 09efc1c3f6d6489eb08c9e5e3287d110 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "09efc1c3f6d6489eb08c9e5e3287d110" member_type: VOTER last_known_addr { host: "127.22.167.126" port: 37145 } }
I20260810 04:41:09.547956 25072 raft_consensus.cc:388] T 00000000000000000000000000000000 P 09efc1c3f6d6489eb08c9e5e3287d110 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 04:41:09.547979 25072 raft_consensus.cc:482] T 00000000000000000000000000000000 P 09efc1c3f6d6489eb08c9e5e3287d110 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 04:41:09.548026 25072 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 09efc1c3f6d6489eb08c9e5e3287d110 [term 0 FOLLOWER]: Advancing to term 1
Aug 10 04:41:09 dist-test-slave-bqcl krb5kdc[24924](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786336869, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.167.126@KRBTEST.COM
I20260810 04:41:09.548588 25072 raft_consensus.cc:504] T 00000000000000000000000000000000 P 09efc1c3f6d6489eb08c9e5e3287d110 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "09efc1c3f6d6489eb08c9e5e3287d110" member_type: VOTER last_known_addr { host: "127.22.167.126" port: 37145 } }
I20260810 04:41:09.548686 25072 leader_election.cc:302] T 00000000000000000000000000000000 P 09efc1c3f6d6489eb08c9e5e3287d110 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 09efc1c3f6d6489eb08c9e5e3287d110; no voters:
I20260810 04:41:09.548856 25072 leader_election.cc:288] T 00000000000000000000000000000000 P 09efc1c3f6d6489eb08c9e5e3287d110 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 04:41:09.548882 25082 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 09efc1c3f6d6489eb08c9e5e3287d110 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 04:41:09.549028 25082 raft_consensus.cc:686] T 00000000000000000000000000000000 P 09efc1c3f6d6489eb08c9e5e3287d110 [term 1 LEADER]: Becoming Leader. State: Replica: 09efc1c3f6d6489eb08c9e5e3287d110, State: Running, Role: LEADER
I20260810 04:41:09.549122 25072 sys_catalog.cc:564] T 00000000000000000000000000000000 P 09efc1c3f6d6489eb08c9e5e3287d110 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 04:41:09.549109 25082 consensus_queue.cc:237] T 00000000000000000000000000000000 P 09efc1c3f6d6489eb08c9e5e3287d110 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "09efc1c3f6d6489eb08c9e5e3287d110" member_type: VOTER last_known_addr { host: "127.22.167.126" port: 37145 } }
I20260810 04:41:09.549446 25082 sys_catalog.cc:455] T 00000000000000000000000000000000 P 09efc1c3f6d6489eb08c9e5e3287d110 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 09efc1c3f6d6489eb08c9e5e3287d110. Latest consensus state: current_term: 1 leader_uuid: "09efc1c3f6d6489eb08c9e5e3287d110" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "09efc1c3f6d6489eb08c9e5e3287d110" member_type: VOTER last_known_addr { host: "127.22.167.126" port: 37145 } } }
I20260810 04:41:09.549530 25082 sys_catalog.cc:458] T 00000000000000000000000000000000 P 09efc1c3f6d6489eb08c9e5e3287d110 [sys.catalog]: This master's current role is: LEADER
I20260810 04:41:09.549461 25084 sys_catalog.cc:455] T 00000000000000000000000000000000 P 09efc1c3f6d6489eb08c9e5e3287d110 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "09efc1c3f6d6489eb08c9e5e3287d110" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "09efc1c3f6d6489eb08c9e5e3287d110" member_type: VOTER last_known_addr { host: "127.22.167.126" port: 37145 } } }
I20260810 04:41:09.549743 25084 sys_catalog.cc:458] T 00000000000000000000000000000000 P 09efc1c3f6d6489eb08c9e5e3287d110 [sys.catalog]: This master's current role is: LEADER
I20260810 04:41:09.549866 25089 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 04:41:09.550570 25089 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 04:41:09.552444 25089 catalog_manager.cc:1347] Generated new cluster ID: c2ee8c8af8e14b478a8baaef40e00b61
I20260810 04:41:09.552500 25089 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 04:41:09.553179 25076 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:41:09.542112 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53208 (local address 127.22.167.126:37145)
0810 04:41:09.542298 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0810 04:41:09.542307 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 04:41:09.542332 (+ 25us) server_negotiation.cc:407] Connection header received
0810 04:41:09.542876 (+ 544us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:41:09.542883 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:41:09.542931 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:41:09.543044 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0810 04:41:09.543801 (+ 757us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:41:09.544787 (+ 986us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:41:09.545818 (+ 1031us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:41:09.546262 (+ 444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:41:09.546349 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:41:09.548845 (+ 2496us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 04:41:09.548849 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 04:41:09.548850 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 04:41:09.548862 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0810 04:41:09.551350 (+ 2488us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 04:41:09.551887 (+ 537us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 04:41:09.551890 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 04:41:09.551891 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 04:41:09.552017 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 04:41:09.552228 (+ 211us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 04:41:09.552229 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 04:41:09.552229 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 04:41:09.552525 (+ 296us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 04:41:09.552651 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0810 04:41:09.552866 (+ 215us) server_negotiation.cc:299] Negotiation successful
0810 04:41:09.553001 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":103,"thread_start_us":66,"threads_started":1}
I20260810 04:41:09.569443 25089 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 04:41:09.570222 25089 catalog_manager.cc:1504] Loading token signing keys...
I20260810 04:41:09.578383 25089 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 09efc1c3f6d6489eb08c9e5e3287d110: Generated new TSK 0
I20260810 04:41:09.578569 25089 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.22.167.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.167.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.167.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786336863388938-23197-0/krb5kdc/kudu_127.22.167.65.keytab.
Entry for principal kudu/127.22.167.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786336863388938-23197-0/krb5kdc/kudu_127.22.167.65.keytab.
I20260810 04:41:09.607102 23197 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskemv_AM/build/release/bin/kudu
/tmp/dist-test-taskemv_AM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786336863388938-23197-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786336863388938-23197-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786336863388938-23197-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786336863388938-23197-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.167.65:0
--local_ip_for_outbound_sockets=127.22.167.65
--webserver_interface=127.22.167.65
--webserver_port=0
--tserver_master_addrs=127.22.167.126:37145
--keytab_file=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786336863388938-23197-0/krb5kdc/kudu_127.22.167.65.keytab
--principal=kudu/127.22.167.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.167.84:37045
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786336863388938-23197-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786336863388938-23197-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786336863388938-23197-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 04:41:09.702630 25113 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 04:41:09.702894 25113 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 04:41:09.702929 25113 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 04:41:09.704614 25113 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 04:41:09.704664 25113 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 04:41:09.704692 25113 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260810 04:41:09.704723 25113 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.167.65
I20260810 04:41:09.706815 25113 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.167.84:37045
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786336863388938-23197-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786336863388938-23197-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786336863388938-23197-0/krb5kdc/kudu_127.22.167.65.keytab
--principal=kudu/127.22.167.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.167.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786336863388938-23197-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.22.167.65
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.22.167.126:37145
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25113
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.167.65
--log_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786336863388938-23197-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 03:54:01 UTC on ccycloud.quasar-icijmt.root.comops.site
build id 4577009
I20260810 04:41:09.707047 25113 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 04:41:09.707283 25113 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 04:41:09.716888 25123 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 04:41:09.716936 25125 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 04:41:09.717216 25131 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 04:41:09.718200 25113 server_base.cc:1034] running on GCE node
I20260810 04:41:09.718438 25113 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 04:41:09.718719 25113 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 04:41:09.719879 25113 hybrid_clock.cc:648] HybridClock initialized: now 1786336869719861 us; error 32 us; skew 500 ppm
Aug 10 04:41:09 dist-test-slave-bqcl krb5kdc[24924](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786336869, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.167.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 04:41:09.722527 25113 init.cc:377] Logged in from keytab as kudu/127.22.167.65@KRBTEST.COM (short username kudu)
I20260810 04:41:09.723016 25113 webserver.cc:460] Webserver started at http://127.22.167.65:35795/ using document root <none> and password file <none>
I20260810 04:41:09.723259 25113 fs_manager.cc:359] Metadata directory not provided
I20260810 04:41:09.723320 25113 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 04:41:09.723449 25113 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 04:41:09.724885 25113 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786336863388938-23197-0/minicluster-data/ts-0/data/instance:
uuid: "c7293e88c49744ccacd6f1cb5a1923b9"
format_stamp: "Formatted at 2026-08-10 04:41:09 on dist-test-slave-bqcl"
server_key: "be2eb39c7fef393593d3131cbb4fe93c"
server_key_iv: "d24499d12fee364e8ab0498a37b0a80b"
server_key_version: "encryptionkey@0"
I20260810 04:41:09.725287 25113 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786336863388938-23197-0/minicluster-data/ts-0/wal/instance:
uuid: "c7293e88c49744ccacd6f1cb5a1923b9"
format_stamp: "Formatted at 2026-08-10 04:41:09 on dist-test-slave-bqcl"
server_key: "be2eb39c7fef393593d3131cbb4fe93c"
server_key_iv: "d24499d12fee364e8ab0498a37b0a80b"
server_key_version: "encryptionkey@0"
I20260810 04:41:09.728183 25113 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.000s sys 0.004s
I20260810 04:41:09.729171 25140 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 04:41:09.729351 25113 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 04:41:09.729453 25113 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786336863388938-23197-0/minicluster-data/ts-0/data,/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786336863388938-23197-0/minicluster-data/ts-0/wal
uuid: "c7293e88c49744ccacd6f1cb5a1923b9"
format_stamp: "Formatted at 2026-08-10 04:41:09 on dist-test-slave-bqcl"
server_key: "be2eb39c7fef393593d3131cbb4fe93c"
server_key_iv: "d24499d12fee364e8ab0498a37b0a80b"
server_key_version: "encryptionkey@0"
I20260810 04:41:09.729540 25113 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786336863388938-23197-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786336863388938-23197-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786336863388938-23197-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 04:41:09.742333 25113 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 04:41:09.744803 25113 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 04:41:09.744937 25113 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 04:41:09.745491 25113 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 04:41:09.745533 25113 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 04:41:09.745575 25113 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 04:41:09.745595 25113 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 10 04:41:09 dist-test-slave-bqcl krb5kdc[24924](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786336869, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.167.65@KRBTEST.COM for kudu/127.22.167.126@KRBTEST.COM
I20260810 04:41:09.754489 25113 rpc_server.cc:307] RPC server started. Bound to: 127.22.167.65:41643
I20260810 04:41:09.754583 25333 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.167.65:41643 every 8 connection(s)
I20260810 04:41:09.754894 25113 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786336863388938-23197-0/minicluster-data/ts-0/data/info.pb
I20260810 04:41:09.757413 25076 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:41:09.746324 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.167.65:45619 (local address 127.22.167.126:37145)
0810 04:41:09.746365 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0810 04:41:09.746367 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 04:41:09.746376 (+ 9us) server_negotiation.cc:407] Connection header received
0810 04:41:09.746898 (+ 522us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:41:09.746899 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:41:09.746912 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:41:09.746965 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0810 04:41:09.747742 (+ 777us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:41:09.748497 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:41:09.750097 (+ 1600us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:41:09.750425 (+ 328us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:41:09.750489 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:41:09.754051 (+ 3562us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 04:41:09.754052 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 04:41:09.754053 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 04:41:09.754062 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0810 04:41:09.756067 (+ 2005us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 04:41:09.756475 (+ 408us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 04:41:09.756477 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 04:41:09.756477 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 04:41:09.756548 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 04:41:09.756769 (+ 221us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 04:41:09.756770 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 04:41:09.756771 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 04:41:09.756890 (+ 119us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 04:41:09.756955 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0810 04:41:09.757223 (+ 268us) server_negotiation.cc:299] Negotiation successful
0810 04:41:09.757317 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260810 04:41:09.757573 25157 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:41:09.745972 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.126:37145 (local address 127.22.167.65:45619)
0810 04:41:09.746150 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0810 04:41:09.746194 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0810 04:41:09.746797 (+ 603us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 04:41:09.747039 (+ 242us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 04:41:09.747043 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 04:41:09.747366 (+ 323us) client_negotiation.cc:189] Negotiated authn=SASL
0810 04:41:09.747630 (+ 264us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:41:09.747637 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:41:09.748600 (+ 963us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:41:09.748601 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:41:09.749993 (+ 1392us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:41:09.749997 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:41:09.750501 (+ 504us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:41:09.750502 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:41:09.750643 (+ 141us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:41:09.751675 (+ 1032us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 04:41:09.751686 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0810 04:41:09.753968 (+ 2282us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 04:41:09.756150 (+ 2182us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 04:41:09.756158 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 04:41:09.756160 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 04:41:09.756407 (+ 247us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 04:41:09.756603 (+ 196us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 04:41:09.756604 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 04:41:09.756604 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 04:41:09.756713 (+ 109us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 04:41:09.756946 (+ 233us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 04:41:09.756953 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 04:41:09.757100 (+ 147us) client_negotiation.cc:769] Sending connection context
0810 04:41:09.757192 (+ 92us) client_negotiation.cc:240] Negotiation successful
0810 04:41:09.757353 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":79,"threads_started":1}
I20260810 04:41:09.758177 25334 heartbeater.cc:347] Connected to a master server at 127.22.167.126:37145
I20260810 04:41:09.758270 25334 heartbeater.cc:464] Registering TS with master...
I20260810 04:41:09.758457 25334 heartbeater.cc:511] Master 127.22.167.126:37145 requested a full tablet report, sending...
I20260810 04:41:09.759004 24994 ts_manager.cc:194] Registered new tserver with Master: c7293e88c49744ccacd6f1cb5a1923b9 (127.22.167.65:41643)
I20260810 04:41:09.759784 24994 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.167.65@KRBTEST.COM'} at 127.22.167.65:45619
I20260810 04:41:09.762908 23197 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskemv_AM/build/release/bin/kudu as pid 25113
I20260810 04:41:09.763022 23197 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786336863388938-23197-0/minicluster-data/ts-0/wal/instance
I20260810 04:41:09.763195 23197 external_mini_cluster.cc:1428] Setting key 940499b655c5131fb9f939369165c316
I20260810 04:41:09.764473 25076 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:41:09.759287 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.167.65:42041 (local address 127.22.167.126:37145)
0810 04:41:09.759324 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0810 04:41:09.759326 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 04:41:09.759419 (+ 93us) server_negotiation.cc:407] Connection header received
0810 04:41:09.759472 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:41:09.759473 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:41:09.759483 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:41:09.759531 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0810 04:41:09.759813 (+ 282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:41:09.760407 (+ 594us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:41:09.761130 (+ 723us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:41:09.761309 (+ 179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:41:09.761362 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:41:09.762171 (+ 809us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 04:41:09.762173 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 04:41:09.762173 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 04:41:09.762179 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0810 04:41:09.763376 (+ 1197us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 04:41:09.763691 (+ 315us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 04:41:09.763692 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 04:41:09.763693 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 04:41:09.763751 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 04:41:09.764014 (+ 263us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 04:41:09.764015 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 04:41:09.764015 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 04:41:09.764120 (+ 105us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 04:41:09.764183 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0810 04:41:09.764310 (+ 127us) server_negotiation.cc:299] Negotiation successful
0810 04:41:09.764382 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260810 04:41:09.764487 25345 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:41:09.759212 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.126:37145 (local address 127.22.167.65:42041)
0810 04:41:09.759357 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0810 04:41:09.759368 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0810 04:41:09.759421 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 04:41:09.759554 (+ 133us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 04:41:09.759555 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 04:41:09.759705 (+ 150us) client_negotiation.cc:189] Negotiated authn=SASL
0810 04:41:09.759758 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:41:09.759759 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:41:09.760500 (+ 741us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:41:09.760501 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:41:09.761060 (+ 559us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:41:09.761062 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:41:09.761363 (+ 301us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:41:09.761364 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:41:09.761412 (+ 48us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:41:09.761794 (+ 382us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 04:41:09.761798 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0810 04:41:09.762083 (+ 285us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 04:41:09.763455 (+ 1372us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 04:41:09.763456 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 04:41:09.763456 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 04:41:09.763618 (+ 162us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 04:41:09.763900 (+ 282us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 04:41:09.763901 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 04:41:09.763901 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 04:41:09.763955 (+ 54us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 04:41:09.764176 (+ 221us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 04:41:09.764176 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 04:41:09.764221 (+ 45us) client_negotiation.cc:769] Sending connection context
0810 04:41:09.764282 (+ 61us) client_negotiation.cc:240] Negotiation successful
0810 04:41:09.764383 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":100,"mutex_wait_us":7,"thread_start_us":72,"threads_started":1}
WARNING: no policy specified for kudu/127.22.167.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.167.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.167.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786336863388938-23197-0/krb5kdc/kudu_127.22.167.66.keytab.
Entry for principal kudu/127.22.167.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786336863388938-23197-0/krb5kdc/kudu_127.22.167.66.keytab.
I20260810 04:41:09.789422 23197 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskemv_AM/build/release/bin/kudu
/tmp/dist-test-taskemv_AM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786336863388938-23197-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786336863388938-23197-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786336863388938-23197-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786336863388938-23197-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.167.66:0
--local_ip_for_outbound_sockets=127.22.167.66
--webserver_interface=127.22.167.66
--webserver_port=0
--tserver_master_addrs=127.22.167.126:37145
--keytab_file=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786336863388938-23197-0/krb5kdc/kudu_127.22.167.66.keytab
--principal=kudu/127.22.167.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.167.84:37045
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786336863388938-23197-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786336863388938-23197-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786336863388938-23197-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 04:41:09.880296 25349 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 04:41:09.880501 25349 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 04:41:09.880535 25349 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 04:41:09.881645 25349 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 04:41:09.881685 25349 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 04:41:09.881711 25349 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260810 04:41:09.881732 25349 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.167.66
I20260810 04:41:09.883105 25349 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.167.84:37045
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786336863388938-23197-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786336863388938-23197-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786336863388938-23197-0/krb5kdc/kudu_127.22.167.66.keytab
--principal=kudu/127.22.167.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.167.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786336863388938-23197-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.22.167.66
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.22.167.126:37145
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25349
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.167.66
--log_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786336863388938-23197-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 03:54:01 UTC on ccycloud.quasar-icijmt.root.comops.site
build id 4577009
I20260810 04:41:09.883324 25349 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 04:41:09.883526 25349 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 04:41:09.888183 25364 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 04:41:09.888240 25359 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 04:41:09.888260 25349 server_base.cc:1034] running on GCE node
W20260810 04:41:09.888190 25358 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 04:41:09.888562 25349 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 04:41:09.888767 25349 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 04:41:09.889901 25349 hybrid_clock.cc:648] HybridClock initialized: now 1786336869889887 us; error 32 us; skew 500 ppm
Aug 10 04:41:09 dist-test-slave-bqcl krb5kdc[24924](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786336869, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.167.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 04:41:09.894534 25349 init.cc:377] Logged in from keytab as kudu/127.22.167.66@KRBTEST.COM (short username kudu)
I20260810 04:41:09.894953 25349 webserver.cc:460] Webserver started at http://127.22.167.66:45985/ using document root <none> and password file <none>
I20260810 04:41:09.895143 25349 fs_manager.cc:359] Metadata directory not provided
I20260810 04:41:09.895188 25349 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 04:41:09.895313 25349 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 04:41:09.896306 25349 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786336863388938-23197-0/minicluster-data/ts-1/data/instance:
uuid: "ccb6186938e8416eaf9f83ca2f8eb78f"
format_stamp: "Formatted at 2026-08-10 04:41:09 on dist-test-slave-bqcl"
server_key: "d2cc271a040bf4e5c7ee5b475e03e235"
server_key_iv: "43d21ff8b62f2133f7eab0134991e03d"
server_key_version: "encryptionkey@0"
I20260810 04:41:09.896716 25349 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786336863388938-23197-0/minicluster-data/ts-1/wal/instance:
uuid: "ccb6186938e8416eaf9f83ca2f8eb78f"
format_stamp: "Formatted at 2026-08-10 04:41:09 on dist-test-slave-bqcl"
server_key: "d2cc271a040bf4e5c7ee5b475e03e235"
server_key_iv: "43d21ff8b62f2133f7eab0134991e03d"
server_key_version: "encryptionkey@0"
I20260810 04:41:09.897969 25349 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260810 04:41:09.898676 25376 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 04:41:09.898820 25349 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260810 04:41:09.898902 25349 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786336863388938-23197-0/minicluster-data/ts-1/data,/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786336863388938-23197-0/minicluster-data/ts-1/wal
uuid: "ccb6186938e8416eaf9f83ca2f8eb78f"
format_stamp: "Formatted at 2026-08-10 04:41:09 on dist-test-slave-bqcl"
server_key: "d2cc271a040bf4e5c7ee5b475e03e235"
server_key_iv: "43d21ff8b62f2133f7eab0134991e03d"
server_key_version: "encryptionkey@0"
I20260810 04:41:09.898959 25349 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786336863388938-23197-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786336863388938-23197-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786336863388938-23197-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 04:41:09.921895 25349 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 04:41:09.923491 25349 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 04:41:09.923619 25349 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 04:41:09.924396 25349 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 04:41:09.924443 25349 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 04:41:09.924471 25349 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 04:41:09.924495 25349 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 10 04:41:09 dist-test-slave-bqcl krb5kdc[24924](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786336869, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.167.66@KRBTEST.COM for kudu/127.22.167.126@KRBTEST.COM
I20260810 04:41:09.932650 25349 rpc_server.cc:307] RPC server started. Bound to: 127.22.167.66:43099
I20260810 04:41:09.932866 25076 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:41:09.924709 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.167.66:53369 (local address 127.22.167.126:37145)
0810 04:41:09.924756 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0810 04:41:09.924758 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 04:41:09.925030 (+ 272us) server_negotiation.cc:407] Connection header received
0810 04:41:09.925502 (+ 472us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:41:09.925504 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:41:09.925518 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:41:09.925585 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0810 04:41:09.926148 (+ 563us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:41:09.926913 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:41:09.927943 (+ 1030us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:41:09.928180 (+ 237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:41:09.928239 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:41:09.930236 (+ 1997us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 04:41:09.930237 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 04:41:09.930238 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 04:41:09.930246 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0810 04:41:09.931585 (+ 1339us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 04:41:09.931965 (+ 380us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 04:41:09.931967 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 04:41:09.931967 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 04:41:09.932022 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 04:41:09.932241 (+ 219us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 04:41:09.932242 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 04:41:09.932242 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 04:41:09.932354 (+ 112us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 04:41:09.932410 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0810 04:41:09.932695 (+ 285us) server_negotiation.cc:299] Negotiation successful
0810 04:41:09.932776 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260810 04:41:09.932976 25349 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786336863388938-23197-0/minicluster-data/ts-1/data/info.pb
I20260810 04:41:09.933025 25394 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:41:09.924710 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.126:37145 (local address 127.22.167.66:53369)
0810 04:41:09.924912 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0810 04:41:09.924945 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0810 04:41:09.925403 (+ 458us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 04:41:09.925647 (+ 244us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 04:41:09.925650 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 04:41:09.925871 (+ 221us) client_negotiation.cc:189] Negotiated authn=SASL
0810 04:41:09.926044 (+ 173us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:41:09.926049 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:41:09.927004 (+ 955us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:41:09.927004 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:41:09.927853 (+ 849us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:41:09.927856 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:41:09.928253 (+ 397us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:41:09.928253 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:41:09.928366 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:41:09.928831 (+ 465us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 04:41:09.928839 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0810 04:41:09.930147 (+ 1308us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 04:41:09.931656 (+ 1509us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 04:41:09.931671 (+ 15us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 04:41:09.931673 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 04:41:09.931897 (+ 224us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 04:41:09.932077 (+ 180us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 04:41:09.932077 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 04:41:09.932078 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 04:41:09.932184 (+ 106us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 04:41:09.932411 (+ 227us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 04:41:09.932417 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 04:41:09.932567 (+ 150us) client_negotiation.cc:769] Sending connection context
0810 04:41:09.932657 (+ 90us) client_negotiation.cc:240] Negotiation successful
0810 04:41:09.932818 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":167,"thread_start_us":135,"threads_started":1}
I20260810 04:41:09.933427 25588 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.167.66:43099 every 8 connection(s)
I20260810 04:41:09.933809 25589 heartbeater.cc:347] Connected to a master server at 127.22.167.126:37145
I20260810 04:41:09.933881 25589 heartbeater.cc:464] Registering TS with master...
I20260810 04:41:09.934048 25589 heartbeater.cc:511] Master 127.22.167.126:37145 requested a full tablet report, sending...
I20260810 04:41:09.934336 23197 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskemv_AM/build/release/bin/kudu as pid 25349
I20260810 04:41:09.934399 24994 ts_manager.cc:194] Registered new tserver with Master: ccb6186938e8416eaf9f83ca2f8eb78f (127.22.167.66:43099)
I20260810 04:41:09.934437 23197 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786336863388938-23197-0/minicluster-data/ts-1/wal/instance
I20260810 04:41:09.934623 23197 external_mini_cluster.cc:1428] Setting key f8e60d302e21decfedc4716d7429c81f
I20260810 04:41:09.934886 24994 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.167.66@KRBTEST.COM'} at 127.22.167.66:53369
I20260810 04:41:09.940135 25602 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:41:09.934667 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.126:37145 (local address 127.22.167.66:51553)
0810 04:41:09.934818 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0810 04:41:09.934829 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0810 04:41:09.934873 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 04:41:09.934989 (+ 116us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 04:41:09.934990 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 04:41:09.935143 (+ 153us) client_negotiation.cc:189] Negotiated authn=SASL
0810 04:41:09.935208 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:41:09.935209 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:41:09.936022 (+ 813us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:41:09.936023 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:41:09.936690 (+ 667us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:41:09.936693 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:41:09.937012 (+ 319us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:41:09.937013 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:41:09.937093 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:41:09.937444 (+ 351us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 04:41:09.937448 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0810 04:41:09.937712 (+ 264us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 04:41:09.939170 (+ 1458us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 04:41:09.939171 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 04:41:09.939171 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 04:41:09.939331 (+ 160us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 04:41:09.939511 (+ 180us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 04:41:09.939511 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 04:41:09.939511 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 04:41:09.939556 (+ 45us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 04:41:09.939876 (+ 320us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 04:41:09.939876 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 04:41:09.939909 (+ 33us) client_negotiation.cc:769] Sending connection context
0810 04:41:09.939974 (+ 65us) client_negotiation.cc:240] Negotiation successful
0810 04:41:09.940066 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":87,"threads_started":1}
I20260810 04:41:09.940164 25076 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:41:09.934846 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.167.66:51553 (local address 127.22.167.126:37145)
0810 04:41:09.934880 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0810 04:41:09.934882 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 04:41:09.934912 (+ 30us) server_negotiation.cc:407] Connection header received
0810 04:41:09.934930 (+ 18us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:41:09.934931 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:41:09.934940 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:41:09.934982 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0810 04:41:09.935280 (+ 298us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:41:09.935922 (+ 642us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:41:09.936767 (+ 845us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:41:09.936942 (+ 175us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:41:09.936995 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:41:09.937792 (+ 797us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 04:41:09.937793 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 04:41:09.937794 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 04:41:09.937800 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0810 04:41:09.939093 (+ 1293us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 04:41:09.939394 (+ 301us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 04:41:09.939395 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 04:41:09.939395 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 04:41:09.939456 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 04:41:09.939614 (+ 158us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 04:41:09.939615 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 04:41:09.939615 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 04:41:09.939816 (+ 201us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 04:41:09.939889 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0810 04:41:09.940000 (+ 111us) server_negotiation.cc:299] Negotiation successful
0810 04:41:09.940074 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
WARNING: no policy specified for kudu/127.22.167.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.167.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.167.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786336863388938-23197-0/krb5kdc/kudu_127.22.167.67.keytab.
Entry for principal kudu/127.22.167.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786336863388938-23197-0/krb5kdc/kudu_127.22.167.67.keytab.
I20260810 04:41:09.960820 23197 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskemv_AM/build/release/bin/kudu
/tmp/dist-test-taskemv_AM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786336863388938-23197-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786336863388938-23197-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786336863388938-23197-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786336863388938-23197-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.167.67:0
--local_ip_for_outbound_sockets=127.22.167.67
--webserver_interface=127.22.167.67
--webserver_port=0
--tserver_master_addrs=127.22.167.126:37145
--keytab_file=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786336863388938-23197-0/krb5kdc/kudu_127.22.167.67.keytab
--principal=kudu/127.22.167.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.167.84:37045
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786336863388938-23197-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786336863388938-23197-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786336863388938-23197-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 04:41:10.041303 25606 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 04:41:10.041510 25606 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 04:41:10.041543 25606 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 04:41:10.042711 25606 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 04:41:10.042753 25606 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 04:41:10.042780 25606 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260810 04:41:10.042802 25606 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.167.67
I20260810 04:41:10.044245 25606 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.167.84:37045
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786336863388938-23197-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786336863388938-23197-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786336863388938-23197-0/krb5kdc/kudu_127.22.167.67.keytab
--principal=kudu/127.22.167.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.167.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786336863388938-23197-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.22.167.67
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.22.167.126:37145
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25606
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.167.67
--log_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786336863388938-23197-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 03:54:01 UTC on ccycloud.quasar-icijmt.root.comops.site
build id 4577009
I20260810 04:41:10.044564 25606 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 04:41:10.044819 25606 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 04:41:10.050086 25613 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 04:41:10.050141 25614 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 04:41:10.050086 25618 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 04:41:10.050491 25606 server_base.cc:1034] running on GCE node
I20260810 04:41:10.050709 25606 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 04:41:10.050979 25606 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 04:41:10.052124 25606 hybrid_clock.cc:648] HybridClock initialized: now 1786336870052107 us; error 30 us; skew 500 ppm
Aug 10 04:41:10 dist-test-slave-bqcl krb5kdc[24924](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786336870, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.167.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 04:41:10.054679 25606 init.cc:377] Logged in from keytab as kudu/127.22.167.67@KRBTEST.COM (short username kudu)
I20260810 04:41:10.055121 25606 webserver.cc:460] Webserver started at http://127.22.167.67:45443/ using document root <none> and password file <none>
I20260810 04:41:10.055346 25606 fs_manager.cc:359] Metadata directory not provided
I20260810 04:41:10.055395 25606 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 04:41:10.055533 25606 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 04:41:10.056660 25606 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786336863388938-23197-0/minicluster-data/ts-2/data/instance:
uuid: "72ed48ce93664347aabfaf0f2174f058"
format_stamp: "Formatted at 2026-08-10 04:41:10 on dist-test-slave-bqcl"
server_key: "75ac36a55e7ccab9b6e7f2e813c03696"
server_key_iv: "efda7cce5e699e66b3ccf88d71c56882"
server_key_version: "encryptionkey@0"
I20260810 04:41:10.057022 25606 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786336863388938-23197-0/minicluster-data/ts-2/wal/instance:
uuid: "72ed48ce93664347aabfaf0f2174f058"
format_stamp: "Formatted at 2026-08-10 04:41:10 on dist-test-slave-bqcl"
server_key: "75ac36a55e7ccab9b6e7f2e813c03696"
server_key_iv: "efda7cce5e699e66b3ccf88d71c56882"
server_key_version: "encryptionkey@0"
I20260810 04:41:10.058300 25606 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260810 04:41:10.059021 25628 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 04:41:10.059178 25606 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260810 04:41:10.059254 25606 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786336863388938-23197-0/minicluster-data/ts-2/data,/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786336863388938-23197-0/minicluster-data/ts-2/wal
uuid: "72ed48ce93664347aabfaf0f2174f058"
format_stamp: "Formatted at 2026-08-10 04:41:10 on dist-test-slave-bqcl"
server_key: "75ac36a55e7ccab9b6e7f2e813c03696"
server_key_iv: "efda7cce5e699e66b3ccf88d71c56882"
server_key_version: "encryptionkey@0"
I20260810 04:41:10.059322 25606 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786336863388938-23197-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786336863388938-23197-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786336863388938-23197-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 04:41:10.066957 25606 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 04:41:10.068934 25606 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 04:41:10.069047 25606 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 04:41:10.069509 25606 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 04:41:10.069555 25606 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 04:41:10.069578 25606 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 04:41:10.069589 25606 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 10 04:41:10 dist-test-slave-bqcl krb5kdc[24924](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786336870, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.167.67@KRBTEST.COM for kudu/127.22.167.126@KRBTEST.COM
I20260810 04:41:10.076200 25606 rpc_server.cc:307] RPC server started. Bound to: 127.22.167.67:40365
I20260810 04:41:10.076295 25823 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.167.67:40365 every 8 connection(s)
I20260810 04:41:10.076543 25606 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786336863388938-23197-0/minicluster-data/ts-2/data/info.pb
I20260810 04:41:10.079075 25076 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:41:10.069987 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.167.67:50545 (local address 127.22.167.126:37145)
0810 04:41:10.070029 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0810 04:41:10.070032 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 04:41:10.070305 (+ 273us) server_negotiation.cc:407] Connection header received
0810 04:41:10.070814 (+ 509us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:41:10.070816 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:41:10.070831 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:41:10.070902 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0810 04:41:10.071513 (+ 611us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:41:10.072020 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:41:10.073009 (+ 989us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:41:10.073240 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:41:10.073299 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:41:10.076387 (+ 3088us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 04:41:10.076388 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 04:41:10.076388 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 04:41:10.076397 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0810 04:41:10.077751 (+ 1354us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 04:41:10.078232 (+ 481us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 04:41:10.078234 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 04:41:10.078234 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 04:41:10.078298 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 04:41:10.078530 (+ 232us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 04:41:10.078531 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 04:41:10.078531 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 04:41:10.078653 (+ 122us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 04:41:10.078704 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0810 04:41:10.078932 (+ 228us) server_negotiation.cc:299] Negotiation successful
0810 04:41:10.079005 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260810 04:41:10.079152 25650 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:41:10.069997 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.126:37145 (local address 127.22.167.67:50545)
0810 04:41:10.070186 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0810 04:41:10.070228 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0810 04:41:10.070698 (+ 470us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 04:41:10.070980 (+ 282us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 04:41:10.070984 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 04:41:10.071236 (+ 252us) client_negotiation.cc:189] Negotiated authn=SASL
0810 04:41:10.071419 (+ 183us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:41:10.071423 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:41:10.072103 (+ 680us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:41:10.072104 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:41:10.072927 (+ 823us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:41:10.072929 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:41:10.073313 (+ 384us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:41:10.073314 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:41:10.073453 (+ 139us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:41:10.073933 (+ 480us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 04:41:10.073943 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0810 04:41:10.076280 (+ 2337us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 04:41:10.077840 (+ 1560us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 04:41:10.077845 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 04:41:10.077847 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 04:41:10.078172 (+ 325us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 04:41:10.078352 (+ 180us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 04:41:10.078352 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 04:41:10.078353 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 04:41:10.078452 (+ 99us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 04:41:10.078699 (+ 247us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 04:41:10.078701 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 04:41:10.078806 (+ 105us) client_negotiation.cc:769] Sending connection context
0810 04:41:10.078877 (+ 71us) client_negotiation.cc:240] Negotiation successful
0810 04:41:10.078991 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":80,"threads_started":1}
I20260810 04:41:10.079604 25824 heartbeater.cc:347] Connected to a master server at 127.22.167.126:37145
I20260810 04:41:10.079680 25824 heartbeater.cc:464] Registering TS with master...
I20260810 04:41:10.079845 25824 heartbeater.cc:511] Master 127.22.167.126:37145 requested a full tablet report, sending...
I20260810 04:41:10.080189 24996 ts_manager.cc:194] Registered new tserver with Master: 72ed48ce93664347aabfaf0f2174f058 (127.22.167.67:40365)
I20260810 04:41:10.080605 24996 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.167.67@KRBTEST.COM'} at 127.22.167.67:50545
I20260810 04:41:10.085345 25836 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:41:10.080607 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.126:37145 (local address 127.22.167.67:54375)
0810 04:41:10.080781 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0810 04:41:10.080790 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0810 04:41:10.080847 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 04:41:10.080959 (+ 112us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 04:41:10.080960 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 04:41:10.081113 (+ 153us) client_negotiation.cc:189] Negotiated authn=SASL
0810 04:41:10.081164 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:41:10.081165 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:41:10.081800 (+ 635us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:41:10.081800 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:41:10.082366 (+ 566us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:41:10.082368 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:41:10.082640 (+ 272us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:41:10.082641 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:41:10.082688 (+ 47us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:41:10.082985 (+ 297us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 04:41:10.082988 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0810 04:41:10.083234 (+ 246us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 04:41:10.084564 (+ 1330us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 04:41:10.084565 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 04:41:10.084565 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 04:41:10.084706 (+ 141us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 04:41:10.084866 (+ 160us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 04:41:10.084867 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 04:41:10.084867 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 04:41:10.084930 (+ 63us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 04:41:10.085109 (+ 179us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 04:41:10.085109 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 04:41:10.085143 (+ 34us) client_negotiation.cc:769] Sending connection context
0810 04:41:10.085192 (+ 49us) client_negotiation.cc:240] Negotiation successful
0810 04:41:10.085274 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":153,"thread_start_us":77,"threads_started":1}
I20260810 04:41:10.085350 23197 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskemv_AM/build/release/bin/kudu as pid 25606
I20260810 04:41:10.085386 25076 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:41:10.080659 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.167.67:54375 (local address 127.22.167.126:37145)
0810 04:41:10.080729 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0810 04:41:10.080731 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 04:41:10.080834 (+ 103us) server_negotiation.cc:407] Connection header received
0810 04:41:10.080898 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:41:10.080899 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:41:10.080909 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:41:10.080955 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0810 04:41:10.081227 (+ 272us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:41:10.081726 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:41:10.082427 (+ 701us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:41:10.082596 (+ 169us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:41:10.082638 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:41:10.083297 (+ 659us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 04:41:10.083298 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 04:41:10.083298 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 04:41:10.083303 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0810 04:41:10.084499 (+ 1196us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 04:41:10.084771 (+ 272us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 04:41:10.084771 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 04:41:10.084772 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 04:41:10.084816 (+ 44us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 04:41:10.084986 (+ 170us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 04:41:10.084986 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 04:41:10.084986 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 04:41:10.085065 (+ 79us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 04:41:10.085106 (+ 41us) server_negotiation.cc:1035] Waiting for connection context
0810 04:41:10.085227 (+ 121us) server_negotiation.cc:299] Negotiation successful
0810 04:41:10.085304 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260810 04:41:10.085427 23197 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786336863388938-23197-0/minicluster-data/ts-2/wal/instance
I20260810 04:41:10.085580 23197 external_mini_cluster.cc:1428] Setting key 5f861c8f7456e0939ccdd8c239ea1cbc
I20260810 04:41:10.086483 23197 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 10 04:41:10 dist-test-slave-bqcl krb5kdc[24924](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786336870, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 10 04:41:10 dist-test-slave-bqcl krb5kdc[24924](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786336870, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.22.167.126@KRBTEST.COM
I20260810 04:41:10.108404 25076 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:41:10.102429 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53220 (local address 127.22.167.126:37145)
0810 04:41:10.102482 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0810 04:41:10.102484 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 04:41:10.102493 (+ 9us) server_negotiation.cc:407] Connection header received
0810 04:41:10.102560 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:41:10.102561 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:41:10.102570 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:41:10.102618 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0810 04:41:10.103131 (+ 513us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:41:10.103602 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:41:10.104301 (+ 699us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:41:10.104483 (+ 182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:41:10.104537 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:41:10.106548 (+ 2011us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 04:41:10.106550 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 04:41:10.106550 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 04:41:10.106557 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 04:41:10.107572 (+ 1015us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 04:41:10.107863 (+ 291us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 04:41:10.107864 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 04:41:10.107864 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 04:41:10.107912 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 04:41:10.108052 (+ 140us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 04:41:10.108053 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 04:41:10.108053 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 04:41:10.108141 (+ 88us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 04:41:10.108215 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0810 04:41:10.108287 (+ 72us) server_negotiation.cc:299] Negotiation successful
0810 04:41:10.108343 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260810 04:41:10.109754 24996 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:53220:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 04:41:10.111315 24996 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 04:41:10.117651 25852 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:41:10.113348 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.66:43099 (local address 127.0.0.1:37082)
0810 04:41:10.113647 (+ 299us) negotiation.cc:107] Waiting for socket to connect
0810 04:41:10.113681 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0810 04:41:10.113736 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 04:41:10.113939 (+ 203us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 04:41:10.113941 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 04:41:10.113943 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 04:41:10.113991 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:41:10.113991 (+ 0us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:41:10.115081 (+ 1090us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:41:10.115082 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:41:10.116415 (+ 1333us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:41:10.116418 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:41:10.117330 (+ 912us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:41:10.117331 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:41:10.117414 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:41:10.117421 (+ 7us) client_negotiation.cc:769] Sending connection context
0810 04:41:10.117492 (+ 71us) client_negotiation.cc:240] Negotiation successful
0810 04:41:10.117566 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":273,"mutex_wait_us":22,"thread_start_us":46,"threads_started":1}
I20260810 04:41:10.117655 25857 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:41:10.113548 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.65:41643 (local address 127.0.0.1:36654)
0810 04:41:10.114248 (+ 700us) negotiation.cc:107] Waiting for socket to connect
0810 04:41:10.114256 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0810 04:41:10.114297 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 04:41:10.114479 (+ 182us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 04:41:10.114480 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 04:41:10.114483 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 04:41:10.114522 (+ 39us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:41:10.114523 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:41:10.115730 (+ 1207us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:41:10.115731 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:41:10.116487 (+ 756us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:41:10.116489 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:41:10.117326 (+ 837us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:41:10.117327 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:41:10.117406 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:41:10.117419 (+ 13us) client_negotiation.cc:769] Sending connection context
0810 04:41:10.117491 (+ 72us) client_negotiation.cc:240] Negotiation successful
0810 04:41:10.117557 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":683,"mutex_wait_us":3,"thread_start_us":57,"threads_started":1}
I20260810 04:41:10.117889 25859 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:41:10.113646 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36654 (local address 127.22.167.65:41643)
0810 04:41:10.113807 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0810 04:41:10.113812 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 04:41:10.114351 (+ 539us) server_negotiation.cc:407] Connection header received
0810 04:41:10.114393 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:41:10.114395 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:41:10.114411 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:41:10.114480 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 04:41:10.114588 (+ 108us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:41:10.115646 (+ 1058us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:41:10.116568 (+ 922us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:41:10.117167 (+ 599us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:41:10.117235 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:41:10.117292 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0810 04:41:10.117613 (+ 321us) server_negotiation.cc:299] Negotiation successful
0810 04:41:10.117776 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":122,"thread_start_us":68,"threads_started":1}
I20260810 04:41:10.118259 25855 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:41:10.113460 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37082 (local address 127.22.167.66:43099)
0810 04:41:10.113805 (+ 345us) server_negotiation.cc:206] Beginning negotiation
0810 04:41:10.113810 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 04:41:10.113819 (+ 9us) server_negotiation.cc:407] Connection header received
0810 04:41:10.113850 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:41:10.113852 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:41:10.113864 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:41:10.113931 (+ 67us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 04:41:10.114060 (+ 129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:41:10.115010 (+ 950us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:41:10.116493 (+ 1483us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:41:10.117183 (+ 690us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:41:10.117245 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:41:10.117296 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0810 04:41:10.117530 (+ 234us) server_negotiation.cc:299] Negotiation successful
0810 04:41:10.117684 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":315,"thread_start_us":89,"threads_started":1}
I20260810 04:41:10.118263 25854 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:41:10.113475 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57222 (local address 127.22.167.67:40365)
0810 04:41:10.114014 (+ 539us) server_negotiation.cc:206] Beginning negotiation
0810 04:41:10.114017 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 04:41:10.114149 (+ 132us) server_negotiation.cc:407] Connection header received
0810 04:41:10.114231 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:41:10.114233 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:41:10.114249 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:41:10.114464 (+ 215us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 04:41:10.114522 (+ 58us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:41:10.115617 (+ 1095us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:41:10.116915 (+ 1298us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:41:10.117659 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:41:10.117883 (+ 224us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:41:10.118013 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0810 04:41:10.118094 (+ 81us) server_negotiation.cc:299] Negotiation successful
0810 04:41:10.118169 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":506,"thread_start_us":62,"threads_started":1}
I20260810 04:41:10.118499 25853 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:41:10.113376 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.67:40365 (local address 127.0.0.1:57222)
0810 04:41:10.114056 (+ 680us) negotiation.cc:107] Waiting for socket to connect
0810 04:41:10.114064 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0810 04:41:10.114158 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 04:41:10.114345 (+ 187us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 04:41:10.114346 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 04:41:10.114348 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 04:41:10.114392 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:41:10.114393 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:41:10.115718 (+ 1325us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:41:10.115719 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:41:10.116833 (+ 1114us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:41:10.116835 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:41:10.117889 (+ 1054us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:41:10.117890 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:41:10.117981 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:41:10.117987 (+ 6us) client_negotiation.cc:769] Sending connection context
0810 04:41:10.118079 (+ 92us) client_negotiation.cc:240] Negotiation successful
0810 04:41:10.118189 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":664,"mutex_wait_us":33,"thread_start_us":77,"threads_started":1}
I20260810 04:41:10.118633 25209 tablet_service.cc:1467] Processing CreateTablet for tablet c24a4e7afbab46c89ddf7df4c59f71ee (DEFAULT_TABLE table=test-table [id=71180f0e027c4c02a0733ee4d8d15547]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 04:41:10.118848 25474 tablet_service.cc:1467] Processing CreateTablet for tablet c24a4e7afbab46c89ddf7df4c59f71ee (DEFAULT_TABLE table=test-table [id=71180f0e027c4c02a0733ee4d8d15547]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 04:41:10.119016 25209 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c24a4e7afbab46c89ddf7df4c59f71ee. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 04:41:10.119108 25474 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c24a4e7afbab46c89ddf7df4c59f71ee. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 04:41:10.119120 25720 tablet_service.cc:1467] Processing CreateTablet for tablet c24a4e7afbab46c89ddf7df4c59f71ee (DEFAULT_TABLE table=test-table [id=71180f0e027c4c02a0733ee4d8d15547]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 04:41:10.119431 25720 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c24a4e7afbab46c89ddf7df4c59f71ee. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 04:41:10.121770 25860 tablet_bootstrap.cc:492] T c24a4e7afbab46c89ddf7df4c59f71ee P ccb6186938e8416eaf9f83ca2f8eb78f: Bootstrap starting.
I20260810 04:41:10.122375 25860 tablet_bootstrap.cc:654] T c24a4e7afbab46c89ddf7df4c59f71ee P ccb6186938e8416eaf9f83ca2f8eb78f: Neither blocks nor log segments found. Creating new log.
I20260810 04:41:10.122503 25861 tablet_bootstrap.cc:492] T c24a4e7afbab46c89ddf7df4c59f71ee P 72ed48ce93664347aabfaf0f2174f058: Bootstrap starting.
I20260810 04:41:10.122735 25860 log.cc:824] T c24a4e7afbab46c89ddf7df4c59f71ee P ccb6186938e8416eaf9f83ca2f8eb78f: Log is configured to *not* fsync() on all Append() calls
I20260810 04:41:10.122807 25864 tablet_bootstrap.cc:492] T c24a4e7afbab46c89ddf7df4c59f71ee P c7293e88c49744ccacd6f1cb5a1923b9: Bootstrap starting.
I20260810 04:41:10.123392 25864 tablet_bootstrap.cc:654] T c24a4e7afbab46c89ddf7df4c59f71ee P c7293e88c49744ccacd6f1cb5a1923b9: Neither blocks nor log segments found. Creating new log.
I20260810 04:41:10.123421 25861 tablet_bootstrap.cc:654] T c24a4e7afbab46c89ddf7df4c59f71ee P 72ed48ce93664347aabfaf0f2174f058: Neither blocks nor log segments found. Creating new log.
I20260810 04:41:10.123536 25860 tablet_bootstrap.cc:492] T c24a4e7afbab46c89ddf7df4c59f71ee P ccb6186938e8416eaf9f83ca2f8eb78f: No bootstrap required, opened a new log
I20260810 04:41:10.123612 25860 ts_tablet_manager.cc:1397] T c24a4e7afbab46c89ddf7df4c59f71ee P ccb6186938e8416eaf9f83ca2f8eb78f: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260810 04:41:10.123746 25864 log.cc:824] T c24a4e7afbab46c89ddf7df4c59f71ee P c7293e88c49744ccacd6f1cb5a1923b9: Log is configured to *not* fsync() on all Append() calls
I20260810 04:41:10.123749 25861 log.cc:824] T c24a4e7afbab46c89ddf7df4c59f71ee P 72ed48ce93664347aabfaf0f2174f058: Log is configured to *not* fsync() on all Append() calls
I20260810 04:41:10.124373 25864 tablet_bootstrap.cc:492] T c24a4e7afbab46c89ddf7df4c59f71ee P c7293e88c49744ccacd6f1cb5a1923b9: No bootstrap required, opened a new log
I20260810 04:41:10.124432 25864 ts_tablet_manager.cc:1397] T c24a4e7afbab46c89ddf7df4c59f71ee P c7293e88c49744ccacd6f1cb5a1923b9: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260810 04:41:10.125182 25860 raft_consensus.cc:348] T c24a4e7afbab46c89ddf7df4c59f71ee P ccb6186938e8416eaf9f83ca2f8eb78f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccb6186938e8416eaf9f83ca2f8eb78f" member_type: VOTER last_known_addr { host: "127.22.167.66" port: 43099 } } peers { permanent_uuid: "c7293e88c49744ccacd6f1cb5a1923b9" member_type: VOTER last_known_addr { host: "127.22.167.65" port: 41643 } } peers { permanent_uuid: "72ed48ce93664347aabfaf0f2174f058" member_type: VOTER last_known_addr { host: "127.22.167.67" port: 40365 } }
I20260810 04:41:10.125320 25860 raft_consensus.cc:374] T c24a4e7afbab46c89ddf7df4c59f71ee P ccb6186938e8416eaf9f83ca2f8eb78f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 04:41:10.125342 25860 raft_consensus.cc:729] T c24a4e7afbab46c89ddf7df4c59f71ee P ccb6186938e8416eaf9f83ca2f8eb78f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ccb6186938e8416eaf9f83ca2f8eb78f, State: Initialized, Role: FOLLOWER
I20260810 04:41:10.125442 25860 consensus_queue.cc:260] T c24a4e7afbab46c89ddf7df4c59f71ee P ccb6186938e8416eaf9f83ca2f8eb78f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccb6186938e8416eaf9f83ca2f8eb78f" member_type: VOTER last_known_addr { host: "127.22.167.66" port: 43099 } } peers { permanent_uuid: "c7293e88c49744ccacd6f1cb5a1923b9" member_type: VOTER last_known_addr { host: "127.22.167.65" port: 41643 } } peers { permanent_uuid: "72ed48ce93664347aabfaf0f2174f058" member_type: VOTER last_known_addr { host: "127.22.167.67" port: 40365 } }
I20260810 04:41:10.125690 25860 ts_tablet_manager.cc:1428] T c24a4e7afbab46c89ddf7df4c59f71ee P ccb6186938e8416eaf9f83ca2f8eb78f: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260810 04:41:10.125738 25589 heartbeater.cc:503] Master 127.22.167.126:37145 was elected leader, sending a full tablet report...
I20260810 04:41:10.126065 25864 raft_consensus.cc:348] T c24a4e7afbab46c89ddf7df4c59f71ee P c7293e88c49744ccacd6f1cb5a1923b9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccb6186938e8416eaf9f83ca2f8eb78f" member_type: VOTER last_known_addr { host: "127.22.167.66" port: 43099 } } peers { permanent_uuid: "c7293e88c49744ccacd6f1cb5a1923b9" member_type: VOTER last_known_addr { host: "127.22.167.65" port: 41643 } } peers { permanent_uuid: "72ed48ce93664347aabfaf0f2174f058" member_type: VOTER last_known_addr { host: "127.22.167.67" port: 40365 } }
I20260810 04:41:10.126260 25864 raft_consensus.cc:374] T c24a4e7afbab46c89ddf7df4c59f71ee P c7293e88c49744ccacd6f1cb5a1923b9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 04:41:10.126305 25864 raft_consensus.cc:729] T c24a4e7afbab46c89ddf7df4c59f71ee P c7293e88c49744ccacd6f1cb5a1923b9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c7293e88c49744ccacd6f1cb5a1923b9, State: Initialized, Role: FOLLOWER
I20260810 04:41:10.126314 25861 tablet_bootstrap.cc:492] T c24a4e7afbab46c89ddf7df4c59f71ee P 72ed48ce93664347aabfaf0f2174f058: No bootstrap required, opened a new log
I20260810 04:41:10.126371 25861 ts_tablet_manager.cc:1397] T c24a4e7afbab46c89ddf7df4c59f71ee P 72ed48ce93664347aabfaf0f2174f058: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260810 04:41:10.126413 25864 consensus_queue.cc:260] T c24a4e7afbab46c89ddf7df4c59f71ee P c7293e88c49744ccacd6f1cb5a1923b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccb6186938e8416eaf9f83ca2f8eb78f" member_type: VOTER last_known_addr { host: "127.22.167.66" port: 43099 } } peers { permanent_uuid: "c7293e88c49744ccacd6f1cb5a1923b9" member_type: VOTER last_known_addr { host: "127.22.167.65" port: 41643 } } peers { permanent_uuid: "72ed48ce93664347aabfaf0f2174f058" member_type: VOTER last_known_addr { host: "127.22.167.67" port: 40365 } }
I20260810 04:41:10.126684 25864 ts_tablet_manager.cc:1428] T c24a4e7afbab46c89ddf7df4c59f71ee P c7293e88c49744ccacd6f1cb5a1923b9: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260810 04:41:10.126796 25334 heartbeater.cc:503] Master 127.22.167.126:37145 was elected leader, sending a full tablet report...
I20260810 04:41:10.127959 25861 raft_consensus.cc:348] T c24a4e7afbab46c89ddf7df4c59f71ee P 72ed48ce93664347aabfaf0f2174f058 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccb6186938e8416eaf9f83ca2f8eb78f" member_type: VOTER last_known_addr { host: "127.22.167.66" port: 43099 } } peers { permanent_uuid: "c7293e88c49744ccacd6f1cb5a1923b9" member_type: VOTER last_known_addr { host: "127.22.167.65" port: 41643 } } peers { permanent_uuid: "72ed48ce93664347aabfaf0f2174f058" member_type: VOTER last_known_addr { host: "127.22.167.67" port: 40365 } }
I20260810 04:41:10.128079 25861 raft_consensus.cc:374] T c24a4e7afbab46c89ddf7df4c59f71ee P 72ed48ce93664347aabfaf0f2174f058 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 04:41:10.128110 25861 raft_consensus.cc:729] T c24a4e7afbab46c89ddf7df4c59f71ee P 72ed48ce93664347aabfaf0f2174f058 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 72ed48ce93664347aabfaf0f2174f058, State: Initialized, Role: FOLLOWER
I20260810 04:41:10.128211 25861 consensus_queue.cc:260] T c24a4e7afbab46c89ddf7df4c59f71ee P 72ed48ce93664347aabfaf0f2174f058 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccb6186938e8416eaf9f83ca2f8eb78f" member_type: VOTER last_known_addr { host: "127.22.167.66" port: 43099 } } peers { permanent_uuid: "c7293e88c49744ccacd6f1cb5a1923b9" member_type: VOTER last_known_addr { host: "127.22.167.65" port: 41643 } } peers { permanent_uuid: "72ed48ce93664347aabfaf0f2174f058" member_type: VOTER last_known_addr { host: "127.22.167.67" port: 40365 } }
I20260810 04:41:10.128412 25861 ts_tablet_manager.cc:1428] T c24a4e7afbab46c89ddf7df4c59f71ee P 72ed48ce93664347aabfaf0f2174f058: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260810 04:41:10.128479 25824 heartbeater.cc:503] Master 127.22.167.126:37145 was elected leader, sending a full tablet report...
W20260810 04:41:10.183596 25590 tablet.cc:2369] T c24a4e7afbab46c89ddf7df4c59f71ee P ccb6186938e8416eaf9f83ca2f8eb78f: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 04:41:10.222930 25870 raft_consensus.cc:482] T c24a4e7afbab46c89ddf7df4c59f71ee P ccb6186938e8416eaf9f83ca2f8eb78f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 04:41:10.223086 25870 raft_consensus.cc:504] T c24a4e7afbab46c89ddf7df4c59f71ee P ccb6186938e8416eaf9f83ca2f8eb78f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccb6186938e8416eaf9f83ca2f8eb78f" member_type: VOTER last_known_addr { host: "127.22.167.66" port: 43099 } } peers { permanent_uuid: "c7293e88c49744ccacd6f1cb5a1923b9" member_type: VOTER last_known_addr { host: "127.22.167.65" port: 41643 } } peers { permanent_uuid: "72ed48ce93664347aabfaf0f2174f058" member_type: VOTER last_known_addr { host: "127.22.167.67" port: 40365 } }
I20260810 04:41:10.223445 25870 leader_election.cc:288] T c24a4e7afbab46c89ddf7df4c59f71ee P ccb6186938e8416eaf9f83ca2f8eb78f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c7293e88c49744ccacd6f1cb5a1923b9 (127.22.167.65:41643), 72ed48ce93664347aabfaf0f2174f058 (127.22.167.67:40365)
I20260810 04:41:10.226176 25394 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:41:10.223634 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.65:41643 (local address 127.22.167.66:35705)
0810 04:41:10.223687 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0810 04:41:10.223698 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0810 04:41:10.223750 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 04:41:10.223933 (+ 183us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 04:41:10.223933 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 04:41:10.223936 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 04:41:10.224012 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:41:10.224015 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:41:10.224624 (+ 609us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:41:10.224625 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:41:10.225367 (+ 742us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:41:10.225369 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:41:10.225889 (+ 520us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:41:10.225890 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:41:10.225962 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:41:10.225965 (+ 3us) client_negotiation.cc:769] Sending connection context
0810 04:41:10.226008 (+ 43us) client_negotiation.cc:240] Negotiation successful
0810 04:41:10.226042 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":31}
I20260810 04:41:10.226163 25859 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:41:10.223765 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.167.66:35705 (local address 127.22.167.65:41643)
0810 04:41:10.223812 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0810 04:41:10.223814 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 04:41:10.223824 (+ 10us) server_negotiation.cc:407] Connection header received
0810 04:41:10.223850 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:41:10.223850 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:41:10.223857 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:41:10.223908 (+ 51us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 04:41:10.224079 (+ 171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:41:10.224539 (+ 460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:41:10.225451 (+ 912us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:41:10.225820 (+ 369us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:41:10.225881 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:41:10.225891 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0810 04:41:10.226019 (+ 128us) server_negotiation.cc:299] Negotiation successful
0810 04:41:10.226060 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260810 04:41:10.226477 25264 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c24a4e7afbab46c89ddf7df4c59f71ee" candidate_uuid: "ccb6186938e8416eaf9f83ca2f8eb78f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c7293e88c49744ccacd6f1cb5a1923b9" is_pre_election: true
I20260810 04:41:10.226634 25264 raft_consensus.cc:2454] T c24a4e7afbab46c89ddf7df4c59f71ee P c7293e88c49744ccacd6f1cb5a1923b9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ccb6186938e8416eaf9f83ca2f8eb78f in term 0.
I20260810 04:41:10.226689 25854 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:41:10.223743 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.167.66:58019 (local address 127.22.167.67:40365)
0810 04:41:10.223802 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0810 04:41:10.223804 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 04:41:10.224116 (+ 312us) server_negotiation.cc:407] Connection header received
0810 04:41:10.224189 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:41:10.224190 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:41:10.224200 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:41:10.224246 (+ 46us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 04:41:10.224379 (+ 133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:41:10.224836 (+ 457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:41:10.225895 (+ 1059us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:41:10.226343 (+ 448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:41:10.226406 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:41:10.226414 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0810 04:41:10.226590 (+ 176us) server_negotiation.cc:299] Negotiation successful
0810 04:41:10.226627 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260810 04:41:10.226706 25875 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:41:10.223647 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.67:40365 (local address 127.22.167.66:58019)
0810 04:41:10.223846 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0810 04:41:10.223853 (+ 7us) client_negotiation.cc:174] Beginning negotiation
0810 04:41:10.224127 (+ 274us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 04:41:10.224256 (+ 129us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 04:41:10.224257 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 04:41:10.224258 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 04:41:10.224325 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:41:10.224326 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:41:10.224909 (+ 583us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:41:10.224910 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:41:10.225812 (+ 902us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:41:10.225814 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:41:10.226458 (+ 644us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:41:10.226459 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:41:10.226518 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:41:10.226521 (+ 3us) client_negotiation.cc:769] Sending connection context
0810 04:41:10.226567 (+ 46us) client_negotiation.cc:240] Negotiation successful
0810 04:41:10.226606 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":187,"mutex_wait_us":1,"thread_start_us":111,"threads_started":1}
I20260810 04:41:10.227078 25377 leader_election.cc:302] T c24a4e7afbab46c89ddf7df4c59f71ee P ccb6186938e8416eaf9f83ca2f8eb78f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c7293e88c49744ccacd6f1cb5a1923b9, ccb6186938e8416eaf9f83ca2f8eb78f; no voters:
I20260810 04:41:10.227222 25870 raft_consensus.cc:2781] T c24a4e7afbab46c89ddf7df4c59f71ee P ccb6186938e8416eaf9f83ca2f8eb78f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 04:41:10.227295 25870 raft_consensus.cc:482] T c24a4e7afbab46c89ddf7df4c59f71ee P ccb6186938e8416eaf9f83ca2f8eb78f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 04:41:10.227326 25870 raft_consensus.cc:3037] T c24a4e7afbab46c89ddf7df4c59f71ee P ccb6186938e8416eaf9f83ca2f8eb78f [term 0 FOLLOWER]: Advancing to term 1
I20260810 04:41:10.227288 25746 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c24a4e7afbab46c89ddf7df4c59f71ee" candidate_uuid: "ccb6186938e8416eaf9f83ca2f8eb78f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "72ed48ce93664347aabfaf0f2174f058" is_pre_election: true
I20260810 04:41:10.227427 25746 raft_consensus.cc:2454] T c24a4e7afbab46c89ddf7df4c59f71ee P 72ed48ce93664347aabfaf0f2174f058 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ccb6186938e8416eaf9f83ca2f8eb78f in term 0.
I20260810 04:41:10.228335 25870 raft_consensus.cc:504] T c24a4e7afbab46c89ddf7df4c59f71ee P ccb6186938e8416eaf9f83ca2f8eb78f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccb6186938e8416eaf9f83ca2f8eb78f" member_type: VOTER last_known_addr { host: "127.22.167.66" port: 43099 } } peers { permanent_uuid: "c7293e88c49744ccacd6f1cb5a1923b9" member_type: VOTER last_known_addr { host: "127.22.167.65" port: 41643 } } peers { permanent_uuid: "72ed48ce93664347aabfaf0f2174f058" member_type: VOTER last_known_addr { host: "127.22.167.67" port: 40365 } }
I20260810 04:41:10.228454 25870 leader_election.cc:288] T c24a4e7afbab46c89ddf7df4c59f71ee P ccb6186938e8416eaf9f83ca2f8eb78f [CANDIDATE]: Term 1 election: Requested vote from peers c7293e88c49744ccacd6f1cb5a1923b9 (127.22.167.65:41643), 72ed48ce93664347aabfaf0f2174f058 (127.22.167.67:40365)
I20260810 04:41:10.228592 25264 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c24a4e7afbab46c89ddf7df4c59f71ee" candidate_uuid: "ccb6186938e8416eaf9f83ca2f8eb78f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c7293e88c49744ccacd6f1cb5a1923b9"
I20260810 04:41:10.228607 25746 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c24a4e7afbab46c89ddf7df4c59f71ee" candidate_uuid: "ccb6186938e8416eaf9f83ca2f8eb78f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "72ed48ce93664347aabfaf0f2174f058"
I20260810 04:41:10.228677 25264 raft_consensus.cc:3037] T c24a4e7afbab46c89ddf7df4c59f71ee P c7293e88c49744ccacd6f1cb5a1923b9 [term 0 FOLLOWER]: Advancing to term 1
I20260810 04:41:10.228689 25746 raft_consensus.cc:3037] T c24a4e7afbab46c89ddf7df4c59f71ee P 72ed48ce93664347aabfaf0f2174f058 [term 0 FOLLOWER]: Advancing to term 1
I20260810 04:41:10.229362 25746 raft_consensus.cc:2454] T c24a4e7afbab46c89ddf7df4c59f71ee P 72ed48ce93664347aabfaf0f2174f058 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ccb6186938e8416eaf9f83ca2f8eb78f in term 1.
I20260810 04:41:10.229414 25264 raft_consensus.cc:2454] T c24a4e7afbab46c89ddf7df4c59f71ee P c7293e88c49744ccacd6f1cb5a1923b9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ccb6186938e8416eaf9f83ca2f8eb78f in term 1.
I20260810 04:41:10.229568 25386 leader_election.cc:302] T c24a4e7afbab46c89ddf7df4c59f71ee P ccb6186938e8416eaf9f83ca2f8eb78f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 72ed48ce93664347aabfaf0f2174f058, ccb6186938e8416eaf9f83ca2f8eb78f; no voters:
I20260810 04:41:10.229692 25870 raft_consensus.cc:2781] T c24a4e7afbab46c89ddf7df4c59f71ee P ccb6186938e8416eaf9f83ca2f8eb78f [term 1 FOLLOWER]: Leader election won for term 1
I20260810 04:41:10.229866 25870 raft_consensus.cc:686] T c24a4e7afbab46c89ddf7df4c59f71ee P ccb6186938e8416eaf9f83ca2f8eb78f [term 1 LEADER]: Becoming Leader. State: Replica: ccb6186938e8416eaf9f83ca2f8eb78f, State: Running, Role: LEADER
I20260810 04:41:10.229951 25870 consensus_queue.cc:237] T c24a4e7afbab46c89ddf7df4c59f71ee P ccb6186938e8416eaf9f83ca2f8eb78f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccb6186938e8416eaf9f83ca2f8eb78f" member_type: VOTER last_known_addr { host: "127.22.167.66" port: 43099 } } peers { permanent_uuid: "c7293e88c49744ccacd6f1cb5a1923b9" member_type: VOTER last_known_addr { host: "127.22.167.65" port: 41643 } } peers { permanent_uuid: "72ed48ce93664347aabfaf0f2174f058" member_type: VOTER last_known_addr { host: "127.22.167.67" port: 40365 } }
I20260810 04:41:10.230732 24996 catalog_manager.cc:5495] T c24a4e7afbab46c89ddf7df4c59f71ee P ccb6186938e8416eaf9f83ca2f8eb78f reported cstate change: term changed from 0 to 1, leader changed from <none> to ccb6186938e8416eaf9f83ca2f8eb78f (127.22.167.66). New cstate: current_term: 1 leader_uuid: "ccb6186938e8416eaf9f83ca2f8eb78f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccb6186938e8416eaf9f83ca2f8eb78f" member_type: VOTER last_known_addr { host: "127.22.167.66" port: 43099 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c7293e88c49744ccacd6f1cb5a1923b9" member_type: VOTER last_known_addr { host: "127.22.167.65" port: 41643 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "72ed48ce93664347aabfaf0f2174f058" member_type: VOTER last_known_addr { host: "127.22.167.67" port: 40365 } health_report { overall_health: UNKNOWN } } }
W20260810 04:41:10.255685 25337 tablet.cc:2369] T c24a4e7afbab46c89ddf7df4c59f71ee P c7293e88c49744ccacd6f1cb5a1923b9: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 04:41:10.261799 25855 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:41:10.259646 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37090 (local address 127.22.167.66:43099)
0810 04:41:10.259688 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0810 04:41:10.259690 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 04:41:10.259716 (+ 26us) server_negotiation.cc:407] Connection header received
0810 04:41:10.259800 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:41:10.259800 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:41:10.259817 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:41:10.259871 (+ 54us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 04:41:10.259980 (+ 109us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:41:10.260444 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:41:10.261133 (+ 689us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:41:10.261302 (+ 169us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:41:10.261343 (+ 41us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:41:10.261490 (+ 147us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 04:41:10.261597 (+ 107us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 04:41:10.261654 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0810 04:41:10.261716 (+ 62us) server_negotiation.cc:299] Negotiation successful
0810 04:41:10.261733 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260810 04:41:10.263576 25264 raft_consensus.cc:1261] T c24a4e7afbab46c89ddf7df4c59f71ee P c7293e88c49744ccacd6f1cb5a1923b9 [term 1 FOLLOWER]: Refusing update from remote peer ccb6186938e8416eaf9f83ca2f8eb78f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 04:41:10.263597 25746 raft_consensus.cc:1261] T c24a4e7afbab46c89ddf7df4c59f71ee P 72ed48ce93664347aabfaf0f2174f058 [term 1 FOLLOWER]: Refusing update from remote peer ccb6186938e8416eaf9f83ca2f8eb78f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 04:41:10.263895 25882 consensus_queue.cc:1035] T c24a4e7afbab46c89ddf7df4c59f71ee P ccb6186938e8416eaf9f83ca2f8eb78f [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7293e88c49744ccacd6f1cb5a1923b9" member_type: VOTER last_known_addr { host: "127.22.167.65" port: 41643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 04:41:10.264041 25870 consensus_queue.cc:1035] T c24a4e7afbab46c89ddf7df4c59f71ee P ccb6186938e8416eaf9f83ca2f8eb78f [LEADER]: Connected to new peer: Peer: permanent_uuid: "72ed48ce93664347aabfaf0f2174f058" member_type: VOTER last_known_addr { host: "127.22.167.67" port: 40365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 04:41:10.266604 25886 mvcc.cc:205] Tried to move back new op lower bound from 7316835820596944896 to 7316835820462505984. Current Snapshot: MvccSnapshot[applied={T|T < 7316835820596944896}]
I20260810 04:41:10.267269 25889 mvcc.cc:205] Tried to move back new op lower bound from 7316835820596944896 to 7316835820462505984. Current Snapshot: MvccSnapshot[applied={T|T < 7316835820596944896}]
I20260810 04:41:10.267370 25887 mvcc.cc:205] Tried to move back new op lower bound from 7316835820596944896 to 7316835820462505984. Current Snapshot: MvccSnapshot[applied={T|T < 7316835820596944896 or (T in {7316835820596944896})}]
I20260810 04:41:10.271198 24994 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:53220:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 04:41:10.271375 24994 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:53220:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 04:41:10.272214 24994 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 09efc1c3f6d6489eb08c9e5e3287d110: Sending DeleteTablet for 3 replicas of tablet c24a4e7afbab46c89ddf7df4c59f71ee
I20260810 04:41:10.272523 25474 tablet_service.cc:1514] Processing DeleteTablet for tablet c24a4e7afbab46c89ddf7df4c59f71ee with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 04:41:10 UTC) from {username='kudu', principal='kudu/127.22.167.126@KRBTEST.COM'} at 127.0.0.1:37082
I20260810 04:41:10.272599 25209 tablet_service.cc:1514] Processing DeleteTablet for tablet c24a4e7afbab46c89ddf7df4c59f71ee with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 04:41:10 UTC) from {username='kudu', principal='kudu/127.22.167.126@KRBTEST.COM'} at 127.0.0.1:36654
I20260810 04:41:10.272725 25720 tablet_service.cc:1514] Processing DeleteTablet for tablet c24a4e7afbab46c89ddf7df4c59f71ee with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 04:41:10 UTC) from {username='kudu', principal='kudu/127.22.167.126@KRBTEST.COM'} at 127.0.0.1:57222
I20260810 04:41:10.272782 25906 tablet_replica.cc:330] T c24a4e7afbab46c89ddf7df4c59f71ee P c7293e88c49744ccacd6f1cb5a1923b9: stopping tablet replica
I20260810 04:41:10.272773 25905 tablet_replica.cc:330] T c24a4e7afbab46c89ddf7df4c59f71ee P ccb6186938e8416eaf9f83ca2f8eb78f: stopping tablet replica
I20260810 04:41:10.272871 25906 raft_consensus.cc:2229] T c24a4e7afbab46c89ddf7df4c59f71ee P c7293e88c49744ccacd6f1cb5a1923b9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 04:41:10.272902 25905 raft_consensus.cc:2229] T c24a4e7afbab46c89ddf7df4c59f71ee P ccb6186938e8416eaf9f83ca2f8eb78f [term 1 LEADER]: Raft consensus shutting down.
I20260810 04:41:10.272984 25906 raft_consensus.cc:2258] T c24a4e7afbab46c89ddf7df4c59f71ee P c7293e88c49744ccacd6f1cb5a1923b9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 04:41:10.273097 25905 raft_consensus.cc:2258] T c24a4e7afbab46c89ddf7df4c59f71ee P ccb6186938e8416eaf9f83ca2f8eb78f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 04:41:10.273259 24994 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:53220:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 04:41:10.273412 24994 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 04:41:10.273494 25906 ts_tablet_manager.cc:1905] T c24a4e7afbab46c89ddf7df4c59f71ee P c7293e88c49744ccacd6f1cb5a1923b9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 04:41:10.273793 25905 ts_tablet_manager.cc:1905] T c24a4e7afbab46c89ddf7df4c59f71ee P ccb6186938e8416eaf9f83ca2f8eb78f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 04:41:10.274935 25720 tablet_service.cc:1467] Processing CreateTablet for tablet 0d6d38199a8d44459fd6e1b1be560162 (DEFAULT_TABLE table=test-table [id=18c845d5d88545e5bd4e976227e42a3a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 04:41:10.274945 25209 tablet_service.cc:1467] Processing CreateTablet for tablet 0d6d38199a8d44459fd6e1b1be560162 (DEFAULT_TABLE table=test-table [id=18c845d5d88545e5bd4e976227e42a3a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 04:41:10.275055 25720 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0d6d38199a8d44459fd6e1b1be560162. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 04:41:10.275068 25209 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0d6d38199a8d44459fd6e1b1be560162. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 04:41:10.275369 25906 ts_tablet_manager.cc:1918] T c24a4e7afbab46c89ddf7df4c59f71ee P c7293e88c49744ccacd6f1cb5a1923b9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 04:41:10.275434 25906 log.cc:1196] T c24a4e7afbab46c89ddf7df4c59f71ee P c7293e88c49744ccacd6f1cb5a1923b9: Deleting WAL directory at /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786336863388938-23197-0/minicluster-data/ts-0/wal/wals/c24a4e7afbab46c89ddf7df4c59f71ee
I20260810 04:41:10.275722 25906 ts_tablet_manager.cc:1939] T c24a4e7afbab46c89ddf7df4c59f71ee P c7293e88c49744ccacd6f1cb5a1923b9: Deleting consensus metadata
I20260810 04:41:10.276111 25907 tablet_replica.cc:330] T c24a4e7afbab46c89ddf7df4c59f71ee P 72ed48ce93664347aabfaf0f2174f058: stopping tablet replica
I20260810 04:41:10.276211 25907 raft_consensus.cc:2229] T c24a4e7afbab46c89ddf7df4c59f71ee P 72ed48ce93664347aabfaf0f2174f058 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 04:41:10.276324 25907 raft_consensus.cc:2258] T c24a4e7afbab46c89ddf7df4c59f71ee P 72ed48ce93664347aabfaf0f2174f058 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 04:41:10.276535 25474 tablet_service.cc:1467] Processing CreateTablet for tablet 0d6d38199a8d44459fd6e1b1be560162 (DEFAULT_TABLE table=test-table [id=18c845d5d88545e5bd4e976227e42a3a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 04:41:10.276573 24971 catalog_manager.cc:4841] TS c7293e88c49744ccacd6f1cb5a1923b9 (127.22.167.65:41643): tablet c24a4e7afbab46c89ddf7df4c59f71ee (table test-table [id=71180f0e027c4c02a0733ee4d8d15547]) successfully deleted
I20260810 04:41:10.276695 25861 tablet_bootstrap.cc:492] T 0d6d38199a8d44459fd6e1b1be560162 P 72ed48ce93664347aabfaf0f2174f058: Bootstrap starting.
I20260810 04:41:10.276722 25864 tablet_bootstrap.cc:492] T 0d6d38199a8d44459fd6e1b1be560162 P c7293e88c49744ccacd6f1cb5a1923b9: Bootstrap starting.
I20260810 04:41:10.277097 25864 tablet_bootstrap.cc:654] T 0d6d38199a8d44459fd6e1b1be560162 P c7293e88c49744ccacd6f1cb5a1923b9: Neither blocks nor log segments found. Creating new log.
I20260810 04:41:10.277383 25907 ts_tablet_manager.cc:1905] T c24a4e7afbab46c89ddf7df4c59f71ee P 72ed48ce93664347aabfaf0f2174f058: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 04:41:10.277449 25474 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0d6d38199a8d44459fd6e1b1be560162. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 04:41:10.278628 25864 tablet_bootstrap.cc:492] T 0d6d38199a8d44459fd6e1b1be560162 P c7293e88c49744ccacd6f1cb5a1923b9: No bootstrap required, opened a new log
I20260810 04:41:10.278683 25864 ts_tablet_manager.cc:1397] T 0d6d38199a8d44459fd6e1b1be560162 P c7293e88c49744ccacd6f1cb5a1923b9: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260810 04:41:10.278780 25860 tablet_bootstrap.cc:492] T 0d6d38199a8d44459fd6e1b1be560162 P ccb6186938e8416eaf9f83ca2f8eb78f: Bootstrap starting.
I20260810 04:41:10.278831 25864 raft_consensus.cc:348] T 0d6d38199a8d44459fd6e1b1be560162 P c7293e88c49744ccacd6f1cb5a1923b9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c7293e88c49744ccacd6f1cb5a1923b9" member_type: VOTER last_known_addr { host: "127.22.167.65" port: 41643 } } peers { permanent_uuid: "ccb6186938e8416eaf9f83ca2f8eb78f" member_type: VOTER last_known_addr { host: "127.22.167.66" port: 43099 } } peers { permanent_uuid: "72ed48ce93664347aabfaf0f2174f058" member_type: VOTER last_known_addr { host: "127.22.167.67" port: 40365 } }
I20260810 04:41:10.278892 25864 raft_consensus.cc:374] T 0d6d38199a8d44459fd6e1b1be560162 P c7293e88c49744ccacd6f1cb5a1923b9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 04:41:10.278913 25864 raft_consensus.cc:729] T 0d6d38199a8d44459fd6e1b1be560162 P c7293e88c49744ccacd6f1cb5a1923b9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c7293e88c49744ccacd6f1cb5a1923b9, State: Initialized, Role: FOLLOWER
I20260810 04:41:10.277397 25861 tablet_bootstrap.cc:654] T 0d6d38199a8d44459fd6e1b1be560162 P 72ed48ce93664347aabfaf0f2174f058: Neither blocks nor log segments found. Creating new log.
I20260810 04:41:10.278956 25864 consensus_queue.cc:260] T 0d6d38199a8d44459fd6e1b1be560162 P c7293e88c49744ccacd6f1cb5a1923b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c7293e88c49744ccacd6f1cb5a1923b9" member_type: VOTER last_known_addr { host: "127.22.167.65" port: 41643 } } peers { permanent_uuid: "ccb6186938e8416eaf9f83ca2f8eb78f" member_type: VOTER last_known_addr { host: "127.22.167.66" port: 43099 } } peers { permanent_uuid: "72ed48ce93664347aabfaf0f2174f058" member_type: VOTER last_known_addr { host: "127.22.167.67" port: 40365 } }
I20260810 04:41:10.279053 25864 ts_tablet_manager.cc:1428] T 0d6d38199a8d44459fd6e1b1be560162 P c7293e88c49744ccacd6f1cb5a1923b9: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260810 04:41:10.279206 25860 tablet_bootstrap.cc:654] T 0d6d38199a8d44459fd6e1b1be560162 P ccb6186938e8416eaf9f83ca2f8eb78f: Neither blocks nor log segments found. Creating new log.
I20260810 04:41:10.279932 25905 ts_tablet_manager.cc:1918] T c24a4e7afbab46c89ddf7df4c59f71ee P ccb6186938e8416eaf9f83ca2f8eb78f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 04:41:10.280011 25905 log.cc:1196] T c24a4e7afbab46c89ddf7df4c59f71ee P ccb6186938e8416eaf9f83ca2f8eb78f: Deleting WAL directory at /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786336863388938-23197-0/minicluster-data/ts-1/wal/wals/c24a4e7afbab46c89ddf7df4c59f71ee
I20260810 04:41:10.280290 25905 ts_tablet_manager.cc:1939] T c24a4e7afbab46c89ddf7df4c59f71ee P ccb6186938e8416eaf9f83ca2f8eb78f: Deleting consensus metadata
I20260810 04:41:10.280359 25860 tablet_bootstrap.cc:492] T 0d6d38199a8d44459fd6e1b1be560162 P ccb6186938e8416eaf9f83ca2f8eb78f: No bootstrap required, opened a new log
I20260810 04:41:10.280406 25860 ts_tablet_manager.cc:1397] T 0d6d38199a8d44459fd6e1b1be560162 P ccb6186938e8416eaf9f83ca2f8eb78f: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260810 04:41:10.280553 25860 raft_consensus.cc:348] T 0d6d38199a8d44459fd6e1b1be560162 P ccb6186938e8416eaf9f83ca2f8eb78f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c7293e88c49744ccacd6f1cb5a1923b9" member_type: VOTER last_known_addr { host: "127.22.167.65" port: 41643 } } peers { permanent_uuid: "ccb6186938e8416eaf9f83ca2f8eb78f" member_type: VOTER last_known_addr { host: "127.22.167.66" port: 43099 } } peers { permanent_uuid: "72ed48ce93664347aabfaf0f2174f058" member_type: VOTER last_known_addr { host: "127.22.167.67" port: 40365 } }
I20260810 04:41:10.280627 25860 raft_consensus.cc:374] T 0d6d38199a8d44459fd6e1b1be560162 P ccb6186938e8416eaf9f83ca2f8eb78f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 04:41:10.280650 25860 raft_consensus.cc:729] T 0d6d38199a8d44459fd6e1b1be560162 P ccb6186938e8416eaf9f83ca2f8eb78f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ccb6186938e8416eaf9f83ca2f8eb78f, State: Initialized, Role: FOLLOWER
I20260810 04:41:10.280711 25860 consensus_queue.cc:260] T 0d6d38199a8d44459fd6e1b1be560162 P ccb6186938e8416eaf9f83ca2f8eb78f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c7293e88c49744ccacd6f1cb5a1923b9" member_type: VOTER last_known_addr { host: "127.22.167.65" port: 41643 } } peers { permanent_uuid: "ccb6186938e8416eaf9f83ca2f8eb78f" member_type: VOTER last_known_addr { host: "127.22.167.66" port: 43099 } } peers { permanent_uuid: "72ed48ce93664347aabfaf0f2174f058" member_type: VOTER last_known_addr { host: "127.22.167.67" port: 40365 } }
I20260810 04:41:10.280726 24971 catalog_manager.cc:4841] TS ccb6186938e8416eaf9f83ca2f8eb78f (127.22.167.66:43099): tablet c24a4e7afbab46c89ddf7df4c59f71ee (table test-table [id=71180f0e027c4c02a0733ee4d8d15547]) successfully deleted
I20260810 04:41:10.280799 25860 ts_tablet_manager.cc:1428] T 0d6d38199a8d44459fd6e1b1be560162 P ccb6186938e8416eaf9f83ca2f8eb78f: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260810 04:41:10.281140 25907 ts_tablet_manager.cc:1918] T c24a4e7afbab46c89ddf7df4c59f71ee P 72ed48ce93664347aabfaf0f2174f058: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 04:41:10.281222 25907 log.cc:1196] T c24a4e7afbab46c89ddf7df4c59f71ee P 72ed48ce93664347aabfaf0f2174f058: Deleting WAL directory at /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786336863388938-23197-0/minicluster-data/ts-2/wal/wals/c24a4e7afbab46c89ddf7df4c59f71ee
I20260810 04:41:10.281507 25907 ts_tablet_manager.cc:1939] T c24a4e7afbab46c89ddf7df4c59f71ee P 72ed48ce93664347aabfaf0f2174f058: Deleting consensus metadata
I20260810 04:41:10.281930 24975 catalog_manager.cc:4841] TS 72ed48ce93664347aabfaf0f2174f058 (127.22.167.67:40365): tablet c24a4e7afbab46c89ddf7df4c59f71ee (table test-table [id=71180f0e027c4c02a0733ee4d8d15547]) successfully deleted
I20260810 04:41:10.282174 25861 tablet_bootstrap.cc:492] T 0d6d38199a8d44459fd6e1b1be560162 P 72ed48ce93664347aabfaf0f2174f058: No bootstrap required, opened a new log
I20260810 04:41:10.282226 25861 ts_tablet_manager.cc:1397] T 0d6d38199a8d44459fd6e1b1be560162 P 72ed48ce93664347aabfaf0f2174f058: Time spent bootstrapping tablet: real 0.006s user 0.001s sys 0.000s
I20260810 04:41:10.282366 25861 raft_consensus.cc:348] T 0d6d38199a8d44459fd6e1b1be560162 P 72ed48ce93664347aabfaf0f2174f058 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c7293e88c49744ccacd6f1cb5a1923b9" member_type: VOTER last_known_addr { host: "127.22.167.65" port: 41643 } } peers { permanent_uuid: "ccb6186938e8416eaf9f83ca2f8eb78f" member_type: VOTER last_known_addr { host: "127.22.167.66" port: 43099 } } peers { permanent_uuid: "72ed48ce93664347aabfaf0f2174f058" member_type: VOTER last_known_addr { host: "127.22.167.67" port: 40365 } }
I20260810 04:41:10.282434 25861 raft_consensus.cc:374] T 0d6d38199a8d44459fd6e1b1be560162 P 72ed48ce93664347aabfaf0f2174f058 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 04:41:10.282450 25861 raft_consensus.cc:729] T 0d6d38199a8d44459fd6e1b1be560162 P 72ed48ce93664347aabfaf0f2174f058 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 72ed48ce93664347aabfaf0f2174f058, State: Initialized, Role: FOLLOWER
I20260810 04:41:10.282500 25861 consensus_queue.cc:260] T 0d6d38199a8d44459fd6e1b1be560162 P 72ed48ce93664347aabfaf0f2174f058 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c7293e88c49744ccacd6f1cb5a1923b9" member_type: VOTER last_known_addr { host: "127.22.167.65" port: 41643 } } peers { permanent_uuid: "ccb6186938e8416eaf9f83ca2f8eb78f" member_type: VOTER last_known_addr { host: "127.22.167.66" port: 43099 } } peers { permanent_uuid: "72ed48ce93664347aabfaf0f2174f058" member_type: VOTER last_known_addr { host: "127.22.167.67" port: 40365 } }
I20260810 04:41:10.282594 25861 ts_tablet_manager.cc:1428] T 0d6d38199a8d44459fd6e1b1be560162 P 72ed48ce93664347aabfaf0f2174f058: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
W20260810 04:41:10.327163 25825 tablet.cc:2369] T 0d6d38199a8d44459fd6e1b1be560162 P 72ed48ce93664347aabfaf0f2174f058: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 04:41:10.451005 25873 raft_consensus.cc:482] T 0d6d38199a8d44459fd6e1b1be560162 P 72ed48ce93664347aabfaf0f2174f058 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 04:41:10.451103 25873 raft_consensus.cc:504] T 0d6d38199a8d44459fd6e1b1be560162 P 72ed48ce93664347aabfaf0f2174f058 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c7293e88c49744ccacd6f1cb5a1923b9" member_type: VOTER last_known_addr { host: "127.22.167.65" port: 41643 } } peers { permanent_uuid: "ccb6186938e8416eaf9f83ca2f8eb78f" member_type: VOTER last_known_addr { host: "127.22.167.66" port: 43099 } } peers { permanent_uuid: "72ed48ce93664347aabfaf0f2174f058" member_type: VOTER last_known_addr { host: "127.22.167.67" port: 40365 } }
I20260810 04:41:10.451436 25873 leader_election.cc:288] T 0d6d38199a8d44459fd6e1b1be560162 P 72ed48ce93664347aabfaf0f2174f058 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c7293e88c49744ccacd6f1cb5a1923b9 (127.22.167.65:41643), ccb6186938e8416eaf9f83ca2f8eb78f (127.22.167.66:43099)
I20260810 04:41:10.454350 25650 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:41:10.451612 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.65:41643 (local address 127.22.167.67:48035)
0810 04:41:10.451680 (+ 68us) negotiation.cc:107] Waiting for socket to connect
0810 04:41:10.451692 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0810 04:41:10.451753 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 04:41:10.451928 (+ 175us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 04:41:10.451929 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 04:41:10.451931 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 04:41:10.452009 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:41:10.452012 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:41:10.452687 (+ 675us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:41:10.452688 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:41:10.453583 (+ 895us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:41:10.453585 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:41:10.454084 (+ 499us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:41:10.454085 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:41:10.454187 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:41:10.454191 (+ 4us) client_negotiation.cc:769] Sending connection context
0810 04:41:10.454233 (+ 42us) client_negotiation.cc:240] Negotiation successful
0810 04:41:10.454263 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":30}
I20260810 04:41:10.454375 25859 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:41:10.451694 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.167.67:48035 (local address 127.22.167.65:41643)
0810 04:41:10.451748 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0810 04:41:10.451750 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 04:41:10.451763 (+ 13us) server_negotiation.cc:407] Connection header received
0810 04:41:10.451840 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:41:10.451842 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:41:10.451853 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:41:10.451907 (+ 54us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 04:41:10.452083 (+ 176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:41:10.452600 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:41:10.453670 (+ 1070us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:41:10.454014 (+ 344us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:41:10.454077 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:41:10.454086 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0810 04:41:10.454260 (+ 174us) server_negotiation.cc:299] Negotiation successful
0810 04:41:10.454302 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260810 04:41:10.454401 25855 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:41:10.451791 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.167.67:39619 (local address 127.22.167.66:43099)
0810 04:41:10.451833 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0810 04:41:10.451835 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 04:41:10.451893 (+ 58us) server_negotiation.cc:407] Connection header received
0810 04:41:10.451954 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:41:10.451955 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:41:10.451966 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:41:10.452015 (+ 49us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 04:41:10.452186 (+ 171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:41:10.452621 (+ 435us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:41:10.453688 (+ 1067us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:41:10.454034 (+ 346us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:41:10.454100 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:41:10.454109 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0810 04:41:10.454305 (+ 196us) server_negotiation.cc:299] Negotiation successful
0810 04:41:10.454345 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260810 04:41:10.454602 25913 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:41:10.451717 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.66:43099 (local address 127.22.167.67:39619)
0810 04:41:10.451832 (+ 115us) negotiation.cc:107] Waiting for socket to connect
0810 04:41:10.451842 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0810 04:41:10.451905 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 04:41:10.452029 (+ 124us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 04:41:10.452030 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 04:41:10.452031 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 04:41:10.452125 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:41:10.452127 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:41:10.452692 (+ 565us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:41:10.452693 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:41:10.453604 (+ 911us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:41:10.453607 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:41:10.454109 (+ 502us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:41:10.454110 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:41:10.454232 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:41:10.454237 (+ 5us) client_negotiation.cc:769] Sending connection context
0810 04:41:10.454291 (+ 54us) client_negotiation.cc:240] Negotiation successful
0810 04:41:10.454329 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":103,"thread_start_us":63,"threads_started":1}
I20260810 04:41:10.454689 25264 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0d6d38199a8d44459fd6e1b1be560162" candidate_uuid: "72ed48ce93664347aabfaf0f2174f058" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c7293e88c49744ccacd6f1cb5a1923b9" is_pre_election: true
I20260810 04:41:10.454792 25264 raft_consensus.cc:2454] T 0d6d38199a8d44459fd6e1b1be560162 P c7293e88c49744ccacd6f1cb5a1923b9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 72ed48ce93664347aabfaf0f2174f058 in term 0.
I20260810 04:41:10.454839 25504 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0d6d38199a8d44459fd6e1b1be560162" candidate_uuid: "72ed48ce93664347aabfaf0f2174f058" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccb6186938e8416eaf9f83ca2f8eb78f" is_pre_election: true
I20260810 04:41:10.454947 25504 raft_consensus.cc:2454] T 0d6d38199a8d44459fd6e1b1be560162 P ccb6186938e8416eaf9f83ca2f8eb78f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 72ed48ce93664347aabfaf0f2174f058 in term 0.
I20260810 04:41:10.454967 25629 leader_election.cc:302] T 0d6d38199a8d44459fd6e1b1be560162 P 72ed48ce93664347aabfaf0f2174f058 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 72ed48ce93664347aabfaf0f2174f058, c7293e88c49744ccacd6f1cb5a1923b9; no voters:
I20260810 04:41:10.455130 25873 raft_consensus.cc:2781] T 0d6d38199a8d44459fd6e1b1be560162 P 72ed48ce93664347aabfaf0f2174f058 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 04:41:10.455171 25873 raft_consensus.cc:482] T 0d6d38199a8d44459fd6e1b1be560162 P 72ed48ce93664347aabfaf0f2174f058 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 04:41:10.455189 25873 raft_consensus.cc:3037] T 0d6d38199a8d44459fd6e1b1be560162 P 72ed48ce93664347aabfaf0f2174f058 [term 0 FOLLOWER]: Advancing to term 1
I20260810 04:41:10.455832 25873 raft_consensus.cc:504] T 0d6d38199a8d44459fd6e1b1be560162 P 72ed48ce93664347aabfaf0f2174f058 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c7293e88c49744ccacd6f1cb5a1923b9" member_type: VOTER last_known_addr { host: "127.22.167.65" port: 41643 } } peers { permanent_uuid: "ccb6186938e8416eaf9f83ca2f8eb78f" member_type: VOTER last_known_addr { host: "127.22.167.66" port: 43099 } } peers { permanent_uuid: "72ed48ce93664347aabfaf0f2174f058" member_type: VOTER last_known_addr { host: "127.22.167.67" port: 40365 } }
I20260810 04:41:10.455938 25873 leader_election.cc:288] T 0d6d38199a8d44459fd6e1b1be560162 P 72ed48ce93664347aabfaf0f2174f058 [CANDIDATE]: Term 1 election: Requested vote from peers c7293e88c49744ccacd6f1cb5a1923b9 (127.22.167.65:41643), ccb6186938e8416eaf9f83ca2f8eb78f (127.22.167.66:43099)
I20260810 04:41:10.456079 25264 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0d6d38199a8d44459fd6e1b1be560162" candidate_uuid: "72ed48ce93664347aabfaf0f2174f058" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c7293e88c49744ccacd6f1cb5a1923b9"
I20260810 04:41:10.456101 25504 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0d6d38199a8d44459fd6e1b1be560162" candidate_uuid: "72ed48ce93664347aabfaf0f2174f058" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccb6186938e8416eaf9f83ca2f8eb78f"
I20260810 04:41:10.456161 25504 raft_consensus.cc:3037] T 0d6d38199a8d44459fd6e1b1be560162 P ccb6186938e8416eaf9f83ca2f8eb78f [term 0 FOLLOWER]: Advancing to term 1
I20260810 04:41:10.456169 25264 raft_consensus.cc:3037] T 0d6d38199a8d44459fd6e1b1be560162 P c7293e88c49744ccacd6f1cb5a1923b9 [term 0 FOLLOWER]: Advancing to term 1
I20260810 04:41:10.456928 25264 raft_consensus.cc:2454] T 0d6d38199a8d44459fd6e1b1be560162 P c7293e88c49744ccacd6f1cb5a1923b9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 72ed48ce93664347aabfaf0f2174f058 in term 1.
I20260810 04:41:10.456928 25504 raft_consensus.cc:2454] T 0d6d38199a8d44459fd6e1b1be560162 P ccb6186938e8416eaf9f83ca2f8eb78f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 72ed48ce93664347aabfaf0f2174f058 in term 1.
I20260810 04:41:10.457080 25629 leader_election.cc:302] T 0d6d38199a8d44459fd6e1b1be560162 P 72ed48ce93664347aabfaf0f2174f058 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 72ed48ce93664347aabfaf0f2174f058, c7293e88c49744ccacd6f1cb5a1923b9; no voters:
I20260810 04:41:10.457197 25873 raft_consensus.cc:2781] T 0d6d38199a8d44459fd6e1b1be560162 P 72ed48ce93664347aabfaf0f2174f058 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 04:41:10.457325 25873 raft_consensus.cc:686] T 0d6d38199a8d44459fd6e1b1be560162 P 72ed48ce93664347aabfaf0f2174f058 [term 1 LEADER]: Becoming Leader. State: Replica: 72ed48ce93664347aabfaf0f2174f058, State: Running, Role: LEADER
I20260810 04:41:10.457382 25873 consensus_queue.cc:237] T 0d6d38199a8d44459fd6e1b1be560162 P 72ed48ce93664347aabfaf0f2174f058 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c7293e88c49744ccacd6f1cb5a1923b9" member_type: VOTER last_known_addr { host: "127.22.167.65" port: 41643 } } peers { permanent_uuid: "ccb6186938e8416eaf9f83ca2f8eb78f" member_type: VOTER last_known_addr { host: "127.22.167.66" port: 43099 } } peers { permanent_uuid: "72ed48ce93664347aabfaf0f2174f058" member_type: VOTER last_known_addr { host: "127.22.167.67" port: 40365 } }
I20260810 04:41:10.458063 24994 catalog_manager.cc:5495] T 0d6d38199a8d44459fd6e1b1be560162 P 72ed48ce93664347aabfaf0f2174f058 reported cstate change: term changed from 0 to 1, leader changed from <none> to 72ed48ce93664347aabfaf0f2174f058 (127.22.167.67). New cstate: current_term: 1 leader_uuid: "72ed48ce93664347aabfaf0f2174f058" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c7293e88c49744ccacd6f1cb5a1923b9" member_type: VOTER last_known_addr { host: "127.22.167.65" port: 41643 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ccb6186938e8416eaf9f83ca2f8eb78f" member_type: VOTER last_known_addr { host: "127.22.167.66" port: 43099 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "72ed48ce93664347aabfaf0f2174f058" member_type: VOTER last_known_addr { host: "127.22.167.67" port: 40365 } health_report { overall_health: HEALTHY } } }
Aug 10 04:41:10 dist-test-slave-bqcl krb5kdc[24924](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786336869, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.167.126@KRBTEST.COM for kudu/127.22.167.126@KRBTEST.COM
I20260810 04:41:10.481014 25076 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:41:10.474811 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53230 (local address 127.22.167.126:37145)
0810 04:41:10.474867 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0810 04:41:10.474869 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 04:41:10.474991 (+ 122us) server_negotiation.cc:407] Connection header received
0810 04:41:10.475043 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:41:10.475044 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:41:10.475053 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:41:10.475092 (+ 39us) server_negotiation.cc:226] Negotiated authn=SASL
0810 04:41:10.475553 (+ 461us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:41:10.476031 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:41:10.476719 (+ 688us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:41:10.476887 (+ 168us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:41:10.476928 (+ 41us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:41:10.478898 (+ 1970us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 04:41:10.478899 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 04:41:10.478899 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 04:41:10.478906 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 04:41:10.480027 (+ 1121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 04:41:10.480448 (+ 421us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 04:41:10.480449 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 04:41:10.480449 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 04:41:10.480502 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 04:41:10.480641 (+ 139us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 04:41:10.480641 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 04:41:10.480641 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 04:41:10.480740 (+ 99us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 04:41:10.480782 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0810 04:41:10.480881 (+ 99us) server_negotiation.cc:299] Negotiation successful
0810 04:41:10.480948 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260810 04:41:10.481129 25926 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:41:10.474764 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.126:37145 (local address 127.0.0.1:53230)
0810 04:41:10.474924 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0810 04:41:10.474935 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0810 04:41:10.474994 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 04:41:10.475135 (+ 141us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 04:41:10.475136 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 04:41:10.475435 (+ 299us) client_negotiation.cc:189] Negotiated authn=SASL
0810 04:41:10.475488 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:41:10.475489 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:41:10.476115 (+ 626us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:41:10.476116 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:41:10.476659 (+ 543us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:41:10.476661 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:41:10.476944 (+ 283us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:41:10.476945 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:41:10.476990 (+ 45us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:41:10.477459 (+ 469us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 04:41:10.477464 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0810 04:41:10.478822 (+ 1358us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 04:41:10.480125 (+ 1303us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 04:41:10.480132 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 04:41:10.480134 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 04:41:10.480374 (+ 240us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 04:41:10.480551 (+ 177us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 04:41:10.480551 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 04:41:10.480551 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 04:41:10.480607 (+ 56us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 04:41:10.480778 (+ 171us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 04:41:10.480782 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 04:41:10.480821 (+ 39us) client_negotiation.cc:769] Sending connection context
0810 04:41:10.480867 (+ 46us) client_negotiation.cc:240] Negotiation successful
0810 04:41:10.481007 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":133,"mutex_wait_us":57,"thread_start_us":95,"threads_started":1}
I20260810 04:41:10.483037 24994 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.22.167.126@KRBTEST.COM'} at 127.0.0.1:53230:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260810 04:41:10.483294 24994 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 04:41:10.484591 25209 tablet_service.cc:1467] Processing CreateTablet for tablet ce11cac633ec47f6a2684dc6bd33993f (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=b108092fc6204c0eaaf10380cda6a264]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260810 04:41:10.484647 25474 tablet_service.cc:1467] Processing CreateTablet for tablet ce11cac633ec47f6a2684dc6bd33993f (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=b108092fc6204c0eaaf10380cda6a264]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260810 04:41:10.484639 25720 tablet_service.cc:1467] Processing CreateTablet for tablet ce11cac633ec47f6a2684dc6bd33993f (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=b108092fc6204c0eaaf10380cda6a264]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260810 04:41:10.484743 25209 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ce11cac633ec47f6a2684dc6bd33993f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 04:41:10.484792 25720 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ce11cac633ec47f6a2684dc6bd33993f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 04:41:10.484792 25474 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ce11cac633ec47f6a2684dc6bd33993f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 04:41:10.486238 25860 tablet_bootstrap.cc:492] T ce11cac633ec47f6a2684dc6bd33993f P ccb6186938e8416eaf9f83ca2f8eb78f: Bootstrap starting.
I20260810 04:41:10.486236 25861 tablet_bootstrap.cc:492] T ce11cac633ec47f6a2684dc6bd33993f P 72ed48ce93664347aabfaf0f2174f058: Bootstrap starting.
I20260810 04:41:10.486265 25864 tablet_bootstrap.cc:492] T ce11cac633ec47f6a2684dc6bd33993f P c7293e88c49744ccacd6f1cb5a1923b9: Bootstrap starting.
I20260810 04:41:10.486629 25860 tablet_bootstrap.cc:654] T ce11cac633ec47f6a2684dc6bd33993f P ccb6186938e8416eaf9f83ca2f8eb78f: Neither blocks nor log segments found. Creating new log.
I20260810 04:41:10.486629 25861 tablet_bootstrap.cc:654] T ce11cac633ec47f6a2684dc6bd33993f P 72ed48ce93664347aabfaf0f2174f058: Neither blocks nor log segments found. Creating new log.
I20260810 04:41:10.486644 25864 tablet_bootstrap.cc:654] T ce11cac633ec47f6a2684dc6bd33993f P c7293e88c49744ccacd6f1cb5a1923b9: Neither blocks nor log segments found. Creating new log.
I20260810 04:41:10.487275 25864 tablet_bootstrap.cc:492] T ce11cac633ec47f6a2684dc6bd33993f P c7293e88c49744ccacd6f1cb5a1923b9: No bootstrap required, opened a new log
I20260810 04:41:10.487313 25860 tablet_bootstrap.cc:492] T ce11cac633ec47f6a2684dc6bd33993f P ccb6186938e8416eaf9f83ca2f8eb78f: No bootstrap required, opened a new log
I20260810 04:41:10.487326 25864 ts_tablet_manager.cc:1397] T ce11cac633ec47f6a2684dc6bd33993f P c7293e88c49744ccacd6f1cb5a1923b9: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260810 04:41:10.487344 25861 tablet_bootstrap.cc:492] T ce11cac633ec47f6a2684dc6bd33993f P 72ed48ce93664347aabfaf0f2174f058: No bootstrap required, opened a new log
I20260810 04:41:10.487349 25860 ts_tablet_manager.cc:1397] T ce11cac633ec47f6a2684dc6bd33993f P ccb6186938e8416eaf9f83ca2f8eb78f: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260810 04:41:10.487381 25861 ts_tablet_manager.cc:1397] T ce11cac633ec47f6a2684dc6bd33993f P 72ed48ce93664347aabfaf0f2174f058: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260810 04:41:10.487461 25864 raft_consensus.cc:348] T ce11cac633ec47f6a2684dc6bd33993f P c7293e88c49744ccacd6f1cb5a1923b9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c7293e88c49744ccacd6f1cb5a1923b9" member_type: VOTER last_known_addr { host: "127.22.167.65" port: 41643 } } peers { permanent_uuid: "ccb6186938e8416eaf9f83ca2f8eb78f" member_type: VOTER last_known_addr { host: "127.22.167.66" port: 43099 } } peers { permanent_uuid: "72ed48ce93664347aabfaf0f2174f058" member_type: VOTER last_known_addr { host: "127.22.167.67" port: 40365 } }
I20260810 04:41:10.487453 25860 raft_consensus.cc:348] T ce11cac633ec47f6a2684dc6bd33993f P ccb6186938e8416eaf9f83ca2f8eb78f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c7293e88c49744ccacd6f1cb5a1923b9" member_type: VOTER last_known_addr { host: "127.22.167.65" port: 41643 } } peers { permanent_uuid: "ccb6186938e8416eaf9f83ca2f8eb78f" member_type: VOTER last_known_addr { host: "127.22.167.66" port: 43099 } } peers { permanent_uuid: "72ed48ce93664347aabfaf0f2174f058" member_type: VOTER last_known_addr { host: "127.22.167.67" port: 40365 } }
I20260810 04:41:10.487488 25861 raft_consensus.cc:348] T ce11cac633ec47f6a2684dc6bd33993f P 72ed48ce93664347aabfaf0f2174f058 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c7293e88c49744ccacd6f1cb5a1923b9" member_type: VOTER last_known_addr { host: "127.22.167.65" port: 41643 } } peers { permanent_uuid: "ccb6186938e8416eaf9f83ca2f8eb78f" member_type: VOTER last_known_addr { host: "127.22.167.66" port: 43099 } } peers { permanent_uuid: "72ed48ce93664347aabfaf0f2174f058" member_type: VOTER last_known_addr { host: "127.22.167.67" port: 40365 } }
I20260810 04:41:10.487521 25860 raft_consensus.cc:374] T ce11cac633ec47f6a2684dc6bd33993f P ccb6186938e8416eaf9f83ca2f8eb78f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 04:41:10.487532 25861 raft_consensus.cc:374] T ce11cac633ec47f6a2684dc6bd33993f P 72ed48ce93664347aabfaf0f2174f058 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 04:41:10.487527 25864 raft_consensus.cc:374] T ce11cac633ec47f6a2684dc6bd33993f P c7293e88c49744ccacd6f1cb5a1923b9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 04:41:10.487545 25861 raft_consensus.cc:729] T ce11cac633ec47f6a2684dc6bd33993f P 72ed48ce93664347aabfaf0f2174f058 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 72ed48ce93664347aabfaf0f2174f058, State: Initialized, Role: FOLLOWER
I20260810 04:41:10.487543 25860 raft_consensus.cc:729] T ce11cac633ec47f6a2684dc6bd33993f P ccb6186938e8416eaf9f83ca2f8eb78f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ccb6186938e8416eaf9f83ca2f8eb78f, State: Initialized, Role: FOLLOWER
I20260810 04:41:10.487550 25864 raft_consensus.cc:729] T ce11cac633ec47f6a2684dc6bd33993f P c7293e88c49744ccacd6f1cb5a1923b9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c7293e88c49744ccacd6f1cb5a1923b9, State: Initialized, Role: FOLLOWER
I20260810 04:41:10.487583 25861 consensus_queue.cc:260] T ce11cac633ec47f6a2684dc6bd33993f P 72ed48ce93664347aabfaf0f2174f058 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c7293e88c49744ccacd6f1cb5a1923b9" member_type: VOTER last_known_addr { host: "127.22.167.65" port: 41643 } } peers { permanent_uuid: "ccb6186938e8416eaf9f83ca2f8eb78f" member_type: VOTER last_known_addr { host: "127.22.167.66" port: 43099 } } peers { permanent_uuid: "72ed48ce93664347aabfaf0f2174f058" member_type: VOTER last_known_addr { host: "127.22.167.67" port: 40365 } }
I20260810 04:41:10.487591 25860 consensus_queue.cc:260] T ce11cac633ec47f6a2684dc6bd33993f P ccb6186938e8416eaf9f83ca2f8eb78f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c7293e88c49744ccacd6f1cb5a1923b9" member_type: VOTER last_known_addr { host: "127.22.167.65" port: 41643 } } peers { permanent_uuid: "ccb6186938e8416eaf9f83ca2f8eb78f" member_type: VOTER last_known_addr { host: "127.22.167.66" port: 43099 } } peers { permanent_uuid: "72ed48ce93664347aabfaf0f2174f058" member_type: VOTER last_known_addr { host: "127.22.167.67" port: 40365 } }
I20260810 04:41:10.487591 25864 consensus_queue.cc:260] T ce11cac633ec47f6a2684dc6bd33993f P c7293e88c49744ccacd6f1cb5a1923b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c7293e88c49744ccacd6f1cb5a1923b9" member_type: VOTER last_known_addr { host: "127.22.167.65" port: 41643 } } peers { permanent_uuid: "ccb6186938e8416eaf9f83ca2f8eb78f" member_type: VOTER last_known_addr { host: "127.22.167.66" port: 43099 } } peers { permanent_uuid: "72ed48ce93664347aabfaf0f2174f058" member_type: VOTER last_known_addr { host: "127.22.167.67" port: 40365 } }
I20260810 04:41:10.487674 25864 ts_tablet_manager.cc:1428] T ce11cac633ec47f6a2684dc6bd33993f P c7293e88c49744ccacd6f1cb5a1923b9: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260810 04:41:10.487674 25860 ts_tablet_manager.cc:1428] T ce11cac633ec47f6a2684dc6bd33993f P ccb6186938e8416eaf9f83ca2f8eb78f: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260810 04:41:10.487675 25861 ts_tablet_manager.cc:1428] T ce11cac633ec47f6a2684dc6bd33993f P 72ed48ce93664347aabfaf0f2174f058: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260810 04:41:10.487704 25873 tablet_replica.cc:439] T ce11cac633ec47f6a2684dc6bd33993f P 72ed48ce93664347aabfaf0f2174f058: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c7293e88c49744ccacd6f1cb5a1923b9" member_type: VOTER last_known_addr { host: "127.22.167.65" port: 41643 } } peers { permanent_uuid: "ccb6186938e8416eaf9f83ca2f8eb78f" member_type: VOTER last_known_addr { host: "127.22.167.66" port: 43099 } } peers { permanent_uuid: "72ed48ce93664347aabfaf0f2174f058" member_type: VOTER last_known_addr { host: "127.22.167.67" port: 40365 } } }
I20260810 04:41:10.487768 25873 tablet_replica.cc:442] T ce11cac633ec47f6a2684dc6bd33993f P 72ed48ce93664347aabfaf0f2174f058: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 04:41:10.487793 25895 tablet_replica.cc:439] T ce11cac633ec47f6a2684dc6bd33993f P ccb6186938e8416eaf9f83ca2f8eb78f: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c7293e88c49744ccacd6f1cb5a1923b9" member_type: VOTER last_known_addr { host: "127.22.167.65" port: 41643 } } peers { permanent_uuid: "ccb6186938e8416eaf9f83ca2f8eb78f" member_type: VOTER last_known_addr { host: "127.22.167.66" port: 43099 } } peers { permanent_uuid: "72ed48ce93664347aabfaf0f2174f058" member_type: VOTER last_known_addr { host: "127.22.167.67" port: 40365 } } }
I20260810 04:41:10.487864 25895 tablet_replica.cc:442] T ce11cac633ec47f6a2684dc6bd33993f P ccb6186938e8416eaf9f83ca2f8eb78f: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 04:41:10.487900 25872 tablet_replica.cc:439] T ce11cac633ec47f6a2684dc6bd33993f P c7293e88c49744ccacd6f1cb5a1923b9: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c7293e88c49744ccacd6f1cb5a1923b9" member_type: VOTER last_known_addr { host: "127.22.167.65" port: 41643 } } peers { permanent_uuid: "ccb6186938e8416eaf9f83ca2f8eb78f" member_type: VOTER last_known_addr { host: "127.22.167.66" port: 43099 } } peers { permanent_uuid: "72ed48ce93664347aabfaf0f2174f058" member_type: VOTER last_known_addr { host: "127.22.167.67" port: 40365 } } }
I20260810 04:41:10.487988 25872 tablet_replica.cc:442] T ce11cac633ec47f6a2684dc6bd33993f P c7293e88c49744ccacd6f1cb5a1923b9: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 04:41:10.570787 25872 raft_consensus.cc:482] T ce11cac633ec47f6a2684dc6bd33993f P c7293e88c49744ccacd6f1cb5a1923b9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 04:41:10.570885 25872 raft_consensus.cc:504] T ce11cac633ec47f6a2684dc6bd33993f P c7293e88c49744ccacd6f1cb5a1923b9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c7293e88c49744ccacd6f1cb5a1923b9" member_type: VOTER last_known_addr { host: "127.22.167.65" port: 41643 } } peers { permanent_uuid: "ccb6186938e8416eaf9f83ca2f8eb78f" member_type: VOTER last_known_addr { host: "127.22.167.66" port: 43099 } } peers { permanent_uuid: "72ed48ce93664347aabfaf0f2174f058" member_type: VOTER last_known_addr { host: "127.22.167.67" port: 40365 } }
I20260810 04:41:10.571223 25872 leader_election.cc:288] T ce11cac633ec47f6a2684dc6bd33993f P c7293e88c49744ccacd6f1cb5a1923b9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ccb6186938e8416eaf9f83ca2f8eb78f (127.22.167.66:43099), 72ed48ce93664347aabfaf0f2174f058 (127.22.167.67:40365)
I20260810 04:41:10.574060 25855 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:41:10.571462 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.167.65:55173 (local address 127.22.167.66:43099)
0810 04:41:10.571514 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0810 04:41:10.571516 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 04:41:10.571668 (+ 152us) server_negotiation.cc:407] Connection header received
0810 04:41:10.571736 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:41:10.571737 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:41:10.571747 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:41:10.571795 (+ 48us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 04:41:10.571941 (+ 146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:41:10.572431 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:41:10.573386 (+ 955us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:41:10.573737 (+ 351us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:41:10.573794 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:41:10.573804 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0810 04:41:10.573954 (+ 150us) server_negotiation.cc:299] Negotiation successful
0810 04:41:10.573993 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260810 04:41:10.574074 25854 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:41:10.571547 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.167.65:40371 (local address 127.22.167.67:40365)
0810 04:41:10.571591 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0810 04:41:10.571593 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 04:41:10.571687 (+ 94us) server_negotiation.cc:407] Connection header received
0810 04:41:10.571744 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:41:10.571745 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:41:10.571754 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:41:10.571803 (+ 49us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 04:41:10.571977 (+ 174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:41:10.572447 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:41:10.573392 (+ 945us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:41:10.573748 (+ 356us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:41:10.573812 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:41:10.573822 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0810 04:41:10.573969 (+ 147us) server_negotiation.cc:299] Negotiation successful
0810 04:41:10.574009 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260810 04:41:10.574093 25933 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:41:10.571394 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.67:40365 (local address 127.22.167.65:40371)
0810 04:41:10.571599 (+ 205us) negotiation.cc:107] Waiting for socket to connect
0810 04:41:10.571610 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0810 04:41:10.571691 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 04:41:10.571817 (+ 126us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 04:41:10.571818 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 04:41:10.571820 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 04:41:10.571907 (+ 87us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:41:10.571908 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:41:10.572523 (+ 615us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:41:10.572524 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:41:10.573301 (+ 777us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:41:10.573303 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:41:10.573828 (+ 525us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:41:10.573829 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:41:10.573898 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:41:10.573901 (+ 3us) client_negotiation.cc:769] Sending connection context
0810 04:41:10.573949 (+ 48us) client_negotiation.cc:240] Negotiation successful
0810 04:41:10.573989 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":184,"mutex_wait_us":29,"thread_start_us":122,"threads_started":1}
I20260810 04:41:10.574100 25932 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:41:10.571394 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.66:43099 (local address 127.22.167.65:55173)
0810 04:41:10.571602 (+ 208us) negotiation.cc:107] Waiting for socket to connect
0810 04:41:10.571612 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0810 04:41:10.571667 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 04:41:10.571809 (+ 142us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 04:41:10.571810 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 04:41:10.571812 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 04:41:10.571885 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:41:10.571887 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:41:10.572505 (+ 618us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:41:10.572506 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:41:10.573301 (+ 795us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:41:10.573303 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:41:10.573823 (+ 520us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:41:10.573824 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:41:10.573891 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:41:10.573895 (+ 4us) client_negotiation.cc:769] Sending connection context
0810 04:41:10.573938 (+ 43us) client_negotiation.cc:240] Negotiation successful
0810 04:41:10.573980 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":189,"mutex_wait_us":3,"thread_start_us":121,"threads_started":1}
I20260810 04:41:10.574447 25746 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ce11cac633ec47f6a2684dc6bd33993f" candidate_uuid: "c7293e88c49744ccacd6f1cb5a1923b9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "72ed48ce93664347aabfaf0f2174f058" is_pre_election: true
I20260810 04:41:10.574532 25746 raft_consensus.cc:2454] T ce11cac633ec47f6a2684dc6bd33993f P 72ed48ce93664347aabfaf0f2174f058 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7293e88c49744ccacd6f1cb5a1923b9 in term 0.
I20260810 04:41:10.574543 25504 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ce11cac633ec47f6a2684dc6bd33993f" candidate_uuid: "c7293e88c49744ccacd6f1cb5a1923b9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccb6186938e8416eaf9f83ca2f8eb78f" is_pre_election: true
I20260810 04:41:10.574645 25504 raft_consensus.cc:2454] T ce11cac633ec47f6a2684dc6bd33993f P ccb6186938e8416eaf9f83ca2f8eb78f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7293e88c49744ccacd6f1cb5a1923b9 in term 0.
I20260810 04:41:10.574734 25149 leader_election.cc:302] T ce11cac633ec47f6a2684dc6bd33993f P c7293e88c49744ccacd6f1cb5a1923b9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 72ed48ce93664347aabfaf0f2174f058, c7293e88c49744ccacd6f1cb5a1923b9; no voters:
I20260810 04:41:10.574883 25872 raft_consensus.cc:2781] T ce11cac633ec47f6a2684dc6bd33993f P c7293e88c49744ccacd6f1cb5a1923b9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 04:41:10.574925 25872 raft_consensus.cc:482] T ce11cac633ec47f6a2684dc6bd33993f P c7293e88c49744ccacd6f1cb5a1923b9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 04:41:10.574939 25872 raft_consensus.cc:3037] T ce11cac633ec47f6a2684dc6bd33993f P c7293e88c49744ccacd6f1cb5a1923b9 [term 0 FOLLOWER]: Advancing to term 1
I20260810 04:41:10.575744 25872 raft_consensus.cc:504] T ce11cac633ec47f6a2684dc6bd33993f P c7293e88c49744ccacd6f1cb5a1923b9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c7293e88c49744ccacd6f1cb5a1923b9" member_type: VOTER last_known_addr { host: "127.22.167.65" port: 41643 } } peers { permanent_uuid: "ccb6186938e8416eaf9f83ca2f8eb78f" member_type: VOTER last_known_addr { host: "127.22.167.66" port: 43099 } } peers { permanent_uuid: "72ed48ce93664347aabfaf0f2174f058" member_type: VOTER last_known_addr { host: "127.22.167.67" port: 40365 } }
I20260810 04:41:10.575856 25872 leader_election.cc:288] T ce11cac633ec47f6a2684dc6bd33993f P c7293e88c49744ccacd6f1cb5a1923b9 [CANDIDATE]: Term 1 election: Requested vote from peers ccb6186938e8416eaf9f83ca2f8eb78f (127.22.167.66:43099), 72ed48ce93664347aabfaf0f2174f058 (127.22.167.67:40365)
I20260810 04:41:10.576021 25746 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ce11cac633ec47f6a2684dc6bd33993f" candidate_uuid: "c7293e88c49744ccacd6f1cb5a1923b9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "72ed48ce93664347aabfaf0f2174f058"
I20260810 04:41:10.576097 25746 raft_consensus.cc:3037] T ce11cac633ec47f6a2684dc6bd33993f P 72ed48ce93664347aabfaf0f2174f058 [term 0 FOLLOWER]: Advancing to term 1
I20260810 04:41:10.576699 25746 raft_consensus.cc:2454] T ce11cac633ec47f6a2684dc6bd33993f P 72ed48ce93664347aabfaf0f2174f058 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c7293e88c49744ccacd6f1cb5a1923b9 in term 1.
I20260810 04:41:10.576804 25504 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ce11cac633ec47f6a2684dc6bd33993f" candidate_uuid: "c7293e88c49744ccacd6f1cb5a1923b9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccb6186938e8416eaf9f83ca2f8eb78f"
I20260810 04:41:10.576841 25149 leader_election.cc:302] T ce11cac633ec47f6a2684dc6bd33993f P c7293e88c49744ccacd6f1cb5a1923b9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 72ed48ce93664347aabfaf0f2174f058, c7293e88c49744ccacd6f1cb5a1923b9; no voters:
I20260810 04:41:10.576880 25504 raft_consensus.cc:3037] T ce11cac633ec47f6a2684dc6bd33993f P ccb6186938e8416eaf9f83ca2f8eb78f [term 0 FOLLOWER]: Advancing to term 1
I20260810 04:41:10.576923 25872 raft_consensus.cc:2781] T ce11cac633ec47f6a2684dc6bd33993f P c7293e88c49744ccacd6f1cb5a1923b9 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 04:41:10.577064 25872 raft_consensus.cc:686] T ce11cac633ec47f6a2684dc6bd33993f P c7293e88c49744ccacd6f1cb5a1923b9 [term 1 LEADER]: Becoming Leader. State: Replica: c7293e88c49744ccacd6f1cb5a1923b9, State: Running, Role: LEADER
I20260810 04:41:10.577132 25872 consensus_queue.cc:237] T ce11cac633ec47f6a2684dc6bd33993f P c7293e88c49744ccacd6f1cb5a1923b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c7293e88c49744ccacd6f1cb5a1923b9" member_type: VOTER last_known_addr { host: "127.22.167.65" port: 41643 } } peers { permanent_uuid: "ccb6186938e8416eaf9f83ca2f8eb78f" member_type: VOTER last_known_addr { host: "127.22.167.66" port: 43099 } } peers { permanent_uuid: "72ed48ce93664347aabfaf0f2174f058" member_type: VOTER last_known_addr { host: "127.22.167.67" port: 40365 } }
I20260810 04:41:10.577607 25504 raft_consensus.cc:2454] T ce11cac633ec47f6a2684dc6bd33993f P ccb6186938e8416eaf9f83ca2f8eb78f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c7293e88c49744ccacd6f1cb5a1923b9 in term 1.
I20260810 04:41:10.577610 25934 tablet_replica.cc:439] T ce11cac633ec47f6a2684dc6bd33993f P c7293e88c49744ccacd6f1cb5a1923b9: TxnStatusTablet state changed. Reason: New leader c7293e88c49744ccacd6f1cb5a1923b9. Latest consensus state: current_term: 1 leader_uuid: "c7293e88c49744ccacd6f1cb5a1923b9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c7293e88c49744ccacd6f1cb5a1923b9" member_type: VOTER last_known_addr { host: "127.22.167.65" port: 41643 } } peers { permanent_uuid: "ccb6186938e8416eaf9f83ca2f8eb78f" member_type: VOTER last_known_addr { host: "127.22.167.66" port: 43099 } } peers { permanent_uuid: "72ed48ce93664347aabfaf0f2174f058" member_type: VOTER last_known_addr { host: "127.22.167.67" port: 40365 } } }
I20260810 04:41:10.577817 25934 tablet_replica.cc:442] T ce11cac633ec47f6a2684dc6bd33993f P c7293e88c49744ccacd6f1cb5a1923b9: This TxnStatusTablet replica's current role is: LEADER
I20260810 04:41:10.577967 24996 catalog_manager.cc:5495] T ce11cac633ec47f6a2684dc6bd33993f P c7293e88c49744ccacd6f1cb5a1923b9 reported cstate change: term changed from 0 to 1, leader changed from <none> to c7293e88c49744ccacd6f1cb5a1923b9 (127.22.167.65). New cstate: current_term: 1 leader_uuid: "c7293e88c49744ccacd6f1cb5a1923b9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c7293e88c49744ccacd6f1cb5a1923b9" member_type: VOTER last_known_addr { host: "127.22.167.65" port: 41643 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ccb6186938e8416eaf9f83ca2f8eb78f" member_type: VOTER last_known_addr { host: "127.22.167.66" port: 43099 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "72ed48ce93664347aabfaf0f2174f058" member_type: VOTER last_known_addr { host: "127.22.167.67" port: 40365 } health_report { overall_health: UNKNOWN } } }
I20260810 04:41:10.578042 25936 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260810 04:41:10.578296 25936 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260810 04:41:10.603965 25859 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:41:10.600969 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36656 (local address 127.22.167.65:41643)
0810 04:41:10.601049 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0810 04:41:10.601052 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 04:41:10.601164 (+ 112us) server_negotiation.cc:407] Connection header received
0810 04:41:10.601277 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:41:10.601278 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:41:10.601298 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:41:10.601358 (+ 60us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 04:41:10.601512 (+ 154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:41:10.602055 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:41:10.603246 (+ 1191us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:41:10.603435 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:41:10.603491 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:41:10.603687 (+ 196us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 04:41:10.603783 (+ 96us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 04:41:10.603832 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0810 04:41:10.603890 (+ 58us) server_negotiation.cc:299] Negotiation successful
0810 04:41:10.603907 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260810 04:41:10.604025 25926 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:41:10.600913 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.65:41643 (local address 127.0.0.1:36656)
0810 04:41:10.601115 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0810 04:41:10.601126 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0810 04:41:10.601180 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 04:41:10.601373 (+ 193us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 04:41:10.601374 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 04:41:10.601375 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0810 04:41:10.601433 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:41:10.601435 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:41:10.602188 (+ 753us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:41:10.602189 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:41:10.603149 (+ 960us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:41:10.603153 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:41:10.603502 (+ 349us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:41:10.603503 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:41:10.603578 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:41:10.603597 (+ 19us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0810 04:41:10.603833 (+ 236us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0810 04:41:10.603838 (+ 5us) client_negotiation.cc:769] Sending connection context
0810 04:41:10.603887 (+ 49us) client_negotiation.cc:240] Negotiation successful
0810 04:41:10.603938 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":163}
I20260810 04:41:10.605307 25746 raft_consensus.cc:1261] T ce11cac633ec47f6a2684dc6bd33993f P 72ed48ce93664347aabfaf0f2174f058 [term 1 FOLLOWER]: Refusing update from remote peer c7293e88c49744ccacd6f1cb5a1923b9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 04:41:10.605330 25504 raft_consensus.cc:1261] T ce11cac633ec47f6a2684dc6bd33993f P ccb6186938e8416eaf9f83ca2f8eb78f [term 1 FOLLOWER]: Refusing update from remote peer c7293e88c49744ccacd6f1cb5a1923b9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 04:41:10.605561 25934 consensus_queue.cc:1035] T ce11cac633ec47f6a2684dc6bd33993f P c7293e88c49744ccacd6f1cb5a1923b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "72ed48ce93664347aabfaf0f2174f058" member_type: VOTER last_known_addr { host: "127.22.167.67" port: 40365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 04:41:10.605758 25872 consensus_queue.cc:1035] T ce11cac633ec47f6a2684dc6bd33993f P c7293e88c49744ccacd6f1cb5a1923b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ccb6186938e8416eaf9f83ca2f8eb78f" member_type: VOTER last_known_addr { host: "127.22.167.66" port: 43099 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 04:41:10.606339 25895 tablet_replica.cc:439] T ce11cac633ec47f6a2684dc6bd33993f P ccb6186938e8416eaf9f83ca2f8eb78f: TxnStatusTablet state changed. Reason: New leader c7293e88c49744ccacd6f1cb5a1923b9. Latest consensus state: current_term: 1 leader_uuid: "c7293e88c49744ccacd6f1cb5a1923b9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c7293e88c49744ccacd6f1cb5a1923b9" member_type: VOTER last_known_addr { host: "127.22.167.65" port: 41643 } } peers { permanent_uuid: "ccb6186938e8416eaf9f83ca2f8eb78f" member_type: VOTER last_known_addr { host: "127.22.167.66" port: 43099 } } peers { permanent_uuid: "72ed48ce93664347aabfaf0f2174f058" member_type: VOTER last_known_addr { host: "127.22.167.67" port: 40365 } } }
I20260810 04:41:10.606429 25895 tablet_replica.cc:442] T ce11cac633ec47f6a2684dc6bd33993f P ccb6186938e8416eaf9f83ca2f8eb78f: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 04:41:10.606383 25873 tablet_replica.cc:439] T ce11cac633ec47f6a2684dc6bd33993f P 72ed48ce93664347aabfaf0f2174f058: TxnStatusTablet state changed. Reason: New leader c7293e88c49744ccacd6f1cb5a1923b9. Latest consensus state: current_term: 1 leader_uuid: "c7293e88c49744ccacd6f1cb5a1923b9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c7293e88c49744ccacd6f1cb5a1923b9" member_type: VOTER last_known_addr { host: "127.22.167.65" port: 41643 } } peers { permanent_uuid: "ccb6186938e8416eaf9f83ca2f8eb78f" member_type: VOTER last_known_addr { host: "127.22.167.66" port: 43099 } } peers { permanent_uuid: "72ed48ce93664347aabfaf0f2174f058" member_type: VOTER last_known_addr { host: "127.22.167.67" port: 40365 } } }
I20260810 04:41:10.606472 25873 tablet_replica.cc:442] T ce11cac633ec47f6a2684dc6bd33993f P 72ed48ce93664347aabfaf0f2174f058: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 04:41:10.607157 25872 tablet_replica.cc:439] T ce11cac633ec47f6a2684dc6bd33993f P c7293e88c49744ccacd6f1cb5a1923b9: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "c7293e88c49744ccacd6f1cb5a1923b9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c7293e88c49744ccacd6f1cb5a1923b9" member_type: VOTER last_known_addr { host: "127.22.167.65" port: 41643 } } peers { permanent_uuid: "ccb6186938e8416eaf9f83ca2f8eb78f" member_type: VOTER last_known_addr { host: "127.22.167.66" port: 43099 } } peers { permanent_uuid: "72ed48ce93664347aabfaf0f2174f058" member_type: VOTER last_known_addr { host: "127.22.167.67" port: 40365 } } }
I20260810 04:41:10.607255 25872 tablet_replica.cc:442] T ce11cac633ec47f6a2684dc6bd33993f P c7293e88c49744ccacd6f1cb5a1923b9: This TxnStatusTablet replica's current role is: LEADER
I20260810 04:41:10.607347 25872 tablet_replica.cc:439] T ce11cac633ec47f6a2684dc6bd33993f P c7293e88c49744ccacd6f1cb5a1923b9: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "c7293e88c49744ccacd6f1cb5a1923b9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c7293e88c49744ccacd6f1cb5a1923b9" member_type: VOTER last_known_addr { host: "127.22.167.65" port: 41643 } } peers { permanent_uuid: "ccb6186938e8416eaf9f83ca2f8eb78f" member_type: VOTER last_known_addr { host: "127.22.167.66" port: 43099 } } peers { permanent_uuid: "72ed48ce93664347aabfaf0f2174f058" member_type: VOTER last_known_addr { host: "127.22.167.67" port: 40365 } } }
I20260810 04:41:10.607403 25872 tablet_replica.cc:442] T ce11cac633ec47f6a2684dc6bd33993f P c7293e88c49744ccacd6f1cb5a1923b9: This TxnStatusTablet replica's current role is: LEADER
I20260810 04:41:10.607383 25873 tablet_replica.cc:439] T ce11cac633ec47f6a2684dc6bd33993f P 72ed48ce93664347aabfaf0f2174f058: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "c7293e88c49744ccacd6f1cb5a1923b9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c7293e88c49744ccacd6f1cb5a1923b9" member_type: VOTER last_known_addr { host: "127.22.167.65" port: 41643 } } peers { permanent_uuid: "ccb6186938e8416eaf9f83ca2f8eb78f" member_type: VOTER last_known_addr { host: "127.22.167.66" port: 43099 } } peers { permanent_uuid: "72ed48ce93664347aabfaf0f2174f058" member_type: VOTER last_known_addr { host: "127.22.167.67" port: 40365 } } }
I20260810 04:41:10.607465 25873 tablet_replica.cc:442] T ce11cac633ec47f6a2684dc6bd33993f P 72ed48ce93664347aabfaf0f2174f058: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 04:41:10.608362 25895 tablet_replica.cc:439] T ce11cac633ec47f6a2684dc6bd33993f P ccb6186938e8416eaf9f83ca2f8eb78f: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "c7293e88c49744ccacd6f1cb5a1923b9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c7293e88c49744ccacd6f1cb5a1923b9" member_type: VOTER last_known_addr { host: "127.22.167.65" port: 41643 } } peers { permanent_uuid: "ccb6186938e8416eaf9f83ca2f8eb78f" member_type: VOTER last_known_addr { host: "127.22.167.66" port: 43099 } } peers { permanent_uuid: "72ed48ce93664347aabfaf0f2174f058" member_type: VOTER last_known_addr { host: "127.22.167.67" port: 40365 } } }
I20260810 04:41:10.608453 25895 tablet_replica.cc:442] T ce11cac633ec47f6a2684dc6bd33993f P ccb6186938e8416eaf9f83ca2f8eb78f: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 04:41:10.611683 25854 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:41:10.609247 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57238 (local address 127.22.167.67:40365)
0810 04:41:10.609287 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0810 04:41:10.609289 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 04:41:10.609298 (+ 9us) server_negotiation.cc:407] Connection header received
0810 04:41:10.609352 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:41:10.609353 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:41:10.609374 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:41:10.609440 (+ 66us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 04:41:10.609516 (+ 76us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:41:10.610274 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:41:10.610929 (+ 655us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:41:10.611236 (+ 307us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:41:10.611301 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:41:10.611385 (+ 84us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 04:41:10.611501 (+ 116us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 04:41:10.611565 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0810 04:41:10.611580 (+ 15us) server_negotiation.cc:299] Negotiation successful
0810 04:41:10.611599 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260810 04:41:10.616900 25859 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:41:10.613833 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.167.67:52865 (local address 127.22.167.65:41643)
0810 04:41:10.613884 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0810 04:41:10.613886 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 04:41:10.614261 (+ 375us) server_negotiation.cc:407] Connection header received
0810 04:41:10.614316 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:41:10.614317 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:41:10.614328 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:41:10.614385 (+ 57us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 04:41:10.614488 (+ 103us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:41:10.615172 (+ 684us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:41:10.616258 (+ 1086us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:41:10.616486 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:41:10.616548 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:41:10.616704 (+ 156us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 04:41:10.616732 (+ 28us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 04:41:10.616775 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0810 04:41:10.616824 (+ 49us) server_negotiation.cc:299] Negotiation successful
0810 04:41:10.616840 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260810 04:41:10.616948 25949 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:41:10.613773 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.65:41643 (local address 127.22.167.67:52865)
0810 04:41:10.613933 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0810 04:41:10.613943 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0810 04:41:10.614264 (+ 321us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 04:41:10.614386 (+ 122us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 04:41:10.614387 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 04:41:10.614388 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0810 04:41:10.614434 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:41:10.614436 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:41:10.615249 (+ 813us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:41:10.615250 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:41:10.616187 (+ 937us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:41:10.616189 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:41:10.616556 (+ 367us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:41:10.616557 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:41:10.616617 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:41:10.616632 (+ 15us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0810 04:41:10.616783 (+ 151us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0810 04:41:10.616784 (+ 1us) client_negotiation.cc:769] Sending connection context
0810 04:41:10.616823 (+ 39us) client_negotiation.cc:240] Negotiation successful
0810 04:41:10.616865 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":81,"threads_started":1}
I20260810 04:41:10.619201 25504 raft_consensus.cc:1261] T 0d6d38199a8d44459fd6e1b1be560162 P ccb6186938e8416eaf9f83ca2f8eb78f [term 1 FOLLOWER]: Refusing update from remote peer 72ed48ce93664347aabfaf0f2174f058: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 04:41:10.619343 25264 raft_consensus.cc:1261] T 0d6d38199a8d44459fd6e1b1be560162 P c7293e88c49744ccacd6f1cb5a1923b9 [term 1 FOLLOWER]: Refusing update from remote peer 72ed48ce93664347aabfaf0f2174f058: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 04:41:10.619671 25916 consensus_queue.cc:1035] T 0d6d38199a8d44459fd6e1b1be560162 P 72ed48ce93664347aabfaf0f2174f058 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ccb6186938e8416eaf9f83ca2f8eb78f" member_type: VOTER last_known_addr { host: "127.22.167.66" port: 43099 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 04:41:10.619784 25873 consensus_queue.cc:1035] T 0d6d38199a8d44459fd6e1b1be560162 P 72ed48ce93664347aabfaf0f2174f058 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7293e88c49744ccacd6f1cb5a1923b9" member_type: VOTER last_known_addr { host: "127.22.167.65" port: 41643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 04:41:10.640354 25854 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:41:10.630170 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.167.65:60697 (local address 127.22.167.67:40365)
0810 04:41:10.638169 (+ 7999us) server_negotiation.cc:206] Beginning negotiation
0810 04:41:10.638171 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 04:41:10.638180 (+ 9us) server_negotiation.cc:407] Connection header received
0810 04:41:10.638201 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:41:10.638202 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:41:10.638211 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:41:10.638272 (+ 61us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 04:41:10.638391 (+ 119us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:41:10.638866 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:41:10.639738 (+ 872us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:41:10.639926 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:41:10.639956 (+ 30us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:41:10.640125 (+ 169us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 04:41:10.640158 (+ 33us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 04:41:10.640203 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0810 04:41:10.640259 (+ 56us) server_negotiation.cc:299] Negotiation successful
0810 04:41:10.640279 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":7974}
I20260810 04:41:10.640368 25959 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:41:10.629907 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.67:40365 (local address 127.22.167.65:60697)
0810 04:41:10.630783 (+ 876us) negotiation.cc:107] Waiting for socket to connect
0810 04:41:10.630794 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0810 04:41:10.630986 (+ 192us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 04:41:10.638281 (+ 7295us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 04:41:10.638282 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 04:41:10.638284 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0810 04:41:10.638324 (+ 40us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:41:10.638325 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:41:10.638935 (+ 610us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:41:10.638936 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:41:10.639648 (+ 712us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:41:10.639651 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:41:10.639998 (+ 347us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:41:10.639999 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:41:10.640038 (+ 39us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:41:10.640052 (+ 14us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0810 04:41:10.640210 (+ 158us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0810 04:41:10.640212 (+ 2us) client_negotiation.cc:769] Sending connection context
0810 04:41:10.640252 (+ 40us) client_negotiation.cc:240] Negotiation successful
0810 04:41:10.640297 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":837,"thread_start_us":795,"threads_started":1}
I20260810 04:41:10.656857 24994 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:53220:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 04:41:10.656986 24994 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:53220:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 04:41:10.657721 24994 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 09efc1c3f6d6489eb08c9e5e3287d110: Sending DeleteTablet for 3 replicas of tablet 0d6d38199a8d44459fd6e1b1be560162
I20260810 04:41:10.657984 25720 tablet_service.cc:1514] Processing DeleteTablet for tablet 0d6d38199a8d44459fd6e1b1be560162 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 04:41:10 UTC) from {username='kudu', principal='kudu/127.22.167.126@KRBTEST.COM'} at 127.0.0.1:57222
I20260810 04:41:10.658092 25907 tablet_replica.cc:330] T 0d6d38199a8d44459fd6e1b1be560162 P 72ed48ce93664347aabfaf0f2174f058: stopping tablet replica
I20260810 04:41:10.658154 25907 raft_consensus.cc:2229] T 0d6d38199a8d44459fd6e1b1be560162 P 72ed48ce93664347aabfaf0f2174f058 [term 1 LEADER]: Raft consensus shutting down.
I20260810 04:41:10.658102 25474 tablet_service.cc:1514] Processing DeleteTablet for tablet 0d6d38199a8d44459fd6e1b1be560162 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 04:41:10 UTC) from {username='kudu', principal='kudu/127.22.167.126@KRBTEST.COM'} at 127.0.0.1:37082
I20260810 04:41:10.658223 25905 tablet_replica.cc:330] T 0d6d38199a8d44459fd6e1b1be560162 P ccb6186938e8416eaf9f83ca2f8eb78f: stopping tablet replica
I20260810 04:41:10.658241 25907 raft_consensus.cc:2258] T 0d6d38199a8d44459fd6e1b1be560162 P 72ed48ce93664347aabfaf0f2174f058 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 04:41:10.658269 25905 raft_consensus.cc:2229] T 0d6d38199a8d44459fd6e1b1be560162 P ccb6186938e8416eaf9f83ca2f8eb78f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 04:41:10.658303 25905 raft_consensus.cc:2258] T 0d6d38199a8d44459fd6e1b1be560162 P ccb6186938e8416eaf9f83ca2f8eb78f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 04:41:10.658710 25905 ts_tablet_manager.cc:1905] T 0d6d38199a8d44459fd6e1b1be560162 P ccb6186938e8416eaf9f83ca2f8eb78f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 04:41:10.658761 25907 ts_tablet_manager.cc:1905] T 0d6d38199a8d44459fd6e1b1be560162 P 72ed48ce93664347aabfaf0f2174f058: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 04:41:10.660540 25905 ts_tablet_manager.cc:1918] T 0d6d38199a8d44459fd6e1b1be560162 P ccb6186938e8416eaf9f83ca2f8eb78f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260810 04:41:10.660542 25907 ts_tablet_manager.cc:1918] T 0d6d38199a8d44459fd6e1b1be560162 P 72ed48ce93664347aabfaf0f2174f058: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260810 04:41:10.660595 25905 log.cc:1196] T 0d6d38199a8d44459fd6e1b1be560162 P ccb6186938e8416eaf9f83ca2f8eb78f: Deleting WAL directory at /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786336863388938-23197-0/minicluster-data/ts-1/wal/wals/0d6d38199a8d44459fd6e1b1be560162
I20260810 04:41:10.660598 25907 log.cc:1196] T 0d6d38199a8d44459fd6e1b1be560162 P 72ed48ce93664347aabfaf0f2174f058: Deleting WAL directory at /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786336863388938-23197-0/minicluster-data/ts-2/wal/wals/0d6d38199a8d44459fd6e1b1be560162
I20260810 04:41:10.660830 25907 ts_tablet_manager.cc:1939] T 0d6d38199a8d44459fd6e1b1be560162 P 72ed48ce93664347aabfaf0f2174f058: Deleting consensus metadata
I20260810 04:41:10.660833 25905 ts_tablet_manager.cc:1939] T 0d6d38199a8d44459fd6e1b1be560162 P ccb6186938e8416eaf9f83ca2f8eb78f: Deleting consensus metadata
I20260810 04:41:10.661180 24975 catalog_manager.cc:4841] TS 72ed48ce93664347aabfaf0f2174f058 (127.22.167.67:40365): tablet 0d6d38199a8d44459fd6e1b1be560162 (table test-table [id=18c845d5d88545e5bd4e976227e42a3a]) successfully deleted
I20260810 04:41:10.661275 24971 catalog_manager.cc:4841] TS ccb6186938e8416eaf9f83ca2f8eb78f (127.22.167.66:43099): tablet 0d6d38199a8d44459fd6e1b1be560162 (table test-table [id=18c845d5d88545e5bd4e976227e42a3a]) successfully deleted
I20260810 04:41:10.661435 25209 tablet_service.cc:1514] Processing DeleteTablet for tablet 0d6d38199a8d44459fd6e1b1be560162 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 04:41:10 UTC) from {username='kudu', principal='kudu/127.22.167.126@KRBTEST.COM'} at 127.0.0.1:36654
I20260810 04:41:10.661531 25906 tablet_replica.cc:330] T 0d6d38199a8d44459fd6e1b1be560162 P c7293e88c49744ccacd6f1cb5a1923b9: stopping tablet replica
I20260810 04:41:10.661590 25906 raft_consensus.cc:2229] T 0d6d38199a8d44459fd6e1b1be560162 P c7293e88c49744ccacd6f1cb5a1923b9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 04:41:10.661628 25906 raft_consensus.cc:2258] T 0d6d38199a8d44459fd6e1b1be560162 P c7293e88c49744ccacd6f1cb5a1923b9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 04:41:10.662423 25906 ts_tablet_manager.cc:1905] T 0d6d38199a8d44459fd6e1b1be560162 P c7293e88c49744ccacd6f1cb5a1923b9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 04:41:10.664150 25906 ts_tablet_manager.cc:1918] T 0d6d38199a8d44459fd6e1b1be560162 P c7293e88c49744ccacd6f1cb5a1923b9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260810 04:41:10.664206 25906 log.cc:1196] T 0d6d38199a8d44459fd6e1b1be560162 P c7293e88c49744ccacd6f1cb5a1923b9: Deleting WAL directory at /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786336863388938-23197-0/minicluster-data/ts-0/wal/wals/0d6d38199a8d44459fd6e1b1be560162
I20260810 04:41:10.664418 25906 ts_tablet_manager.cc:1939] T 0d6d38199a8d44459fd6e1b1be560162 P c7293e88c49744ccacd6f1cb5a1923b9: Deleting consensus metadata
I20260810 04:41:10.664772 24971 catalog_manager.cc:4841] TS c7293e88c49744ccacd6f1cb5a1923b9 (127.22.167.65:41643): tablet 0d6d38199a8d44459fd6e1b1be560162 (table test-table [id=18c845d5d88545e5bd4e976227e42a3a]) successfully deleted
Aug 10 04:41:10 dist-test-slave-bqcl krb5kdc[24924](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786336870, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.22.167.65@KRBTEST.COM
I20260810 04:41:10.676148 25859 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:41:10.662459 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36662 (local address 127.22.167.65:41643)
0810 04:41:10.662499 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0810 04:41:10.662500 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0810 04:41:10.664977 (+ 2477us) server_negotiation.cc:407] Connection header received
0810 04:41:10.665148 (+ 171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:41:10.665149 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:41:10.665160 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:41:10.665217 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0810 04:41:10.665994 (+ 777us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:41:10.666760 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:41:10.668470 (+ 1710us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:41:10.668783 (+ 313us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:41:10.668861 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:41:10.672703 (+ 3842us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 04:41:10.672713 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 04:41:10.672713 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 04:41:10.672721 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0810 04:41:10.674825 (+ 2104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 04:41:10.675221 (+ 396us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 04:41:10.675225 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 04:41:10.675225 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 04:41:10.675298 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 04:41:10.675509 (+ 211us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 04:41:10.675510 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 04:41:10.675511 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 04:41:10.675671 (+ 160us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 04:41:10.675803 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0810 04:41:10.675943 (+ 140us) server_negotiation.cc:299] Negotiation successful
0810 04:41:10.676026 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
W20260810 04:41:10.676774 25326 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.SetFlag from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:36662
I20260810 04:41:10.685101 25076 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:41:10.677689 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53244 (local address 127.22.167.126:37145)
0810 04:41:10.677732 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0810 04:41:10.677734 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 04:41:10.677951 (+ 217us) server_negotiation.cc:407] Connection header received
0810 04:41:10.678014 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:41:10.678015 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:41:10.678027 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:41:10.678086 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0810 04:41:10.678707 (+ 621us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:41:10.679443 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:41:10.680471 (+ 1028us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:41:10.680824 (+ 353us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:41:10.680892 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:41:10.682246 (+ 1354us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 04:41:10.682248 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 04:41:10.682248 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 04:41:10.682256 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0810 04:41:10.683884 (+ 1628us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 04:41:10.684295 (+ 411us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 04:41:10.684296 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 04:41:10.684297 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 04:41:10.684357 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 04:41:10.684567 (+ 210us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 04:41:10.684568 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 04:41:10.684568 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 04:41:10.684723 (+ 155us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 04:41:10.684833 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0810 04:41:10.684934 (+ 101us) server_negotiation.cc:299] Negotiation successful
0810 04:41:10.685013 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
W20260810 04:41:10.685595 24994 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:53244
I20260810 04:41:10.686592 23197 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskemv_AM/build/release/bin/kudu with pid 25113
I20260810 04:41:10.696672 23197 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskemv_AM/build/release/bin/kudu with pid 25349
I20260810 04:41:10.707484 23197 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskemv_AM/build/release/bin/kudu with pid 25606
I20260810 04:41:10.717955 23197 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskemv_AM/build/release/bin/kudu with pid 24946
2026-08-10T04:41:10Z chronyd exiting
[ OK ] SecurityITest.SmokeTestAsAuthorizedUser (2598 ms)
[ RUN ] SecurityITest.TxnSmokeWithDifferentUserTypes
Loading random data
Initializing database '/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786336863388938-23197-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 04:41:10 dist-test-slave-bqcl krb5kdc[25977](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 04:41:10 dist-test-slave-bqcl krb5kdc[25977](info): set up 2 sockets
Aug 10 04:41:10 dist-test-slave-bqcl krb5kdc[25977](info): commencing operation
krb5kdc: starting...
W20260810 04:41:11.896057 23197 mini_kdc.cc:121] Time spent starting KDC: real 1.155s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786336863388938-23197-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786336863388938-23197-0/krb5kdc/test-user.keytab.
Aug 10 04:41:11 dist-test-slave-bqcl krb5kdc[25977](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786336871, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T04:41:11Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T04:41:11Z Disabled control of system clock
WARNING: no policy specified for kudu/127.22.167.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.167.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.167.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786336863388938-23197-0/krb5kdc/kudu_127.22.167.126.keytab.
Entry for principal kudu/127.22.167.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786336863388938-23197-0/krb5kdc/kudu_127.22.167.126.keytab.
I20260810 04:41:12.009949 23197 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskemv_AM/build/release/bin/kudu
/tmp/dist-test-taskemv_AM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786336863388938-23197-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786336863388938-23197-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786336863388938-23197-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786336863388938-23197-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.167.126:37755
--webserver_interface=127.22.167.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786336863388938-23197-0/krb5kdc/kudu_127.22.167.126.keytab
--principal=kudu/127.22.167.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.167.84:37067
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.167.126:37755
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786336863388938-23197-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786336863388938-23197-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786336863388938-23197-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 04:41:12.090487 26004 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 04:41:12.090677 26004 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 04:41:12.090708 26004 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 04:41:12.091799 26004 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 04:41:12.091840 26004 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260810 04:41:12.091857 26004 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 04:41:12.091873 26004 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 04:41:12.091888 26004 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 04:41:12.091913 26004 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 04:41:12.093366 26004 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.167.84:37067
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786336863388938-23197-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786336863388938-23197-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.22.167.126:37755
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786336863388938-23197-0/krb5kdc/kudu_127.22.167.126.keytab
--principal=kudu/127.22.167.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.167.126:37755
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786336863388938-23197-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.167.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26004
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786336863388938-23197-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 03:54:01 UTC on ccycloud.quasar-icijmt.root.comops.site
build id 4577009
I20260810 04:41:12.093598 26004 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 04:41:12.093801 26004 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 04:41:12.098367 26016 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 04:41:12.098362 26014 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 04:41:12.098420 26011 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 04:41:12.098443 26004 server_base.cc:1034] running on GCE node
I20260810 04:41:12.098834 26004 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 04:41:12.099098 26004 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 04:41:12.100256 26004 hybrid_clock.cc:648] HybridClock initialized: now 1786336872100243 us; error 33 us; skew 500 ppm
Aug 10 04:41:12 dist-test-slave-bqcl krb5kdc[25977](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786336872, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.167.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 04:41:12.102725 26004 init.cc:377] Logged in from keytab as kudu/127.22.167.126@KRBTEST.COM (short username kudu)
I20260810 04:41:12.103098 26004 webserver.cc:460] Webserver started at http://127.22.167.126:37273/ using document root <none> and password file <none>
I20260810 04:41:12.103291 26004 fs_manager.cc:359] Metadata directory not provided
I20260810 04:41:12.103328 26004 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 04:41:12.103417 26004 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 04:41:12.104425 26004 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786336863388938-23197-0/minicluster-data/master-0/data/instance:
uuid: "5352e0da235343aaa3a83857cadf08c7"
format_stamp: "Formatted at 2026-08-10 04:41:12 on dist-test-slave-bqcl"
server_key: "bcaf24cdd900694486c4cb0af2e2bce4"
server_key_iv: "79fbb2318a2f9d6a5bae6fe170b4b28d"
server_key_version: "encryptionkey@0"
I20260810 04:41:12.104722 26004 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786336863388938-23197-0/minicluster-data/master-0/wal/instance:
uuid: "5352e0da235343aaa3a83857cadf08c7"
format_stamp: "Formatted at 2026-08-10 04:41:12 on dist-test-slave-bqcl"
server_key: "bcaf24cdd900694486c4cb0af2e2bce4"
server_key_iv: "79fbb2318a2f9d6a5bae6fe170b4b28d"
server_key_version: "encryptionkey@0"
I20260810 04:41:12.105940 26004 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260810 04:41:12.106704 26023 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 04:41:12.106855 26004 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260810 04:41:12.106931 26004 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786336863388938-23197-0/minicluster-data/master-0/data,/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786336863388938-23197-0/minicluster-data/master-0/wal
uuid: "5352e0da235343aaa3a83857cadf08c7"
format_stamp: "Formatted at 2026-08-10 04:41:12 on dist-test-slave-bqcl"
server_key: "bcaf24cdd900694486c4cb0af2e2bce4"
server_key_iv: "79fbb2318a2f9d6a5bae6fe170b4b28d"
server_key_version: "encryptionkey@0"
I20260810 04:41:12.107003 26004 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786336863388938-23197-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786336863388938-23197-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786336863388938-23197-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 04:41:12.127081 26004 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 04:41:12.129215 26004 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 04:41:12.129338 26004 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 04:41:12.133761 26004 rpc_server.cc:307] RPC server started. Bound to: 127.22.167.126:37755
I20260810 04:41:12.133845 26118 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.167.126:37755 every 8 connection(s)
I20260810 04:41:12.134153 26004 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786336863388938-23197-0/minicluster-data/master-0/data/info.pb
I20260810 04:41:12.134544 23197 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskemv_AM/build/release/bin/kudu as pid 26004
I20260810 04:41:12.134641 23197 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786336863388938-23197-0/minicluster-data/master-0/wal/instance
I20260810 04:41:12.134809 23197 external_mini_cluster.cc:1428] Setting key 96850ee7f32a436eaceee120d8c896ce
I20260810 04:41:12.134928 26119 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 04:41:12.137665 26119 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5352e0da235343aaa3a83857cadf08c7: Bootstrap starting.
I20260810 04:41:12.138281 26119 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5352e0da235343aaa3a83857cadf08c7: Neither blocks nor log segments found. Creating new log.
I20260810 04:41:12.138553 26119 log.cc:824] T 00000000000000000000000000000000 P 5352e0da235343aaa3a83857cadf08c7: Log is configured to *not* fsync() on all Append() calls
I20260810 04:41:12.139166 26119 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5352e0da235343aaa3a83857cadf08c7: No bootstrap required, opened a new log
Aug 10 04:41:12 dist-test-slave-bqcl krb5kdc[25977](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786336871, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.167.126@KRBTEST.COM
I20260810 04:41:12.141211 26119 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5352e0da235343aaa3a83857cadf08c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5352e0da235343aaa3a83857cadf08c7" member_type: VOTER last_known_addr { host: "127.22.167.126" port: 37755 } }
I20260810 04:41:12.141348 26119 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5352e0da235343aaa3a83857cadf08c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 04:41:12.141371 26119 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5352e0da235343aaa3a83857cadf08c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5352e0da235343aaa3a83857cadf08c7, State: Initialized, Role: FOLLOWER
I20260810 04:41:12.141505 26119 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5352e0da235343aaa3a83857cadf08c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5352e0da235343aaa3a83857cadf08c7" member_type: VOTER last_known_addr { host: "127.22.167.126" port: 37755 } }
I20260810 04:41:12.141573 26119 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5352e0da235343aaa3a83857cadf08c7 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 04:41:12.141597 26119 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5352e0da235343aaa3a83857cadf08c7 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 04:41:12.141639 26119 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5352e0da235343aaa3a83857cadf08c7 [term 0 FOLLOWER]: Advancing to term 1
I20260810 04:41:12.142174 26119 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5352e0da235343aaa3a83857cadf08c7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5352e0da235343aaa3a83857cadf08c7" member_type: VOTER last_known_addr { host: "127.22.167.126" port: 37755 } }
I20260810 04:41:12.142275 26119 leader_election.cc:302] T 00000000000000000000000000000000 P 5352e0da235343aaa3a83857cadf08c7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5352e0da235343aaa3a83857cadf08c7; no voters:
I20260810 04:41:12.142444 26119 leader_election.cc:288] T 00000000000000000000000000000000 P 5352e0da235343aaa3a83857cadf08c7 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 04:41:12.142504 26131 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5352e0da235343aaa3a83857cadf08c7 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 04:41:12.142643 26131 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5352e0da235343aaa3a83857cadf08c7 [term 1 LEADER]: Becoming Leader. State: Replica: 5352e0da235343aaa3a83857cadf08c7, State: Running, Role: LEADER
I20260810 04:41:12.142715 26119 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5352e0da235343aaa3a83857cadf08c7 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 04:41:12.142750 26131 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5352e0da235343aaa3a83857cadf08c7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5352e0da235343aaa3a83857cadf08c7" member_type: VOTER last_known_addr { host: "127.22.167.126" port: 37755 } }
I20260810 04:41:12.143085 26131 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5352e0da235343aaa3a83857cadf08c7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5352e0da235343aaa3a83857cadf08c7. Latest consensus state: current_term: 1 leader_uuid: "5352e0da235343aaa3a83857cadf08c7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5352e0da235343aaa3a83857cadf08c7" member_type: VOTER last_known_addr { host: "127.22.167.126" port: 37755 } } }
I20260810 04:41:12.143164 26131 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5352e0da235343aaa3a83857cadf08c7 [sys.catalog]: This master's current role is: LEADER
I20260810 04:41:12.143316 26132 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5352e0da235343aaa3a83857cadf08c7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5352e0da235343aaa3a83857cadf08c7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5352e0da235343aaa3a83857cadf08c7" member_type: VOTER last_known_addr { host: "127.22.167.126" port: 37755 } } }
I20260810 04:41:12.143394 26132 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5352e0da235343aaa3a83857cadf08c7 [sys.catalog]: This master's current role is: LEADER
W20260810 04:41:12.144533 26160 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 5352e0da235343aaa3a83857cadf08c7: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260810 04:41:12.144588 26160 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260810 04:41:12.144819 26139 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 04:41:12.144994 26139 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 04:41:12.145768 26125 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:41:12.135307 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41376 (local address 127.22.167.126:37755)
0810 04:41:12.135492 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0810 04:41:12.135501 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 04:41:12.135526 (+ 25us) server_negotiation.cc:407] Connection header received
0810 04:41:12.135976 (+ 450us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:41:12.135982 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:41:12.136029 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:41:12.136149 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0810 04:41:12.136750 (+ 601us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:41:12.137633 (+ 883us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:41:12.138465 (+ 832us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:41:12.138925 (+ 460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:41:12.139032 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:41:12.141347 (+ 2315us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 04:41:12.141351 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 04:41:12.141352 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 04:41:12.141364 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0810 04:41:12.143265 (+ 1901us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 04:41:12.143699 (+ 434us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 04:41:12.143702 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 04:41:12.143702 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 04:41:12.143814 (+ 112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 04:41:12.144046 (+ 232us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 04:41:12.144046 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 04:41:12.144047 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 04:41:12.144422 (+ 375us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 04:41:12.144561 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0810 04:41:12.145522 (+ 961us) server_negotiation.cc:299] Negotiation successful
0810 04:41:12.145630 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":140,"thread_start_us":98,"threads_started":1}
I20260810 04:41:12.146574 26139 catalog_manager.cc:1347] Generated new cluster ID: 1514164e31bf46b399a0044c7b9a7388
I20260810 04:41:12.146623 26139 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 04:41:12.160019 26139 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 04:41:12.160712 26139 catalog_manager.cc:1504] Loading token signing keys...
I20260810 04:41:12.177209 26139 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5352e0da235343aaa3a83857cadf08c7: Generated new TSK 0
I20260810 04:41:12.177397 26139 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.22.167.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.167.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.167.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786336863388938-23197-0/krb5kdc/kudu_127.22.167.65.keytab.
Entry for principal kudu/127.22.167.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786336863388938-23197-0/krb5kdc/kudu_127.22.167.65.keytab.
I20260810 04:41:12.207018 23197 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskemv_AM/build/release/bin/kudu
/tmp/dist-test-taskemv_AM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786336863388938-23197-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786336863388938-23197-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786336863388938-23197-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786336863388938-23197-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.167.65:0
--local_ip_for_outbound_sockets=127.22.167.65
--webserver_interface=127.22.167.65
--webserver_port=0
--tserver_master_addrs=127.22.167.126:37755
--keytab_file=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786336863388938-23197-0/krb5kdc/kudu_127.22.167.65.keytab
--principal=kudu/127.22.167.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.167.84:37067
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786336863388938-23197-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786336863388938-23197-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786336863388938-23197-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 04:41:12.311661 26169 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 04:41:12.311872 26169 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 04:41:12.311906 26169 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 04:41:12.313017 26169 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 04:41:12.313057 26169 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 04:41:12.313086 26169 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260810 04:41:12.313102 26169 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260810 04:41:12.313124 26169 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.167.65
I20260810 04:41:12.314582 26169 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.167.84:37067
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786336863388938-23197-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786336863388938-23197-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786336863388938-23197-0/krb5kdc/kudu_127.22.167.65.keytab
--principal=kudu/127.22.167.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.167.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786336863388938-23197-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.22.167.65
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.22.167.126:37755
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26169
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.167.65
--log_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786336863388938-23197-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 03:54:01 UTC on ccycloud.quasar-icijmt.root.comops.site
build id 4577009
I20260810 04:41:12.314814 26169 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 04:41:12.315032 26169 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 04:41:12.319875 26178 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 04:41:12.319900 26176 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 04:41:12.320055 26169 server_base.cc:1034] running on GCE node
W20260810 04:41:12.319936 26180 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 04:41:12.320298 26169 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 04:41:12.320530 26169 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 04:41:12.321677 26169 hybrid_clock.cc:648] HybridClock initialized: now 1786336872321661 us; error 34 us; skew 500 ppm
Aug 10 04:41:12 dist-test-slave-bqcl krb5kdc[25977](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786336872, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.167.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 04:41:12.324090 26169 init.cc:377] Logged in from keytab as kudu/127.22.167.65@KRBTEST.COM (short username kudu)
I20260810 04:41:12.324532 26169 webserver.cc:460] Webserver started at http://127.22.167.65:46049/ using document root <none> and password file <none>
I20260810 04:41:12.324735 26169 fs_manager.cc:359] Metadata directory not provided
I20260810 04:41:12.324785 26169 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 04:41:12.324913 26169 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 04:41:12.325971 26169 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786336863388938-23197-0/minicluster-data/ts-0/data/instance:
uuid: "3b3f370bee5b43f4882dbdcfbabc8dad"
format_stamp: "Formatted at 2026-08-10 04:41:12 on dist-test-slave-bqcl"
server_key: "06afcbefa36ac71bb2574697edd95361"
server_key_iv: "1a67a6fb1e273061fea7651c7fe87aba"
server_key_version: "encryptionkey@0"
I20260810 04:41:12.326328 26169 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786336863388938-23197-0/minicluster-data/ts-0/wal/instance:
uuid: "3b3f370bee5b43f4882dbdcfbabc8dad"
format_stamp: "Formatted at 2026-08-10 04:41:12 on dist-test-slave-bqcl"
server_key: "06afcbefa36ac71bb2574697edd95361"
server_key_iv: "1a67a6fb1e273061fea7651c7fe87aba"
server_key_version: "encryptionkey@0"
I20260810 04:41:12.327551 26169 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260810 04:41:12.328210 26190 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 04:41:12.328363 26169 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260810 04:41:12.328447 26169 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786336863388938-23197-0/minicluster-data/ts-0/data,/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786336863388938-23197-0/minicluster-data/ts-0/wal
uuid: "3b3f370bee5b43f4882dbdcfbabc8dad"
format_stamp: "Formatted at 2026-08-10 04:41:12 on dist-test-slave-bqcl"
server_key: "06afcbefa36ac71bb2574697edd95361"
server_key_iv: "1a67a6fb1e273061fea7651c7fe87aba"
server_key_version: "encryptionkey@0"
I20260810 04:41:12.328508 26169 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786336863388938-23197-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786336863388938-23197-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786336863388938-23197-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 04:41:12.345484 26169 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 04:41:12.347487 26169 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 04:41:12.347728 26169 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 04:41:12.348315 26169 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 04:41:12.348361 26169 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 04:41:12.348392 26169 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 04:41:12.348423 26169 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 10 04:41:12 dist-test-slave-bqcl krb5kdc[25977](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786336872, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.167.65@KRBTEST.COM for kudu/127.22.167.126@KRBTEST.COM
I20260810 04:41:12.355259 26169 rpc_server.cc:307] RPC server started. Bound to: 127.22.167.65:46669
I20260810 04:41:12.355361 26408 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.167.65:46669 every 8 connection(s)
I20260810 04:41:12.355679 26169 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786336863388938-23197-0/minicluster-data/ts-0/data/info.pb
I20260810 04:41:12.357836 26125 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:41:12.348746 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.167.65:49241 (local address 127.22.167.126:37755)
0810 04:41:12.348796 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0810 04:41:12.348798 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 04:41:12.349004 (+ 206us) server_negotiation.cc:407] Connection header received
0810 04:41:12.349563 (+ 559us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:41:12.349565 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:41:12.349580 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:41:12.349649 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0810 04:41:12.350414 (+ 765us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:41:12.350989 (+ 575us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:41:12.351960 (+ 971us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:41:12.352183 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:41:12.352239 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:41:12.355107 (+ 2868us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 04:41:12.355109 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 04:41:12.355109 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 04:41:12.355117 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0810 04:41:12.356486 (+ 1369us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 04:41:12.357001 (+ 515us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 04:41:12.357002 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 04:41:12.357003 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 04:41:12.357071 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 04:41:12.357273 (+ 202us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 04:41:12.357274 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 04:41:12.357274 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 04:41:12.357380 (+ 106us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 04:41:12.357429 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0810 04:41:12.357698 (+ 269us) server_negotiation.cc:299] Negotiation successful
0810 04:41:12.357770 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260810 04:41:12.357990 26210 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:41:12.348719 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.126:37755 (local address 127.22.167.65:49241)
0810 04:41:12.348888 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0810 04:41:12.348925 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0810 04:41:12.349444 (+ 519us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 04:41:12.349751 (+ 307us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 04:41:12.349754 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 04:41:12.350083 (+ 329us) client_negotiation.cc:189] Negotiated authn=SASL
0810 04:41:12.350319 (+ 236us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:41:12.350324 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:41:12.351074 (+ 750us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:41:12.351075 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:41:12.351876 (+ 801us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:41:12.351878 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:41:12.352254 (+ 376us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:41:12.352255 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:41:12.352382 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:41:12.352883 (+ 501us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 04:41:12.352893 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0810 04:41:12.354977 (+ 2084us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 04:41:12.356570 (+ 1593us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 04:41:12.356577 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 04:41:12.356579 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 04:41:12.356911 (+ 332us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 04:41:12.357129 (+ 218us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 04:41:12.357130 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 04:41:12.357130 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 04:41:12.357229 (+ 99us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 04:41:12.357452 (+ 223us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 04:41:12.357458 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 04:41:12.357586 (+ 128us) client_negotiation.cc:769] Sending connection context
0810 04:41:12.357681 (+ 95us) client_negotiation.cc:240] Negotiation successful
0810 04:41:12.357804 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":84,"threads_started":1}
I20260810 04:41:12.358518 26410 heartbeater.cc:347] Connected to a master server at 127.22.167.126:37755
I20260810 04:41:12.358603 26410 heartbeater.cc:464] Registering TS with master...
I20260810 04:41:12.358774 26410 heartbeater.cc:511] Master 127.22.167.126:37755 requested a full tablet report, sending...
I20260810 04:41:12.359318 26051 ts_manager.cc:194] Registered new tserver with Master: 3b3f370bee5b43f4882dbdcfbabc8dad (127.22.167.65:46669)
I20260810 04:41:12.360179 26051 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.167.65@KRBTEST.COM'} at 127.22.167.65:49241
I20260810 04:41:12.362774 23197 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskemv_AM/build/release/bin/kudu as pid 26169
I20260810 04:41:12.362855 23197 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786336863388938-23197-0/minicluster-data/ts-0/wal/instance
I20260810 04:41:12.363030 23197 external_mini_cluster.cc:1428] Setting key 2c85e1c58940ed31987d6cbdc7f3794b
I20260810 04:41:12.364476 26125 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:41:12.359535 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.167.65:43653 (local address 127.22.167.126:37755)
0810 04:41:12.359572 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0810 04:41:12.359574 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 04:41:12.359625 (+ 51us) server_negotiation.cc:407] Connection header received
0810 04:41:12.359684 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:41:12.359685 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:41:12.359692 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:41:12.359734 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0810 04:41:12.360026 (+ 292us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:41:12.360516 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:41:12.361203 (+ 687us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:41:12.361379 (+ 176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:41:12.361432 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:41:12.362178 (+ 746us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 04:41:12.362179 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 04:41:12.362179 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 04:41:12.362184 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0810 04:41:12.363458 (+ 1274us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 04:41:12.363800 (+ 342us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 04:41:12.363801 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 04:41:12.363802 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 04:41:12.363854 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 04:41:12.364022 (+ 168us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 04:41:12.364023 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 04:41:12.364023 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 04:41:12.364122 (+ 99us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 04:41:12.364176 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0810 04:41:12.364316 (+ 140us) server_negotiation.cc:299] Negotiation successful
0810 04:41:12.364387 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260810 04:41:12.364490 26417 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:41:12.359450 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.126:37755 (local address 127.22.167.65:43653)
0810 04:41:12.359571 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0810 04:41:12.359581 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0810 04:41:12.359639 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 04:41:12.359739 (+ 100us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 04:41:12.359740 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 04:41:12.359921 (+ 181us) client_negotiation.cc:189] Negotiated authn=SASL
0810 04:41:12.359973 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:41:12.359975 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:41:12.360604 (+ 629us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:41:12.360604 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:41:12.361133 (+ 529us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:41:12.361135 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:41:12.361436 (+ 301us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:41:12.361436 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:41:12.361487 (+ 51us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:41:12.361820 (+ 333us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 04:41:12.361824 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0810 04:41:12.362089 (+ 265us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 04:41:12.363528 (+ 1439us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 04:41:12.363529 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 04:41:12.363529 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 04:41:12.363736 (+ 207us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 04:41:12.363910 (+ 174us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 04:41:12.363910 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 04:41:12.363911 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 04:41:12.363971 (+ 60us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 04:41:12.364182 (+ 211us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 04:41:12.364183 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 04:41:12.364229 (+ 46us) client_negotiation.cc:769] Sending connection context
0810 04:41:12.364287 (+ 58us) client_negotiation.cc:240] Negotiation successful
0810 04:41:12.364388 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":95,"thread_start_us":65,"threads_started":1}
WARNING: no policy specified for kudu/127.22.167.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.167.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.167.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786336863388938-23197-0/krb5kdc/kudu_127.22.167.66.keytab.
Entry for principal kudu/127.22.167.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786336863388938-23197-0/krb5kdc/kudu_127.22.167.66.keytab.
I20260810 04:41:12.387135 23197 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskemv_AM/build/release/bin/kudu
/tmp/dist-test-taskemv_AM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786336863388938-23197-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786336863388938-23197-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786336863388938-23197-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786336863388938-23197-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.167.66:0
--local_ip_for_outbound_sockets=127.22.167.66
--webserver_interface=127.22.167.66
--webserver_port=0
--tserver_master_addrs=127.22.167.126:37755
--keytab_file=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786336863388938-23197-0/krb5kdc/kudu_127.22.167.66.keytab
--principal=kudu/127.22.167.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.167.84:37067
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786336863388938-23197-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786336863388938-23197-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786336863388938-23197-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 04:41:12.472527 26421 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 04:41:12.472743 26421 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 04:41:12.472766 26421 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 04:41:12.473874 26421 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 04:41:12.473909 26421 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 04:41:12.473927 26421 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260810 04:41:12.473940 26421 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260810 04:41:12.473956 26421 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.167.66
I20260810 04:41:12.475580 26421 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.167.84:37067
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786336863388938-23197-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786336863388938-23197-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786336863388938-23197-0/krb5kdc/kudu_127.22.167.66.keytab
--principal=kudu/127.22.167.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.167.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786336863388938-23197-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.22.167.66
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.22.167.126:37755
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26421
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.167.66
--log_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786336863388938-23197-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 03:54:01 UTC on ccycloud.quasar-icijmt.root.comops.site
build id 4577009
I20260810 04:41:12.475785 26421 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 04:41:12.475962 26421 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 04:41:12.481279 26428 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 04:41:12.481295 26427 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 04:41:12.481334 26432 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 04:41:12.481359 26421 server_base.cc:1034] running on GCE node
I20260810 04:41:12.481748 26421 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 04:41:12.481987 26421 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 04:41:12.483124 26421 hybrid_clock.cc:648] HybridClock initialized: now 1786336872483105 us; error 38 us; skew 500 ppm
Aug 10 04:41:12 dist-test-slave-bqcl krb5kdc[25977](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786336872, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.167.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 04:41:12.485525 26421 init.cc:377] Logged in from keytab as kudu/127.22.167.66@KRBTEST.COM (short username kudu)
I20260810 04:41:12.485958 26421 webserver.cc:460] Webserver started at http://127.22.167.66:46203/ using document root <none> and password file <none>
I20260810 04:41:12.486198 26421 fs_manager.cc:359] Metadata directory not provided
I20260810 04:41:12.486244 26421 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 04:41:12.486368 26421 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 04:41:12.487413 26421 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786336863388938-23197-0/minicluster-data/ts-1/data/instance:
uuid: "3202926ff8c141af80ac121604f0dce3"
format_stamp: "Formatted at 2026-08-10 04:41:12 on dist-test-slave-bqcl"
server_key: "3e60c0eb987e78f597c35418534ed1d1"
server_key_iv: "e12fe9a541706ff67ccfb527536c25d8"
server_key_version: "encryptionkey@0"
I20260810 04:41:12.487746 26421 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786336863388938-23197-0/minicluster-data/ts-1/wal/instance:
uuid: "3202926ff8c141af80ac121604f0dce3"
format_stamp: "Formatted at 2026-08-10 04:41:12 on dist-test-slave-bqcl"
server_key: "3e60c0eb987e78f597c35418534ed1d1"
server_key_iv: "e12fe9a541706ff67ccfb527536c25d8"
server_key_version: "encryptionkey@0"
I20260810 04:41:12.489141 26421 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260810 04:41:12.489876 26445 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 04:41:12.490015 26421 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260810 04:41:12.490097 26421 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786336863388938-23197-0/minicluster-data/ts-1/data,/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786336863388938-23197-0/minicluster-data/ts-1/wal
uuid: "3202926ff8c141af80ac121604f0dce3"
format_stamp: "Formatted at 2026-08-10 04:41:12 on dist-test-slave-bqcl"
server_key: "3e60c0eb987e78f597c35418534ed1d1"
server_key_iv: "e12fe9a541706ff67ccfb527536c25d8"
server_key_version: "encryptionkey@0"
I20260810 04:41:12.490185 26421 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786336863388938-23197-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786336863388938-23197-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786336863388938-23197-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 04:41:12.507443 26421 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 04:41:12.509970 26421 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 04:41:12.510195 26421 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 04:41:12.511056 26421 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 04:41:12.511097 26421 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 04:41:12.511127 26421 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 04:41:12.511159 26421 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 10 04:41:12 dist-test-slave-bqcl krb5kdc[25977](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786336872, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.167.66@KRBTEST.COM for kudu/127.22.167.126@KRBTEST.COM
I20260810 04:41:12.520923 26125 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:41:12.511586 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.167.66:53995 (local address 127.22.167.126:37755)
0810 04:41:12.511635 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0810 04:41:12.511638 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 04:41:12.512089 (+ 451us) server_negotiation.cc:407] Connection header received
0810 04:41:12.512834 (+ 745us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:41:12.512835 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:41:12.512849 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:41:12.512931 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0810 04:41:12.513661 (+ 730us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:41:12.514466 (+ 805us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:41:12.515709 (+ 1243us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:41:12.515907 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:41:12.515961 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:41:12.517971 (+ 2010us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 04:41:12.517973 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 04:41:12.517973 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 04:41:12.517981 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0810 04:41:12.519594 (+ 1613us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 04:41:12.520024 (+ 430us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 04:41:12.520026 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 04:41:12.520026 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 04:41:12.520088 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 04:41:12.520314 (+ 226us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 04:41:12.520315 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 04:41:12.520316 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 04:41:12.520429 (+ 113us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 04:41:12.520486 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0810 04:41:12.520747 (+ 261us) server_negotiation.cc:299] Negotiation successful
0810 04:41:12.520834 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260810 04:41:12.520931 26421 rpc_server.cc:307] RPC server started. Bound to: 127.22.167.66:39673
I20260810 04:41:12.521016 26651 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.167.66:39673 every 8 connection(s)
I20260810 04:41:12.521257 26461 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:41:12.511597 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.126:37755 (local address 127.22.167.66:53995)
0810 04:41:12.511926 (+ 329us) negotiation.cc:107] Waiting for socket to connect
0810 04:41:12.511966 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0810 04:41:12.512719 (+ 753us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 04:41:12.512983 (+ 264us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 04:41:12.512986 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 04:41:12.513257 (+ 271us) client_negotiation.cc:189] Negotiated authn=SASL
0810 04:41:12.513567 (+ 310us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:41:12.513573 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:41:12.514566 (+ 993us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:41:12.514567 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:41:12.515615 (+ 1048us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:41:12.515618 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:41:12.515974 (+ 356us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:41:12.515975 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:41:12.516121 (+ 146us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:41:12.516577 (+ 456us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 04:41:12.516586 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0810 04:41:12.517885 (+ 1299us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 04:41:12.519675 (+ 1790us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 04:41:12.519682 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 04:41:12.519685 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 04:41:12.519946 (+ 261us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 04:41:12.520140 (+ 194us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 04:41:12.520141 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 04:41:12.520141 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 04:41:12.520257 (+ 116us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 04:41:12.520482 (+ 225us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 04:41:12.520489 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 04:41:12.520628 (+ 139us) client_negotiation.cc:769] Sending connection context
0810 04:41:12.520716 (+ 88us) client_negotiation.cc:240] Negotiation successful
0810 04:41:12.520870 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":261,"thread_start_us":188,"threads_started":1}
I20260810 04:41:12.521301 26421 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786336863388938-23197-0/minicluster-data/ts-1/data/info.pb
I20260810 04:41:12.521847 23197 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskemv_AM/build/release/bin/kudu as pid 26421
I20260810 04:41:12.521930 23197 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786336863388938-23197-0/minicluster-data/ts-1/wal/instance
I20260810 04:41:12.522097 23197 external_mini_cluster.cc:1428] Setting key 144aeac1b25452dfbde97e327964fbfb
I20260810 04:41:12.522398 26652 heartbeater.cc:347] Connected to a master server at 127.22.167.126:37755
I20260810 04:41:12.522759 26652 heartbeater.cc:464] Registering TS with master...
I20260810 04:41:12.522938 26652 heartbeater.cc:511] Master 127.22.167.126:37755 requested a full tablet report, sending...
I20260810 04:41:12.523298 26051 ts_manager.cc:194] Registered new tserver with Master: 3202926ff8c141af80ac121604f0dce3 (127.22.167.66:39673)
I20260810 04:41:12.523738 26051 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.167.66@KRBTEST.COM'} at 127.22.167.66:53995
I20260810 04:41:12.529021 26125 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:41:12.522828 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.167.66:58167 (local address 127.22.167.126:37755)
0810 04:41:12.522861 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0810 04:41:12.522862 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0810 04:41:12.523177 (+ 315us) server_negotiation.cc:407] Connection header received
0810 04:41:12.523239 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:41:12.523240 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:41:12.523250 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:41:12.523294 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0810 04:41:12.523906 (+ 612us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:41:12.524413 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:41:12.525499 (+ 1086us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:41:12.525689 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:41:12.525737 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:41:12.526740 (+ 1003us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 04:41:12.526741 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 04:41:12.526741 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 04:41:12.526747 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0810 04:41:12.528017 (+ 1270us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 04:41:12.528374 (+ 357us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 04:41:12.528375 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 04:41:12.528375 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 04:41:12.528419 (+ 44us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 04:41:12.528587 (+ 168us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 04:41:12.528587 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 04:41:12.528587 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 04:41:12.528714 (+ 127us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 04:41:12.528764 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0810 04:41:12.528908 (+ 144us) server_negotiation.cc:299] Negotiation successful
0810 04:41:12.528963 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260810 04:41:12.529080 26661 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:41:12.522893 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.126:37755 (local address 127.22.167.66:58167)
0810 04:41:12.523118 (+ 225us) negotiation.cc:107] Waiting for socket to connect
0810 04:41:12.523128 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0810 04:41:12.523188 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 04:41:12.523296 (+ 108us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 04:41:12.523296 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 04:41:12.523628 (+ 332us) client_negotiation.cc:189] Negotiated authn=SASL
0810 04:41:12.523809 (+ 181us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:41:12.523812 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:41:12.524664 (+ 852us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:41:12.524665 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:41:12.525405 (+ 740us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:41:12.525408 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:41:12.525751 (+ 343us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:41:12.525752 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:41:12.525918 (+ 166us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:41:12.526331 (+ 413us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 04:41:12.526336 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0810 04:41:12.526665 (+ 329us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 04:41:12.528095 (+ 1430us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 04:41:12.528096 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 04:41:12.528096 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 04:41:12.528306 (+ 210us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 04:41:12.528458 (+ 152us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 04:41:12.528459 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 04:41:12.528459 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 04:41:12.528520 (+ 61us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 04:41:12.528772 (+ 252us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 04:41:12.528773 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 04:41:12.528821 (+ 48us) client_negotiation.cc:769] Sending connection context
0810 04:41:12.528892 (+ 71us) client_negotiation.cc:240] Negotiation successful
0810 04:41:12.528987 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":178,"thread_start_us":65,"threads_started":1}
WARNING: no policy specified for kudu/127.22.167.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.167.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.167.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786336863388938-23197-0/krb5kdc/kudu_127.22.167.67.keytab.
Entry for principal kudu/127.22.167.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786336863388938-23197-0/krb5kdc/kudu_127.22.167.67.keytab.
I20260810 04:41:12.548776 23197 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskemv_AM/build/release/bin/kudu
/tmp/dist-test-taskemv_AM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786336863388938-23197-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786336863388938-23197-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786336863388938-23197-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786336863388938-23197-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.167.67:0
--local_ip_for_outbound_sockets=127.22.167.67
--webserver_interface=127.22.167.67
--webserver_port=0
--tserver_master_addrs=127.22.167.126:37755
--keytab_file=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786336863388938-23197-0/krb5kdc/kudu_127.22.167.67.keytab
--principal=kudu/127.22.167.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.167.84:37067
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786336863388938-23197-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786336863388938-23197-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786336863388938-23197-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 04:41:12.636757 26664 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 04:41:12.636956 26664 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 04:41:12.636981 26664 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 04:41:12.638067 26664 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 04:41:12.638098 26664 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 04:41:12.638116 26664 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260810 04:41:12.638151 26664 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260810 04:41:12.638182 26664 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.167.67
I20260810 04:41:12.639688 26664 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.167.84:37067
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786336863388938-23197-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786336863388938-23197-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786336863388938-23197-0/krb5kdc/kudu_127.22.167.67.keytab
--principal=kudu/127.22.167.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.167.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786336863388938-23197-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.22.167.67
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.22.167.126:37755
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26664
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.167.67
--log_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786336863388938-23197-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 03:54:01 UTC on ccycloud.quasar-icijmt.root.comops.site
build id 4577009
I20260810 04:41:12.639873 26664 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 04:41:12.640062 26664 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 04:41:12.645345 26676 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 04:41:12.645368 26672 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 04:41:12.645617 26664 server_base.cc:1034] running on GCE node
W20260810 04:41:12.645691 26681 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 04:41:12.645890 26664 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 04:41:12.646095 26664 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 04:41:12.647279 26664 hybrid_clock.cc:648] HybridClock initialized: now 1786336872647265 us; error 29 us; skew 500 ppm
Aug 10 04:41:12 dist-test-slave-bqcl krb5kdc[25977](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786336872, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.167.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 04:41:12.649748 26664 init.cc:377] Logged in from keytab as kudu/127.22.167.67@KRBTEST.COM (short username kudu)
I20260810 04:41:12.650214 26664 webserver.cc:460] Webserver started at http://127.22.167.67:45385/ using document root <none> and password file <none>
I20260810 04:41:12.650432 26664 fs_manager.cc:359] Metadata directory not provided
I20260810 04:41:12.650480 26664 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 04:41:12.650619 26664 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 04:41:12.651713 26664 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786336863388938-23197-0/minicluster-data/ts-2/data/instance:
uuid: "086ccfa1dd66486bbbcc563857db108b"
format_stamp: "Formatted at 2026-08-10 04:41:12 on dist-test-slave-bqcl"
server_key: "e8afc388d9bf2ab4bbf2f96a279e5821"
server_key_iv: "d59fec8f26b569ab66167364c368bca4"
server_key_version: "encryptionkey@0"
I20260810 04:41:12.652091 26664 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786336863388938-23197-0/minicluster-data/ts-2/wal/instance:
uuid: "086ccfa1dd66486bbbcc563857db108b"
format_stamp: "Formatted at 2026-08-10 04:41:12 on dist-test-slave-bqcl"
server_key: "e8afc388d9bf2ab4bbf2f96a279e5821"
server_key_iv: "d59fec8f26b569ab66167364c368bca4"
server_key_version: "encryptionkey@0"
I20260810 04:41:12.653383 26664 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260810 04:41:12.654191 26693 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 04:41:12.654354 26664 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 04:41:12.654439 26664 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786336863388938-23197-0/minicluster-data/ts-2/data,/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786336863388938-23197-0/minicluster-data/ts-2/wal
uuid: "086ccfa1dd66486bbbcc563857db108b"
format_stamp: "Formatted at 2026-08-10 04:41:12 on dist-test-slave-bqcl"
server_key: "e8afc388d9bf2ab4bbf2f96a279e5821"
server_key_iv: "d59fec8f26b569ab66167364c368bca4"
server_key_version: "encryptionkey@0"
I20260810 04:41:12.654506 26664 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786336863388938-23197-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786336863388938-23197-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786336863388938-23197-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 04:41:12.663902 26664 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 04:41:12.665802 26664 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 04:41:12.665937 26664 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 04:41:12.666651 26664 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 04:41:12.666697 26664 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 04:41:12.666726 26664 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 04:41:12.666752 26664 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 10 04:41:12 dist-test-slave-bqcl krb5kdc[25977](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786336872, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.167.67@KRBTEST.COM for kudu/127.22.167.126@KRBTEST.COM
I20260810 04:41:12.676222 26125 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:41:12.667008 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.167.67:48123 (local address 127.22.167.126:37755)
0810 04:41:12.667062 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0810 04:41:12.667064 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 04:41:12.667278 (+ 214us) server_negotiation.cc:407] Connection header received
0810 04:41:12.667820 (+ 542us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:41:12.667822 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:41:12.667836 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:41:12.667916 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0810 04:41:12.668528 (+ 612us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:41:12.669310 (+ 782us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:41:12.670490 (+ 1180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:41:12.670698 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:41:12.670756 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:41:12.673372 (+ 2616us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 04:41:12.673374 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 04:41:12.673375 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 04:41:12.673383 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0810 04:41:12.674781 (+ 1398us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 04:41:12.675264 (+ 483us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 04:41:12.675265 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 04:41:12.675265 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 04:41:12.675327 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 04:41:12.675588 (+ 261us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 04:41:12.675589 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 04:41:12.675589 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 04:41:12.675723 (+ 134us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 04:41:12.675786 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0810 04:41:12.676039 (+ 253us) server_negotiation.cc:299] Negotiation successful
0810 04:41:12.676118 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260810 04:41:12.676414 26708 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:41:12.666959 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.126:37755 (local address 127.22.167.67:48123)
0810 04:41:12.667133 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0810 04:41:12.667185 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0810 04:41:12.667720 (+ 535us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 04:41:12.667971 (+ 251us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 04:41:12.667974 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 04:41:12.668235 (+ 261us) client_negotiation.cc:189] Negotiated authn=SASL
0810 04:41:12.668443 (+ 208us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:41:12.668448 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:41:12.669389 (+ 941us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:41:12.669389 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:41:12.670408 (+ 1019us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:41:12.670411 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:41:12.670770 (+ 359us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:41:12.670771 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:41:12.670960 (+ 189us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:41:12.671578 (+ 618us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 04:41:12.671586 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0810 04:41:12.673256 (+ 1670us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 04:41:12.674843 (+ 1587us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 04:41:12.674850 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 04:41:12.674852 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 04:41:12.675190 (+ 338us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 04:41:12.675374 (+ 184us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 04:41:12.675375 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 04:41:12.675376 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 04:41:12.675522 (+ 146us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 04:41:12.675787 (+ 265us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 04:41:12.675795 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 04:41:12.675919 (+ 124us) client_negotiation.cc:769] Sending connection context
0810 04:41:12.676009 (+ 90us) client_negotiation.cc:240] Negotiation successful
0810 04:41:12.676162 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":85,"threads_started":1}
I20260810 04:41:12.679790 26664 rpc_server.cc:307] RPC server started. Bound to: 127.22.167.67:43761
I20260810 04:41:12.679831 26891 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.167.67:43761 every 8 connection(s)
I20260810 04:41:12.680234 26664 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786336863388938-23197-0/minicluster-data/ts-2/data/info.pb
I20260810 04:41:12.680812 26893 heartbeater.cc:347] Connected to a master server at 127.22.167.126:37755
I20260810 04:41:12.681090 26893 heartbeater.cc:464] Registering TS with master...
I20260810 04:41:12.681416 26893 heartbeater.cc:511] Master 127.22.167.126:37755 requested a full tablet report, sending...
I20260810 04:41:12.681836 26051 ts_manager.cc:194] Registered new tserver with Master: 086ccfa1dd66486bbbcc563857db108b (127.22.167.67:43761)
I20260810 04:41:12.682322 26051 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.167.67@KRBTEST.COM'} at 127.22.167.67:48123
I20260810 04:41:12.683542 23197 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskemv_AM/build/release/bin/kudu as pid 26664
I20260810 04:41:12.683632 23197 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786336863388938-23197-0/minicluster-data/ts-2/wal/instance
I20260810 04:41:12.683809 23197 external_mini_cluster.cc:1428] Setting key c285e9a2f395009e91d8d3400db4720b
I20260810 04:41:12.684388 26877 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:41:12.678321 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.126:37755 (local address 127.22.167.67:37031)
0810 04:41:12.678541 (+ 220us) negotiation.cc:107] Waiting for socket to connect
0810 04:41:12.678551 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0810 04:41:12.678606 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 04:41:12.678724 (+ 118us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 04:41:12.678725 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 04:41:12.679077 (+ 352us) client_negotiation.cc:189] Negotiated authn=SASL
0810 04:41:12.679175 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:41:12.679177 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:41:12.679834 (+ 657us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:41:12.679835 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:41:12.680469 (+ 634us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:41:12.680471 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:41:12.680973 (+ 502us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:41:12.680974 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:41:12.681049 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:41:12.681441 (+ 392us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 04:41:12.681446 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0810 04:41:12.681888 (+ 442us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 04:41:12.683307 (+ 1419us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 04:41:12.683308 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 04:41:12.683309 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 04:41:12.683500 (+ 191us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 04:41:12.683707 (+ 207us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 04:41:12.683708 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 04:41:12.683708 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 04:41:12.683773 (+ 65us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 04:41:12.684073 (+ 300us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 04:41:12.684074 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 04:41:12.684142 (+ 68us) client_negotiation.cc:769] Sending connection context
0810 04:41:12.684219 (+ 77us) client_negotiation.cc:240] Negotiation successful
0810 04:41:12.684300 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":162,"thread_start_us":130,"threads_started":1}
I20260810 04:41:12.684418 26125 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:41:12.678449 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.167.67:37031 (local address 127.22.167.126:37755)
0810 04:41:12.678487 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0810 04:41:12.678489 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 04:41:12.678607 (+ 118us) server_negotiation.cc:407] Connection header received
0810 04:41:12.678660 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:41:12.678660 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:41:12.678670 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:41:12.678711 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0810 04:41:12.679254 (+ 543us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:41:12.679722 (+ 468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:41:12.680566 (+ 844us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:41:12.680850 (+ 284us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:41:12.680890 (+ 40us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:41:12.681991 (+ 1101us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 04:41:12.681993 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 04:41:12.681993 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 04:41:12.682000 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 04:41:12.683229 (+ 1229us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 04:41:12.683563 (+ 334us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 04:41:12.683564 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 04:41:12.683564 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 04:41:12.683625 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 04:41:12.683831 (+ 206us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 04:41:12.683832 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 04:41:12.683832 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 04:41:12.683957 (+ 125us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 04:41:12.684016 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0810 04:41:12.684258 (+ 242us) server_negotiation.cc:299] Negotiation successful
0810 04:41:12.684339 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260810 04:41:12.684913 23197 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 10 04:41:12 dist-test-slave-bqcl krb5kdc[25977](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786336872, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 10 04:41:12 dist-test-slave-bqcl krb5kdc[25977](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786336872, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.167.126@KRBTEST.COM
I20260810 04:41:12.709738 26125 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:41:12.702142 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41384 (local address 127.22.167.126:37755)
0810 04:41:12.702201 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0810 04:41:12.702203 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 04:41:12.702938 (+ 735us) server_negotiation.cc:407] Connection header received
0810 04:41:12.703076 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:41:12.703077 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:41:12.703087 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:41:12.703118 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0810 04:41:12.703847 (+ 729us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:41:12.704609 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:41:12.705434 (+ 825us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:41:12.705649 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:41:12.705706 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:41:12.707809 (+ 2103us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 04:41:12.707810 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 04:41:12.707811 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 04:41:12.707817 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0810 04:41:12.708858 (+ 1041us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 04:41:12.709136 (+ 278us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 04:41:12.709137 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 04:41:12.709137 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 04:41:12.709181 (+ 44us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 04:41:12.709340 (+ 159us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 04:41:12.709340 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 04:41:12.709340 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 04:41:12.709443 (+ 103us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 04:41:12.709515 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0810 04:41:12.709612 (+ 97us) server_negotiation.cc:299] Negotiation successful
0810 04:41:12.709672 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260810 04:41:12.711141 26051 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41384:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 04:41:12.712746 26051 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 04:41:12.719262 26918 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:41:12.714855 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.66:39673 (local address 127.0.0.1:42740)
0810 04:41:12.715209 (+ 354us) negotiation.cc:107] Waiting for socket to connect
0810 04:41:12.715220 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0810 04:41:12.715275 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 04:41:12.715518 (+ 243us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 04:41:12.715521 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 04:41:12.715524 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 04:41:12.715589 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:41:12.715590 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:41:12.716884 (+ 1294us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:41:12.716886 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:41:12.717967 (+ 1081us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:41:12.717969 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:41:12.718962 (+ 993us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:41:12.718963 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:41:12.719040 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:41:12.719058 (+ 18us) client_negotiation.cc:769] Sending connection context
0810 04:41:12.719122 (+ 64us) client_negotiation.cc:240] Negotiation successful
0810 04:41:12.719187 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":339,"thread_start_us":78,"threads_started":1}
I20260810 04:41:12.719364 26925 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:41:12.715054 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.65:46669 (local address 127.0.0.1:49766)
0810 04:41:12.715325 (+ 271us) negotiation.cc:107] Waiting for socket to connect
0810 04:41:12.715335 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0810 04:41:12.715382 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 04:41:12.715935 (+ 553us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 04:41:12.715936 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 04:41:12.715939 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 04:41:12.716006 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:41:12.716007 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:41:12.717235 (+ 1228us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:41:12.717236 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:41:12.718368 (+ 1132us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:41:12.718371 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:41:12.719095 (+ 724us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:41:12.719096 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:41:12.719177 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:41:12.719182 (+ 5us) client_negotiation.cc:769] Sending connection context
0810 04:41:12.719248 (+ 66us) client_negotiation.cc:240] Negotiation successful
0810 04:41:12.719294 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":243,"mutex_wait_us":1,"thread_start_us":120,"threads_started":1}
I20260810 04:41:12.719496 26930 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:41:12.715349 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49766 (local address 127.22.167.65:46669)
0810 04:41:12.715755 (+ 406us) server_negotiation.cc:206] Beginning negotiation
0810 04:41:12.715760 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 04:41:12.715770 (+ 10us) server_negotiation.cc:407] Connection header received
0810 04:41:12.715838 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:41:12.715840 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:41:12.715856 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:41:12.715926 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 04:41:12.716076 (+ 150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:41:12.717156 (+ 1080us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:41:12.718470 (+ 1314us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:41:12.719017 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:41:12.719091 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:41:12.719151 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0810 04:41:12.719265 (+ 114us) server_negotiation.cc:299] Negotiation successful
0810 04:41:12.719307 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":334,"thread_start_us":139,"threads_started":1}
I20260810 04:41:12.719578 26923 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:41:12.715003 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42740 (local address 127.22.167.66:39673)
0810 04:41:12.715354 (+ 351us) server_negotiation.cc:206] Beginning negotiation
0810 04:41:12.715359 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 04:41:12.715368 (+ 9us) server_negotiation.cc:407] Connection header received
0810 04:41:12.715409 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:41:12.715411 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:41:12.715423 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:41:12.715493 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 04:41:12.715674 (+ 181us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:41:12.716791 (+ 1117us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:41:12.718042 (+ 1251us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:41:12.718874 (+ 832us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:41:12.718948 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:41:12.719007 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0810 04:41:12.719444 (+ 437us) server_negotiation.cc:299] Negotiation successful
0810 04:41:12.719488 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":292,"thread_start_us":165,"threads_started":1}
I20260810 04:41:12.719807 26920 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:41:12.714846 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.67:43761 (local address 127.0.0.1:41078)
0810 04:41:12.715004 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0810 04:41:12.715044 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0810 04:41:12.715104 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 04:41:12.715868 (+ 764us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 04:41:12.715869 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 04:41:12.715871 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 04:41:12.715931 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:41:12.715933 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:41:12.717145 (+ 1212us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:41:12.717147 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:41:12.718376 (+ 1229us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:41:12.718377 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:41:12.719339 (+ 962us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:41:12.719340 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:41:12.719396 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:41:12.719400 (+ 4us) client_negotiation.cc:769] Sending connection context
0810 04:41:12.719446 (+ 46us) client_negotiation.cc:240] Negotiation successful
0810 04:41:12.719746 (+ 300us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":106,"mutex_wait_us":49,"thread_start_us":59,"threads_started":1}
I20260810 04:41:12.720283 26525 tablet_service.cc:1467] Processing CreateTablet for tablet 0d97a94e9e7d44258b7821928b2ca453 (DEFAULT_TABLE table=test-table [id=e583b6d014bf4b6eb133fcbfb430cf5a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 04:41:12.720305 26289 tablet_service.cc:1467] Processing CreateTablet for tablet 0d97a94e9e7d44258b7821928b2ca453 (DEFAULT_TABLE table=test-table [id=e583b6d014bf4b6eb133fcbfb430cf5a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 04:41:12.720773 26525 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0d97a94e9e7d44258b7821928b2ca453. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 04:41:12.720820 26289 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0d97a94e9e7d44258b7821928b2ca453. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 04:41:12.722501 26921 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:41:12.715022 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41078 (local address 127.22.167.67:43761)
0810 04:41:12.715640 (+ 618us) server_negotiation.cc:206] Beginning negotiation
0810 04:41:12.715645 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 04:41:12.715655 (+ 10us) server_negotiation.cc:407] Connection header received
0810 04:41:12.715710 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:41:12.715712 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:41:12.715728 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:41:12.715802 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 04:41:12.716007 (+ 205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:41:12.717076 (+ 1069us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:41:12.718455 (+ 1379us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:41:12.719197 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:41:12.719255 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:41:12.719307 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0810 04:41:12.722269 (+ 2962us) server_negotiation.cc:299] Negotiation successful
0810 04:41:12.722319 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":576,"thread_start_us":85,"threads_started":1}
I20260810 04:41:12.723331 26775 tablet_service.cc:1467] Processing CreateTablet for tablet 0d97a94e9e7d44258b7821928b2ca453 (DEFAULT_TABLE table=test-table [id=e583b6d014bf4b6eb133fcbfb430cf5a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 04:41:12.723675 26775 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0d97a94e9e7d44258b7821928b2ca453. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 04:41:12.724556 26931 tablet_bootstrap.cc:492] T 0d97a94e9e7d44258b7821928b2ca453 P 3202926ff8c141af80ac121604f0dce3: Bootstrap starting.
I20260810 04:41:12.724722 26932 tablet_bootstrap.cc:492] T 0d97a94e9e7d44258b7821928b2ca453 P 3b3f370bee5b43f4882dbdcfbabc8dad: Bootstrap starting.
I20260810 04:41:12.725740 26931 tablet_bootstrap.cc:654] T 0d97a94e9e7d44258b7821928b2ca453 P 3202926ff8c141af80ac121604f0dce3: Neither blocks nor log segments found. Creating new log.
I20260810 04:41:12.726069 26932 tablet_bootstrap.cc:654] T 0d97a94e9e7d44258b7821928b2ca453 P 3b3f370bee5b43f4882dbdcfbabc8dad: Neither blocks nor log segments found. Creating new log.
I20260810 04:41:12.726399 26931 log.cc:824] T 0d97a94e9e7d44258b7821928b2ca453 P 3202926ff8c141af80ac121604f0dce3: Log is configured to *not* fsync() on all Append() calls
I20260810 04:41:12.726919 26932 log.cc:824] T 0d97a94e9e7d44258b7821928b2ca453 P 3b3f370bee5b43f4882dbdcfbabc8dad: Log is configured to *not* fsync() on all Append() calls
I20260810 04:41:12.727167 26931 tablet_bootstrap.cc:492] T 0d97a94e9e7d44258b7821928b2ca453 P 3202926ff8c141af80ac121604f0dce3: No bootstrap required, opened a new log
I20260810 04:41:12.727237 26931 ts_tablet_manager.cc:1397] T 0d97a94e9e7d44258b7821928b2ca453 P 3202926ff8c141af80ac121604f0dce3: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20260810 04:41:12.727803 26932 tablet_bootstrap.cc:492] T 0d97a94e9e7d44258b7821928b2ca453 P 3b3f370bee5b43f4882dbdcfbabc8dad: No bootstrap required, opened a new log
I20260810 04:41:12.727870 26932 ts_tablet_manager.cc:1397] T 0d97a94e9e7d44258b7821928b2ca453 P 3b3f370bee5b43f4882dbdcfbabc8dad: Time spent bootstrapping tablet: real 0.003s user 0.003s sys 0.000s
I20260810 04:41:12.728082 26936 tablet_bootstrap.cc:492] T 0d97a94e9e7d44258b7821928b2ca453 P 086ccfa1dd66486bbbcc563857db108b: Bootstrap starting.
I20260810 04:41:12.729205 26931 raft_consensus.cc:348] T 0d97a94e9e7d44258b7821928b2ca453 P 3202926ff8c141af80ac121604f0dce3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "086ccfa1dd66486bbbcc563857db108b" member_type: VOTER last_known_addr { host: "127.22.167.67" port: 43761 } } peers { permanent_uuid: "3202926ff8c141af80ac121604f0dce3" member_type: VOTER last_known_addr { host: "127.22.167.66" port: 39673 } } peers { permanent_uuid: "3b3f370bee5b43f4882dbdcfbabc8dad" member_type: VOTER last_known_addr { host: "127.22.167.65" port: 46669 } }
I20260810 04:41:12.729388 26936 tablet_bootstrap.cc:654] T 0d97a94e9e7d44258b7821928b2ca453 P 086ccfa1dd66486bbbcc563857db108b: Neither blocks nor log segments found. Creating new log.
I20260810 04:41:12.729387 26931 raft_consensus.cc:374] T 0d97a94e9e7d44258b7821928b2ca453 P 3202926ff8c141af80ac121604f0dce3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 04:41:12.729430 26931 raft_consensus.cc:729] T 0d97a94e9e7d44258b7821928b2ca453 P 3202926ff8c141af80ac121604f0dce3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3202926ff8c141af80ac121604f0dce3, State: Initialized, Role: FOLLOWER
I20260810 04:41:12.729545 26932 raft_consensus.cc:348] T 0d97a94e9e7d44258b7821928b2ca453 P 3b3f370bee5b43f4882dbdcfbabc8dad [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "086ccfa1dd66486bbbcc563857db108b" member_type: VOTER last_known_addr { host: "127.22.167.67" port: 43761 } } peers { permanent_uuid: "3202926ff8c141af80ac121604f0dce3" member_type: VOTER last_known_addr { host: "127.22.167.66" port: 39673 } } peers { permanent_uuid: "3b3f370bee5b43f4882dbdcfbabc8dad" member_type: VOTER last_known_addr { host: "127.22.167.65" port: 46669 } }
I20260810 04:41:12.729604 26931 consensus_queue.cc:260] T 0d97a94e9e7d44258b7821928b2ca453 P 3202926ff8c141af80ac121604f0dce3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "086ccfa1dd66486bbbcc563857db108b" member_type: VOTER last_known_addr { host: "127.22.167.67" port: 43761 } } peers { permanent_uuid: "3202926ff8c141af80ac121604f0dce3" member_type: VOTER last_known_addr { host: "127.22.167.66" port: 39673 } } peers { permanent_uuid: "3b3f370bee5b43f4882dbdcfbabc8dad" member_type: VOTER last_known_addr { host: "127.22.167.65" port: 46669 } }
I20260810 04:41:12.729688 26932 raft_consensus.cc:374] T 0d97a94e9e7d44258b7821928b2ca453 P 3b3f370bee5b43f4882dbdcfbabc8dad [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 04:41:12.729693 26936 log.cc:824] T 0d97a94e9e7d44258b7821928b2ca453 P 086ccfa1dd66486bbbcc563857db108b: Log is configured to *not* fsync() on all Append() calls
I20260810 04:41:12.729732 26932 raft_consensus.cc:729] T 0d97a94e9e7d44258b7821928b2ca453 P 3b3f370bee5b43f4882dbdcfbabc8dad [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3b3f370bee5b43f4882dbdcfbabc8dad, State: Initialized, Role: FOLLOWER
I20260810 04:41:12.729900 26931 ts_tablet_manager.cc:1428] T 0d97a94e9e7d44258b7821928b2ca453 P 3202926ff8c141af80ac121604f0dce3: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260810 04:41:12.729875 26932 consensus_queue.cc:260] T 0d97a94e9e7d44258b7821928b2ca453 P 3b3f370bee5b43f4882dbdcfbabc8dad [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "086ccfa1dd66486bbbcc563857db108b" member_type: VOTER last_known_addr { host: "127.22.167.67" port: 43761 } } peers { permanent_uuid: "3202926ff8c141af80ac121604f0dce3" member_type: VOTER last_known_addr { host: "127.22.167.66" port: 39673 } } peers { permanent_uuid: "3b3f370bee5b43f4882dbdcfbabc8dad" member_type: VOTER last_known_addr { host: "127.22.167.65" port: 46669 } }
I20260810 04:41:12.730014 26652 heartbeater.cc:503] Master 127.22.167.126:37755 was elected leader, sending a full tablet report...
I20260810 04:41:12.730165 26932 ts_tablet_manager.cc:1428] T 0d97a94e9e7d44258b7821928b2ca453 P 3b3f370bee5b43f4882dbdcfbabc8dad: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260810 04:41:12.730552 26936 tablet_bootstrap.cc:492] T 0d97a94e9e7d44258b7821928b2ca453 P 086ccfa1dd66486bbbcc563857db108b: No bootstrap required, opened a new log
I20260810 04:41:12.730624 26936 ts_tablet_manager.cc:1397] T 0d97a94e9e7d44258b7821928b2ca453 P 086ccfa1dd66486bbbcc563857db108b: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260810 04:41:12.732337 26936 raft_consensus.cc:348] T 0d97a94e9e7d44258b7821928b2ca453 P 086ccfa1dd66486bbbcc563857db108b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "086ccfa1dd66486bbbcc563857db108b" member_type: VOTER last_known_addr { host: "127.22.167.67" port: 43761 } } peers { permanent_uuid: "3202926ff8c141af80ac121604f0dce3" member_type: VOTER last_known_addr { host: "127.22.167.66" port: 39673 } } peers { permanent_uuid: "3b3f370bee5b43f4882dbdcfbabc8dad" member_type: VOTER last_known_addr { host: "127.22.167.65" port: 46669 } }
I20260810 04:41:12.732479 26936 raft_consensus.cc:374] T 0d97a94e9e7d44258b7821928b2ca453 P 086ccfa1dd66486bbbcc563857db108b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 04:41:12.732501 26936 raft_consensus.cc:729] T 0d97a94e9e7d44258b7821928b2ca453 P 086ccfa1dd66486bbbcc563857db108b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 086ccfa1dd66486bbbcc563857db108b, State: Initialized, Role: FOLLOWER
I20260810 04:41:12.732583 26936 consensus_queue.cc:260] T 0d97a94e9e7d44258b7821928b2ca453 P 086ccfa1dd66486bbbcc563857db108b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "086ccfa1dd66486bbbcc563857db108b" member_type: VOTER last_known_addr { host: "127.22.167.67" port: 43761 } } peers { permanent_uuid: "3202926ff8c141af80ac121604f0dce3" member_type: VOTER last_known_addr { host: "127.22.167.66" port: 39673 } } peers { permanent_uuid: "3b3f370bee5b43f4882dbdcfbabc8dad" member_type: VOTER last_known_addr { host: "127.22.167.65" port: 46669 } }
I20260810 04:41:12.732769 26936 ts_tablet_manager.cc:1428] T 0d97a94e9e7d44258b7821928b2ca453 P 086ccfa1dd66486bbbcc563857db108b: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260810 04:41:12.732824 26893 heartbeater.cc:503] Master 127.22.167.126:37755 was elected leader, sending a full tablet report...
I20260810 04:41:12.732921 26410 heartbeater.cc:503] Master 127.22.167.126:37755 was elected leader, sending a full tablet report...
I20260810 04:41:12.737193 26941 raft_consensus.cc:482] T 0d97a94e9e7d44258b7821928b2ca453 P 3b3f370bee5b43f4882dbdcfbabc8dad [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 04:41:12.737280 26941 raft_consensus.cc:504] T 0d97a94e9e7d44258b7821928b2ca453 P 3b3f370bee5b43f4882dbdcfbabc8dad [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "086ccfa1dd66486bbbcc563857db108b" member_type: VOTER last_known_addr { host: "127.22.167.67" port: 43761 } } peers { permanent_uuid: "3202926ff8c141af80ac121604f0dce3" member_type: VOTER last_known_addr { host: "127.22.167.66" port: 39673 } } peers { permanent_uuid: "3b3f370bee5b43f4882dbdcfbabc8dad" member_type: VOTER last_known_addr { host: "127.22.167.65" port: 46669 } }
I20260810 04:41:12.737562 26941 leader_election.cc:288] T 0d97a94e9e7d44258b7821928b2ca453 P 3b3f370bee5b43f4882dbdcfbabc8dad [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 086ccfa1dd66486bbbcc563857db108b (127.22.167.67:43761), 3202926ff8c141af80ac121604f0dce3 (127.22.167.66:39673)
I20260810 04:41:12.740331 26923 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:41:12.737769 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.167.65:57223 (local address 127.22.167.66:39673)
0810 04:41:12.737890 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0810 04:41:12.737892 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 04:41:12.737947 (+ 55us) server_negotiation.cc:407] Connection header received
0810 04:41:12.738006 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:41:12.738006 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:41:12.738014 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:41:12.738066 (+ 52us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 04:41:12.738213 (+ 147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:41:12.738671 (+ 458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:41:12.739537 (+ 866us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:41:12.739931 (+ 394us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:41:12.740081 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:41:12.740094 (+ 13us) server_negotiation.cc:1035] Waiting for connection context
0810 04:41:12.740228 (+ 134us) server_negotiation.cc:299] Negotiation successful
0810 04:41:12.740268 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":98}
I20260810 04:41:12.740417 26921 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:41:12.737762 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.167.65:48017 (local address 127.22.167.67:43761)
0810 04:41:12.737897 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0810 04:41:12.737899 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 04:41:12.737904 (+ 5us) server_negotiation.cc:407] Connection header received
0810 04:41:12.737921 (+ 17us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:41:12.737922 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:41:12.737929 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:41:12.737968 (+ 39us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 04:41:12.738118 (+ 150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:41:12.738627 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:41:12.739531 (+ 904us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:41:12.740036 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:41:12.740125 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:41:12.740136 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0810 04:41:12.740313 (+ 177us) server_negotiation.cc:299] Negotiation successful
0810 04:41:12.740355 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":115}
I20260810 04:41:12.740430 26210 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:41:12.737726 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.67:43761 (local address 127.22.167.65:48017)
0810 04:41:12.737788 (+ 62us) negotiation.cc:107] Waiting for socket to connect
0810 04:41:12.737799 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0810 04:41:12.737835 (+ 36us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 04:41:12.737986 (+ 151us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 04:41:12.737987 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 04:41:12.737988 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 04:41:12.738063 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:41:12.738064 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:41:12.738695 (+ 631us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:41:12.738696 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:41:12.739451 (+ 755us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:41:12.739453 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:41:12.740123 (+ 670us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:41:12.740124 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:41:12.740234 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:41:12.740238 (+ 4us) client_negotiation.cc:769] Sending connection context
0810 04:41:12.740288 (+ 50us) client_negotiation.cc:240] Negotiation successful
0810 04:41:12.740328 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":46,"mutex_wait_us":26}
I20260810 04:41:12.740357 26947 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:41:12.737734 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.66:39673 (local address 127.22.167.65:57223)
0810 04:41:12.737897 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0810 04:41:12.737906 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0810 04:41:12.737961 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 04:41:12.738071 (+ 110us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 04:41:12.738072 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 04:41:12.738073 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 04:41:12.738165 (+ 92us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:41:12.738167 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:41:12.738737 (+ 570us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:41:12.738738 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:41:12.739460 (+ 722us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:41:12.739462 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:41:12.740082 (+ 620us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:41:12.740083 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:41:12.740166 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:41:12.740170 (+ 4us) client_negotiation.cc:769] Sending connection context
0810 04:41:12.740217 (+ 47us) client_negotiation.cc:240] Negotiation successful
0810 04:41:12.740261 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":146,"mutex_wait_us":18,"thread_start_us":84,"threads_started":1}
I20260810 04:41:12.740828 26817 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0d97a94e9e7d44258b7821928b2ca453" candidate_uuid: "3b3f370bee5b43f4882dbdcfbabc8dad" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "086ccfa1dd66486bbbcc563857db108b" is_pre_election: true
I20260810 04:41:12.740906 26567 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0d97a94e9e7d44258b7821928b2ca453" candidate_uuid: "3b3f370bee5b43f4882dbdcfbabc8dad" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3202926ff8c141af80ac121604f0dce3" is_pre_election: true
I20260810 04:41:12.740957 26817 raft_consensus.cc:2454] T 0d97a94e9e7d44258b7821928b2ca453 P 086ccfa1dd66486bbbcc563857db108b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b3f370bee5b43f4882dbdcfbabc8dad in term 0.
I20260810 04:41:12.741024 26567 raft_consensus.cc:2454] T 0d97a94e9e7d44258b7821928b2ca453 P 3202926ff8c141af80ac121604f0dce3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b3f370bee5b43f4882dbdcfbabc8dad in term 0.
I20260810 04:41:12.741204 26196 leader_election.cc:302] T 0d97a94e9e7d44258b7821928b2ca453 P 3b3f370bee5b43f4882dbdcfbabc8dad [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 086ccfa1dd66486bbbcc563857db108b, 3b3f370bee5b43f4882dbdcfbabc8dad; no voters:
I20260810 04:41:12.741338 26941 raft_consensus.cc:2781] T 0d97a94e9e7d44258b7821928b2ca453 P 3b3f370bee5b43f4882dbdcfbabc8dad [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 04:41:12.741389 26941 raft_consensus.cc:482] T 0d97a94e9e7d44258b7821928b2ca453 P 3b3f370bee5b43f4882dbdcfbabc8dad [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 04:41:12.741421 26941 raft_consensus.cc:3037] T 0d97a94e9e7d44258b7821928b2ca453 P 3b3f370bee5b43f4882dbdcfbabc8dad [term 0 FOLLOWER]: Advancing to term 1
I20260810 04:41:12.742105 26941 raft_consensus.cc:504] T 0d97a94e9e7d44258b7821928b2ca453 P 3b3f370bee5b43f4882dbdcfbabc8dad [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "086ccfa1dd66486bbbcc563857db108b" member_type: VOTER last_known_addr { host: "127.22.167.67" port: 43761 } } peers { permanent_uuid: "3202926ff8c141af80ac121604f0dce3" member_type: VOTER last_known_addr { host: "127.22.167.66" port: 39673 } } peers { permanent_uuid: "3b3f370bee5b43f4882dbdcfbabc8dad" member_type: VOTER last_known_addr { host: "127.22.167.65" port: 46669 } }
I20260810 04:41:12.742275 26941 leader_election.cc:288] T 0d97a94e9e7d44258b7821928b2ca453 P 3b3f370bee5b43f4882dbdcfbabc8dad [CANDIDATE]: Term 1 election: Requested vote from peers 086ccfa1dd66486bbbcc563857db108b (127.22.167.67:43761), 3202926ff8c141af80ac121604f0dce3 (127.22.167.66:39673)
I20260810 04:41:12.742412 26567 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0d97a94e9e7d44258b7821928b2ca453" candidate_uuid: "3b3f370bee5b43f4882dbdcfbabc8dad" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3202926ff8c141af80ac121604f0dce3"
I20260810 04:41:12.742456 26817 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0d97a94e9e7d44258b7821928b2ca453" candidate_uuid: "3b3f370bee5b43f4882dbdcfbabc8dad" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "086ccfa1dd66486bbbcc563857db108b"
I20260810 04:41:12.742496 26567 raft_consensus.cc:3037] T 0d97a94e9e7d44258b7821928b2ca453 P 3202926ff8c141af80ac121604f0dce3 [term 0 FOLLOWER]: Advancing to term 1
I20260810 04:41:12.742539 26817 raft_consensus.cc:3037] T 0d97a94e9e7d44258b7821928b2ca453 P 086ccfa1dd66486bbbcc563857db108b [term 0 FOLLOWER]: Advancing to term 1
I20260810 04:41:12.743353 26567 raft_consensus.cc:2454] T 0d97a94e9e7d44258b7821928b2ca453 P 3202926ff8c141af80ac121604f0dce3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3b3f370bee5b43f4882dbdcfbabc8dad in term 1.
I20260810 04:41:12.743353 26817 raft_consensus.cc:2454] T 0d97a94e9e7d44258b7821928b2ca453 P 086ccfa1dd66486bbbcc563857db108b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3b3f370bee5b43f4882dbdcfbabc8dad in term 1.
I20260810 04:41:12.743803 26191 leader_election.cc:302] T 0d97a94e9e7d44258b7821928b2ca453 P 3b3f370bee5b43f4882dbdcfbabc8dad [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3202926ff8c141af80ac121604f0dce3, 3b3f370bee5b43f4882dbdcfbabc8dad; no voters:
I20260810 04:41:12.743966 26941 raft_consensus.cc:2781] T 0d97a94e9e7d44258b7821928b2ca453 P 3b3f370bee5b43f4882dbdcfbabc8dad [term 1 FOLLOWER]: Leader election won for term 1
I20260810 04:41:12.744117 26941 raft_consensus.cc:686] T 0d97a94e9e7d44258b7821928b2ca453 P 3b3f370bee5b43f4882dbdcfbabc8dad [term 1 LEADER]: Becoming Leader. State: Replica: 3b3f370bee5b43f4882dbdcfbabc8dad, State: Running, Role: LEADER
I20260810 04:41:12.744269 26941 consensus_queue.cc:237] T 0d97a94e9e7d44258b7821928b2ca453 P 3b3f370bee5b43f4882dbdcfbabc8dad [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "086ccfa1dd66486bbbcc563857db108b" member_type: VOTER last_known_addr { host: "127.22.167.67" port: 43761 } } peers { permanent_uuid: "3202926ff8c141af80ac121604f0dce3" member_type: VOTER last_known_addr { host: "127.22.167.66" port: 39673 } } peers { permanent_uuid: "3b3f370bee5b43f4882dbdcfbabc8dad" member_type: VOTER last_known_addr { host: "127.22.167.65" port: 46669 } }
I20260810 04:41:12.745051 26050 catalog_manager.cc:5495] T 0d97a94e9e7d44258b7821928b2ca453 P 3b3f370bee5b43f4882dbdcfbabc8dad reported cstate change: term changed from 0 to 1, leader changed from <none> to 3b3f370bee5b43f4882dbdcfbabc8dad (127.22.167.65). New cstate: current_term: 1 leader_uuid: "3b3f370bee5b43f4882dbdcfbabc8dad" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "086ccfa1dd66486bbbcc563857db108b" member_type: VOTER last_known_addr { host: "127.22.167.67" port: 43761 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3202926ff8c141af80ac121604f0dce3" member_type: VOTER last_known_addr { host: "127.22.167.66" port: 39673 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3b3f370bee5b43f4882dbdcfbabc8dad" member_type: VOTER last_known_addr { host: "127.22.167.65" port: 46669 } health_report { overall_health: HEALTHY } } }
Aug 10 04:41:12 dist-test-slave-bqcl krb5kdc[25977](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786336872, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.167.126@KRBTEST.COM for kudu/127.22.167.126@KRBTEST.COM
I20260810 04:41:12.769315 26956 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:41:12.762749 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.126:37755 (local address 127.0.0.1:41390)
0810 04:41:12.762891 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0810 04:41:12.762901 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0810 04:41:12.762960 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 04:41:12.763075 (+ 115us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 04:41:12.763076 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 04:41:12.763377 (+ 301us) client_negotiation.cc:189] Negotiated authn=SASL
0810 04:41:12.763423 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:41:12.763425 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:41:12.764107 (+ 682us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:41:12.764107 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:41:12.764622 (+ 515us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:41:12.764623 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:41:12.764978 (+ 355us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:41:12.764980 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:41:12.765035 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:41:12.765610 (+ 575us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 04:41:12.765614 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0810 04:41:12.767020 (+ 1406us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 04:41:12.768377 (+ 1357us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 04:41:12.768384 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 04:41:12.768386 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 04:41:12.768585 (+ 199us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 04:41:12.768800 (+ 215us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 04:41:12.768800 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 04:41:12.768800 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 04:41:12.768864 (+ 64us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 04:41:12.769052 (+ 188us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 04:41:12.769058 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 04:41:12.769099 (+ 41us) client_negotiation.cc:769] Sending connection context
0810 04:41:12.769145 (+ 46us) client_negotiation.cc:240] Negotiation successful
0810 04:41:12.769238 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":78,"threads_started":1}
I20260810 04:41:12.769385 26125 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:41:12.762799 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41390 (local address 127.22.167.126:37755)
0810 04:41:12.762889 (+ 90us) server_negotiation.cc:206] Beginning negotiation
0810 04:41:12.762891 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 04:41:12.762955 (+ 64us) server_negotiation.cc:407] Connection header received
0810 04:41:12.763005 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:41:12.763006 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:41:12.763016 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:41:12.763065 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0810 04:41:12.763511 (+ 446us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:41:12.764040 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:41:12.764679 (+ 639us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:41:12.764850 (+ 171us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:41:12.764895 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:41:12.767145 (+ 2250us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 04:41:12.767146 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 04:41:12.767146 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 04:41:12.767153 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 04:41:12.768296 (+ 1143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 04:41:12.768669 (+ 373us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 04:41:12.768670 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 04:41:12.768670 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 04:41:12.768734 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 04:41:12.768910 (+ 176us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 04:41:12.768911 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 04:41:12.768911 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 04:41:12.769010 (+ 99us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 04:41:12.769058 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0810 04:41:12.769170 (+ 112us) server_negotiation.cc:299] Negotiation successful
0810 04:41:12.769288 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":53,"server-negotiator.queue_time_us":66}
I20260810 04:41:12.771286 26050 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.22.167.126@KRBTEST.COM'} at 127.0.0.1:41390:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260810 04:41:12.771500 26050 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260810 04:41:12.771940 26653 tablet.cc:2369] T 0d97a94e9e7d44258b7821928b2ca453 P 3202926ff8c141af80ac121604f0dce3: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 04:41:12.773083 26289 tablet_service.cc:1467] Processing CreateTablet for tablet 5903b9663bd54231a8fa17891266352e (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=2e970e91b67542b4aab2cba74a47ae78]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260810 04:41:12.773090 26525 tablet_service.cc:1467] Processing CreateTablet for tablet 5903b9663bd54231a8fa17891266352e (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=2e970e91b67542b4aab2cba74a47ae78]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260810 04:41:12.773262 26525 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5903b9663bd54231a8fa17891266352e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 04:41:12.773263 26289 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5903b9663bd54231a8fa17891266352e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 04:41:12.774325 26775 tablet_service.cc:1467] Processing CreateTablet for tablet 5903b9663bd54231a8fa17891266352e (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=2e970e91b67542b4aab2cba74a47ae78]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260810 04:41:12.774483 26775 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5903b9663bd54231a8fa17891266352e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 04:41:12.774772 26931 tablet_bootstrap.cc:492] T 5903b9663bd54231a8fa17891266352e P 3202926ff8c141af80ac121604f0dce3: Bootstrap starting.
I20260810 04:41:12.774781 26932 tablet_bootstrap.cc:492] T 5903b9663bd54231a8fa17891266352e P 3b3f370bee5b43f4882dbdcfbabc8dad: Bootstrap starting.
I20260810 04:41:12.775215 26932 tablet_bootstrap.cc:654] T 5903b9663bd54231a8fa17891266352e P 3b3f370bee5b43f4882dbdcfbabc8dad: Neither blocks nor log segments found. Creating new log.
I20260810 04:41:12.775336 26931 tablet_bootstrap.cc:654] T 5903b9663bd54231a8fa17891266352e P 3202926ff8c141af80ac121604f0dce3: Neither blocks nor log segments found. Creating new log.
I20260810 04:41:12.775954 26932 tablet_bootstrap.cc:492] T 5903b9663bd54231a8fa17891266352e P 3b3f370bee5b43f4882dbdcfbabc8dad: No bootstrap required, opened a new log
I20260810 04:41:12.776015 26932 ts_tablet_manager.cc:1397] T 5903b9663bd54231a8fa17891266352e P 3b3f370bee5b43f4882dbdcfbabc8dad: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260810 04:41:12.776180 26932 raft_consensus.cc:348] T 5903b9663bd54231a8fa17891266352e P 3b3f370bee5b43f4882dbdcfbabc8dad [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b3f370bee5b43f4882dbdcfbabc8dad" member_type: VOTER last_known_addr { host: "127.22.167.65" port: 46669 } } peers { permanent_uuid: "3202926ff8c141af80ac121604f0dce3" member_type: VOTER last_known_addr { host: "127.22.167.66" port: 39673 } } peers { permanent_uuid: "086ccfa1dd66486bbbcc563857db108b" member_type: VOTER last_known_addr { host: "127.22.167.67" port: 43761 } }
I20260810 04:41:12.776285 26932 raft_consensus.cc:374] T 5903b9663bd54231a8fa17891266352e P 3b3f370bee5b43f4882dbdcfbabc8dad [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 04:41:12.776273 26931 tablet_bootstrap.cc:492] T 5903b9663bd54231a8fa17891266352e P 3202926ff8c141af80ac121604f0dce3: No bootstrap required, opened a new log
I20260810 04:41:12.776302 26932 raft_consensus.cc:729] T 5903b9663bd54231a8fa17891266352e P 3b3f370bee5b43f4882dbdcfbabc8dad [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3b3f370bee5b43f4882dbdcfbabc8dad, State: Initialized, Role: FOLLOWER
I20260810 04:41:12.776314 26931 ts_tablet_manager.cc:1397] T 5903b9663bd54231a8fa17891266352e P 3202926ff8c141af80ac121604f0dce3: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260810 04:41:12.776340 26932 consensus_queue.cc:260] T 5903b9663bd54231a8fa17891266352e P 3b3f370bee5b43f4882dbdcfbabc8dad [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b3f370bee5b43f4882dbdcfbabc8dad" member_type: VOTER last_known_addr { host: "127.22.167.65" port: 46669 } } peers { permanent_uuid: "3202926ff8c141af80ac121604f0dce3" member_type: VOTER last_known_addr { host: "127.22.167.66" port: 39673 } } peers { permanent_uuid: "086ccfa1dd66486bbbcc563857db108b" member_type: VOTER last_known_addr { host: "127.22.167.67" port: 43761 } }
I20260810 04:41:12.776408 26932 ts_tablet_manager.cc:1428] T 5903b9663bd54231a8fa17891266352e P 3b3f370bee5b43f4882dbdcfbabc8dad: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260810 04:41:12.776443 26931 raft_consensus.cc:348] T 5903b9663bd54231a8fa17891266352e P 3202926ff8c141af80ac121604f0dce3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b3f370bee5b43f4882dbdcfbabc8dad" member_type: VOTER last_known_addr { host: "127.22.167.65" port: 46669 } } peers { permanent_uuid: "3202926ff8c141af80ac121604f0dce3" member_type: VOTER last_known_addr { host: "127.22.167.66" port: 39673 } } peers { permanent_uuid: "086ccfa1dd66486bbbcc563857db108b" member_type: VOTER last_known_addr { host: "127.22.167.67" port: 43761 } }
I20260810 04:41:12.776492 26936 tablet_bootstrap.cc:492] T 5903b9663bd54231a8fa17891266352e P 086ccfa1dd66486bbbcc563857db108b: Bootstrap starting.
I20260810 04:41:12.776520 26931 raft_consensus.cc:374] T 5903b9663bd54231a8fa17891266352e P 3202926ff8c141af80ac121604f0dce3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 04:41:12.776540 26931 raft_consensus.cc:729] T 5903b9663bd54231a8fa17891266352e P 3202926ff8c141af80ac121604f0dce3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3202926ff8c141af80ac121604f0dce3, State: Initialized, Role: FOLLOWER
I20260810 04:41:12.776484 26941 tablet_replica.cc:439] T 5903b9663bd54231a8fa17891266352e P 3b3f370bee5b43f4882dbdcfbabc8dad: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b3f370bee5b43f4882dbdcfbabc8dad" member_type: VOTER last_known_addr { host: "127.22.167.65" port: 46669 } } peers { permanent_uuid: "3202926ff8c141af80ac121604f0dce3" member_type: VOTER last_known_addr { host: "127.22.167.66" port: 39673 } } peers { permanent_uuid: "086ccfa1dd66486bbbcc563857db108b" member_type: VOTER last_known_addr { host: "127.22.167.67" port: 43761 } } }
I20260810 04:41:12.776563 26941 tablet_replica.cc:442] T 5903b9663bd54231a8fa17891266352e P 3b3f370bee5b43f4882dbdcfbabc8dad: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 04:41:12.776582 26931 consensus_queue.cc:260] T 5903b9663bd54231a8fa17891266352e P 3202926ff8c141af80ac121604f0dce3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b3f370bee5b43f4882dbdcfbabc8dad" member_type: VOTER last_known_addr { host: "127.22.167.65" port: 46669 } } peers { permanent_uuid: "3202926ff8c141af80ac121604f0dce3" member_type: VOTER last_known_addr { host: "127.22.167.66" port: 39673 } } peers { permanent_uuid: "086ccfa1dd66486bbbcc563857db108b" member_type: VOTER last_known_addr { host: "127.22.167.67" port: 43761 } }
I20260810 04:41:12.776683 26931 ts_tablet_manager.cc:1428] T 5903b9663bd54231a8fa17891266352e P 3202926ff8c141af80ac121604f0dce3: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260810 04:41:12.777012 26936 tablet_bootstrap.cc:654] T 5903b9663bd54231a8fa17891266352e P 086ccfa1dd66486bbbcc563857db108b: Neither blocks nor log segments found. Creating new log.
I20260810 04:41:12.777383 26938 tablet_replica.cc:439] T 5903b9663bd54231a8fa17891266352e P 3202926ff8c141af80ac121604f0dce3: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b3f370bee5b43f4882dbdcfbabc8dad" member_type: VOTER last_known_addr { host: "127.22.167.65" port: 46669 } } peers { permanent_uuid: "3202926ff8c141af80ac121604f0dce3" member_type: VOTER last_known_addr { host: "127.22.167.66" port: 39673 } } peers { permanent_uuid: "086ccfa1dd66486bbbcc563857db108b" member_type: VOTER last_known_addr { host: "127.22.167.67" port: 43761 } } }
I20260810 04:41:12.777465 26938 tablet_replica.cc:442] T 5903b9663bd54231a8fa17891266352e P 3202926ff8c141af80ac121604f0dce3: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 04:41:12.777669 26936 tablet_bootstrap.cc:492] T 5903b9663bd54231a8fa17891266352e P 086ccfa1dd66486bbbcc563857db108b: No bootstrap required, opened a new log
I20260810 04:41:12.777725 26936 ts_tablet_manager.cc:1397] T 5903b9663bd54231a8fa17891266352e P 086ccfa1dd66486bbbcc563857db108b: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260810 04:41:12.777854 26936 raft_consensus.cc:348] T 5903b9663bd54231a8fa17891266352e P 086ccfa1dd66486bbbcc563857db108b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b3f370bee5b43f4882dbdcfbabc8dad" member_type: VOTER last_known_addr { host: "127.22.167.65" port: 46669 } } peers { permanent_uuid: "3202926ff8c141af80ac121604f0dce3" member_type: VOTER last_known_addr { host: "127.22.167.66" port: 39673 } } peers { permanent_uuid: "086ccfa1dd66486bbbcc563857db108b" member_type: VOTER last_known_addr { host: "127.22.167.67" port: 43761 } }
I20260810 04:41:12.777944 26936 raft_consensus.cc:374] T 5903b9663bd54231a8fa17891266352e P 086ccfa1dd66486bbbcc563857db108b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 04:41:12.777968 26936 raft_consensus.cc:729] T 5903b9663bd54231a8fa17891266352e P 086ccfa1dd66486bbbcc563857db108b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 086ccfa1dd66486bbbcc563857db108b, State: Initialized, Role: FOLLOWER
I20260810 04:41:12.778007 26936 consensus_queue.cc:260] T 5903b9663bd54231a8fa17891266352e P 086ccfa1dd66486bbbcc563857db108b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b3f370bee5b43f4882dbdcfbabc8dad" member_type: VOTER last_known_addr { host: "127.22.167.65" port: 46669 } } peers { permanent_uuid: "3202926ff8c141af80ac121604f0dce3" member_type: VOTER last_known_addr { host: "127.22.167.66" port: 39673 } } peers { permanent_uuid: "086ccfa1dd66486bbbcc563857db108b" member_type: VOTER last_known_addr { host: "127.22.167.67" port: 43761 } }
I20260810 04:41:12.778102 26936 ts_tablet_manager.cc:1428] T 5903b9663bd54231a8fa17891266352e P 086ccfa1dd66486bbbcc563857db108b: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260810 04:41:12.778203 26942 tablet_replica.cc:439] T 5903b9663bd54231a8fa17891266352e P 086ccfa1dd66486bbbcc563857db108b: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b3f370bee5b43f4882dbdcfbabc8dad" member_type: VOTER last_known_addr { host: "127.22.167.65" port: 46669 } } peers { permanent_uuid: "3202926ff8c141af80ac121604f0dce3" member_type: VOTER last_known_addr { host: "127.22.167.66" port: 39673 } } peers { permanent_uuid: "086ccfa1dd66486bbbcc563857db108b" member_type: VOTER last_known_addr { host: "127.22.167.67" port: 43761 } } }
I20260810 04:41:12.778301 26942 tablet_replica.cc:442] T 5903b9663bd54231a8fa17891266352e P 086ccfa1dd66486bbbcc563857db108b: This TxnStatusTablet replica's current role is: FOLLOWER
W20260810 04:41:12.856460 26412 tablet.cc:2369] T 0d97a94e9e7d44258b7821928b2ca453 P 3b3f370bee5b43f4882dbdcfbabc8dad: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260810 04:41:12.930907 26894 tablet.cc:2369] T 0d97a94e9e7d44258b7821928b2ca453 P 086ccfa1dd66486bbbcc563857db108b: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 04:41:13.089215 26941 raft_consensus.cc:482] T 5903b9663bd54231a8fa17891266352e P 3b3f370bee5b43f4882dbdcfbabc8dad [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 04:41:13.089316 26941 raft_consensus.cc:504] T 5903b9663bd54231a8fa17891266352e P 3b3f370bee5b43f4882dbdcfbabc8dad [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b3f370bee5b43f4882dbdcfbabc8dad" member_type: VOTER last_known_addr { host: "127.22.167.65" port: 46669 } } peers { permanent_uuid: "3202926ff8c141af80ac121604f0dce3" member_type: VOTER last_known_addr { host: "127.22.167.66" port: 39673 } } peers { permanent_uuid: "086ccfa1dd66486bbbcc563857db108b" member_type: VOTER last_known_addr { host: "127.22.167.67" port: 43761 } }
I20260810 04:41:13.089486 26941 leader_election.cc:288] T 5903b9663bd54231a8fa17891266352e P 3b3f370bee5b43f4882dbdcfbabc8dad [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3202926ff8c141af80ac121604f0dce3 (127.22.167.66:39673), 086ccfa1dd66486bbbcc563857db108b (127.22.167.67:43761)
I20260810 04:41:13.089741 26817 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5903b9663bd54231a8fa17891266352e" candidate_uuid: "3b3f370bee5b43f4882dbdcfbabc8dad" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "086ccfa1dd66486bbbcc563857db108b" is_pre_election: true
I20260810 04:41:13.089735 26567 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5903b9663bd54231a8fa17891266352e" candidate_uuid: "3b3f370bee5b43f4882dbdcfbabc8dad" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3202926ff8c141af80ac121604f0dce3" is_pre_election: true
I20260810 04:41:13.089854 26817 raft_consensus.cc:2454] T 5903b9663bd54231a8fa17891266352e P 086ccfa1dd66486bbbcc563857db108b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b3f370bee5b43f4882dbdcfbabc8dad in term 0.
I20260810 04:41:13.089854 26567 raft_consensus.cc:2454] T 5903b9663bd54231a8fa17891266352e P 3202926ff8c141af80ac121604f0dce3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b3f370bee5b43f4882dbdcfbabc8dad in term 0.
I20260810 04:41:13.090020 26191 leader_election.cc:302] T 5903b9663bd54231a8fa17891266352e P 3b3f370bee5b43f4882dbdcfbabc8dad [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3202926ff8c141af80ac121604f0dce3, 3b3f370bee5b43f4882dbdcfbabc8dad; no voters:
I20260810 04:41:13.090154 26941 raft_consensus.cc:2781] T 5903b9663bd54231a8fa17891266352e P 3b3f370bee5b43f4882dbdcfbabc8dad [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 04:41:13.090212 26941 raft_consensus.cc:482] T 5903b9663bd54231a8fa17891266352e P 3b3f370bee5b43f4882dbdcfbabc8dad [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 04:41:13.090226 26941 raft_consensus.cc:3037] T 5903b9663bd54231a8fa17891266352e P 3b3f370bee5b43f4882dbdcfbabc8dad [term 0 FOLLOWER]: Advancing to term 1
I20260810 04:41:13.090905 26941 raft_consensus.cc:504] T 5903b9663bd54231a8fa17891266352e P 3b3f370bee5b43f4882dbdcfbabc8dad [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b3f370bee5b43f4882dbdcfbabc8dad" member_type: VOTER last_known_addr { host: "127.22.167.65" port: 46669 } } peers { permanent_uuid: "3202926ff8c141af80ac121604f0dce3" member_type: VOTER last_known_addr { host: "127.22.167.66" port: 39673 } } peers { permanent_uuid: "086ccfa1dd66486bbbcc563857db108b" member_type: VOTER last_known_addr { host: "127.22.167.67" port: 43761 } }
I20260810 04:41:13.091024 26941 leader_election.cc:288] T 5903b9663bd54231a8fa17891266352e P 3b3f370bee5b43f4882dbdcfbabc8dad [CANDIDATE]: Term 1 election: Requested vote from peers 3202926ff8c141af80ac121604f0dce3 (127.22.167.66:39673), 086ccfa1dd66486bbbcc563857db108b (127.22.167.67:43761)
I20260810 04:41:13.091177 26567 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5903b9663bd54231a8fa17891266352e" candidate_uuid: "3b3f370bee5b43f4882dbdcfbabc8dad" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3202926ff8c141af80ac121604f0dce3"
I20260810 04:41:13.091251 26567 raft_consensus.cc:3037] T 5903b9663bd54231a8fa17891266352e P 3202926ff8c141af80ac121604f0dce3 [term 0 FOLLOWER]: Advancing to term 1
I20260810 04:41:13.091238 26817 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5903b9663bd54231a8fa17891266352e" candidate_uuid: "3b3f370bee5b43f4882dbdcfbabc8dad" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "086ccfa1dd66486bbbcc563857db108b"
I20260810 04:41:13.091315 26817 raft_consensus.cc:3037] T 5903b9663bd54231a8fa17891266352e P 086ccfa1dd66486bbbcc563857db108b [term 0 FOLLOWER]: Advancing to term 1
I20260810 04:41:13.091842 26567 raft_consensus.cc:2454] T 5903b9663bd54231a8fa17891266352e P 3202926ff8c141af80ac121604f0dce3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3b3f370bee5b43f4882dbdcfbabc8dad in term 1.
I20260810 04:41:13.091933 26817 raft_consensus.cc:2454] T 5903b9663bd54231a8fa17891266352e P 086ccfa1dd66486bbbcc563857db108b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3b3f370bee5b43f4882dbdcfbabc8dad in term 1.
I20260810 04:41:13.092016 26191 leader_election.cc:302] T 5903b9663bd54231a8fa17891266352e P 3b3f370bee5b43f4882dbdcfbabc8dad [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3202926ff8c141af80ac121604f0dce3, 3b3f370bee5b43f4882dbdcfbabc8dad; no voters:
I20260810 04:41:13.092105 26941 raft_consensus.cc:2781] T 5903b9663bd54231a8fa17891266352e P 3b3f370bee5b43f4882dbdcfbabc8dad [term 1 FOLLOWER]: Leader election won for term 1
I20260810 04:41:13.092167 26941 raft_consensus.cc:686] T 5903b9663bd54231a8fa17891266352e P 3b3f370bee5b43f4882dbdcfbabc8dad [term 1 LEADER]: Becoming Leader. State: Replica: 3b3f370bee5b43f4882dbdcfbabc8dad, State: Running, Role: LEADER
I20260810 04:41:13.092221 26941 consensus_queue.cc:237] T 5903b9663bd54231a8fa17891266352e P 3b3f370bee5b43f4882dbdcfbabc8dad [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b3f370bee5b43f4882dbdcfbabc8dad" member_type: VOTER last_known_addr { host: "127.22.167.65" port: 46669 } } peers { permanent_uuid: "3202926ff8c141af80ac121604f0dce3" member_type: VOTER last_known_addr { host: "127.22.167.66" port: 39673 } } peers { permanent_uuid: "086ccfa1dd66486bbbcc563857db108b" member_type: VOTER last_known_addr { host: "127.22.167.67" port: 43761 } }
I20260810 04:41:13.092509 26948 tablet_replica.cc:439] T 5903b9663bd54231a8fa17891266352e P 3b3f370bee5b43f4882dbdcfbabc8dad: TxnStatusTablet state changed. Reason: New leader 3b3f370bee5b43f4882dbdcfbabc8dad. Latest consensus state: current_term: 1 leader_uuid: "3b3f370bee5b43f4882dbdcfbabc8dad" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b3f370bee5b43f4882dbdcfbabc8dad" member_type: VOTER last_known_addr { host: "127.22.167.65" port: 46669 } } peers { permanent_uuid: "3202926ff8c141af80ac121604f0dce3" member_type: VOTER last_known_addr { host: "127.22.167.66" port: 39673 } } peers { permanent_uuid: "086ccfa1dd66486bbbcc563857db108b" member_type: VOTER last_known_addr { host: "127.22.167.67" port: 43761 } } }
I20260810 04:41:13.092592 26948 tablet_replica.cc:442] T 5903b9663bd54231a8fa17891266352e P 3b3f370bee5b43f4882dbdcfbabc8dad: This TxnStatusTablet replica's current role is: LEADER
I20260810 04:41:13.092767 26969 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260810 04:41:13.092890 26969 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260810 04:41:13.092831 26050 catalog_manager.cc:5495] T 5903b9663bd54231a8fa17891266352e P 3b3f370bee5b43f4882dbdcfbabc8dad reported cstate change: term changed from 0 to 1, leader changed from <none> to 3b3f370bee5b43f4882dbdcfbabc8dad (127.22.167.65). New cstate: current_term: 1 leader_uuid: "3b3f370bee5b43f4882dbdcfbabc8dad" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b3f370bee5b43f4882dbdcfbabc8dad" member_type: VOTER last_known_addr { host: "127.22.167.65" port: 46669 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3202926ff8c141af80ac121604f0dce3" member_type: VOTER last_known_addr { host: "127.22.167.66" port: 39673 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "086ccfa1dd66486bbbcc563857db108b" member_type: VOTER last_known_addr { host: "127.22.167.67" port: 43761 } health_report { overall_health: UNKNOWN } } }
I20260810 04:41:13.129269 26930 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:41:13.126880 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49782 (local address 127.22.167.65:46669)
0810 04:41:13.126935 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0810 04:41:13.126937 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 04:41:13.126947 (+ 10us) server_negotiation.cc:407] Connection header received
0810 04:41:13.127031 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:41:13.127032 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:41:13.127052 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:41:13.127109 (+ 57us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 04:41:13.127243 (+ 134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:41:13.127759 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:41:13.128499 (+ 740us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:41:13.128700 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:41:13.128756 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:41:13.128940 (+ 184us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 04:41:13.129040 (+ 100us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 04:41:13.129111 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0810 04:41:13.129155 (+ 44us) server_negotiation.cc:299] Negotiation successful
0810 04:41:13.129184 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260810 04:41:13.129273 26956 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:41:13.126818 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.65:46669 (local address 127.0.0.1:49782)
0810 04:41:13.126889 (+ 71us) negotiation.cc:107] Waiting for socket to connect
0810 04:41:13.126900 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0810 04:41:13.126955 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 04:41:13.127125 (+ 170us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 04:41:13.127126 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 04:41:13.127127 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0810 04:41:13.127179 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:41:13.127181 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:41:13.127840 (+ 659us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:41:13.127842 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:41:13.128417 (+ 575us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:41:13.128419 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:41:13.128766 (+ 347us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:41:13.128767 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:41:13.128838 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:41:13.128854 (+ 16us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0810 04:41:13.129100 (+ 246us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0810 04:41:13.129108 (+ 8us) client_negotiation.cc:769] Sending connection context
0810 04:41:13.129152 (+ 44us) client_negotiation.cc:240] Negotiation successful
0810 04:41:13.129202 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":32}
I20260810 04:41:13.131273 26817 raft_consensus.cc:1261] T 5903b9663bd54231a8fa17891266352e P 086ccfa1dd66486bbbcc563857db108b [term 1 FOLLOWER]: Refusing update from remote peer 3b3f370bee5b43f4882dbdcfbabc8dad: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 04:41:13.131341 26567 raft_consensus.cc:1261] T 5903b9663bd54231a8fa17891266352e P 3202926ff8c141af80ac121604f0dce3 [term 1 FOLLOWER]: Refusing update from remote peer 3b3f370bee5b43f4882dbdcfbabc8dad: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 04:41:13.131628 26941 consensus_queue.cc:1035] T 5903b9663bd54231a8fa17891266352e P 3b3f370bee5b43f4882dbdcfbabc8dad [LEADER]: Connected to new peer: Peer: permanent_uuid: "086ccfa1dd66486bbbcc563857db108b" member_type: VOTER last_known_addr { host: "127.22.167.67" port: 43761 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 04:41:13.131759 26948 consensus_queue.cc:1035] T 5903b9663bd54231a8fa17891266352e P 3b3f370bee5b43f4882dbdcfbabc8dad [LEADER]: Connected to new peer: Peer: permanent_uuid: "3202926ff8c141af80ac121604f0dce3" member_type: VOTER last_known_addr { host: "127.22.167.66" port: 39673 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 04:41:13.132956 26938 tablet_replica.cc:439] T 5903b9663bd54231a8fa17891266352e P 3202926ff8c141af80ac121604f0dce3: TxnStatusTablet state changed. Reason: New leader 3b3f370bee5b43f4882dbdcfbabc8dad. Latest consensus state: current_term: 1 leader_uuid: "3b3f370bee5b43f4882dbdcfbabc8dad" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b3f370bee5b43f4882dbdcfbabc8dad" member_type: VOTER last_known_addr { host: "127.22.167.65" port: 46669 } } peers { permanent_uuid: "3202926ff8c141af80ac121604f0dce3" member_type: VOTER last_known_addr { host: "127.22.167.66" port: 39673 } } peers { permanent_uuid: "086ccfa1dd66486bbbcc563857db108b" member_type: VOTER last_known_addr { host: "127.22.167.67" port: 43761 } } }
I20260810 04:41:13.132998 26942 tablet_replica.cc:439] T 5903b9663bd54231a8fa17891266352e P 086ccfa1dd66486bbbcc563857db108b: TxnStatusTablet state changed. Reason: New leader 3b3f370bee5b43f4882dbdcfbabc8dad. Latest consensus state: current_term: 1 leader_uuid: "3b3f370bee5b43f4882dbdcfbabc8dad" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b3f370bee5b43f4882dbdcfbabc8dad" member_type: VOTER last_known_addr { host: "127.22.167.65" port: 46669 } } peers { permanent_uuid: "3202926ff8c141af80ac121604f0dce3" member_type: VOTER last_known_addr { host: "127.22.167.66" port: 39673 } } peers { permanent_uuid: "086ccfa1dd66486bbbcc563857db108b" member_type: VOTER last_known_addr { host: "127.22.167.67" port: 43761 } } }
I20260810 04:41:13.133054 26938 tablet_replica.cc:442] T 5903b9663bd54231a8fa17891266352e P 3202926ff8c141af80ac121604f0dce3: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 04:41:13.133082 26942 tablet_replica.cc:442] T 5903b9663bd54231a8fa17891266352e P 086ccfa1dd66486bbbcc563857db108b: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 04:41:13.134375 26948 tablet_replica.cc:439] T 5903b9663bd54231a8fa17891266352e P 3b3f370bee5b43f4882dbdcfbabc8dad: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "3b3f370bee5b43f4882dbdcfbabc8dad" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b3f370bee5b43f4882dbdcfbabc8dad" member_type: VOTER last_known_addr { host: "127.22.167.65" port: 46669 } } peers { permanent_uuid: "3202926ff8c141af80ac121604f0dce3" member_type: VOTER last_known_addr { host: "127.22.167.66" port: 39673 } } peers { permanent_uuid: "086ccfa1dd66486bbbcc563857db108b" member_type: VOTER last_known_addr { host: "127.22.167.67" port: 43761 } } }
I20260810 04:41:13.134456 26948 tablet_replica.cc:442] T 5903b9663bd54231a8fa17891266352e P 3b3f370bee5b43f4882dbdcfbabc8dad: This TxnStatusTablet replica's current role is: LEADER
I20260810 04:41:13.135084 26942 tablet_replica.cc:439] T 5903b9663bd54231a8fa17891266352e P 086ccfa1dd66486bbbcc563857db108b: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "3b3f370bee5b43f4882dbdcfbabc8dad" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b3f370bee5b43f4882dbdcfbabc8dad" member_type: VOTER last_known_addr { host: "127.22.167.65" port: 46669 } } peers { permanent_uuid: "3202926ff8c141af80ac121604f0dce3" member_type: VOTER last_known_addr { host: "127.22.167.66" port: 39673 } } peers { permanent_uuid: "086ccfa1dd66486bbbcc563857db108b" member_type: VOTER last_known_addr { host: "127.22.167.67" port: 43761 } } }
I20260810 04:41:13.135166 26942 tablet_replica.cc:442] T 5903b9663bd54231a8fa17891266352e P 086ccfa1dd66486bbbcc563857db108b: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 04:41:13.137419 26941 tablet_replica.cc:439] T 5903b9663bd54231a8fa17891266352e P 3b3f370bee5b43f4882dbdcfbabc8dad: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "3b3f370bee5b43f4882dbdcfbabc8dad" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b3f370bee5b43f4882dbdcfbabc8dad" member_type: VOTER last_known_addr { host: "127.22.167.65" port: 46669 } } peers { permanent_uuid: "3202926ff8c141af80ac121604f0dce3" member_type: VOTER last_known_addr { host: "127.22.167.66" port: 39673 } } peers { permanent_uuid: "086ccfa1dd66486bbbcc563857db108b" member_type: VOTER last_known_addr { host: "127.22.167.67" port: 43761 } } }
I20260810 04:41:13.137506 26941 tablet_replica.cc:442] T 5903b9663bd54231a8fa17891266352e P 3b3f370bee5b43f4882dbdcfbabc8dad: This TxnStatusTablet replica's current role is: LEADER
I20260810 04:41:13.137909 26976 mvcc.cc:205] Tried to move back new op lower bound from 7316835832343285760 to 7316835832186101760. Current Snapshot: MvccSnapshot[applied={T|T < 7316835832343285760}]
I20260810 04:41:13.138044 26938 tablet_replica.cc:439] T 5903b9663bd54231a8fa17891266352e P 3202926ff8c141af80ac121604f0dce3: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "3b3f370bee5b43f4882dbdcfbabc8dad" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b3f370bee5b43f4882dbdcfbabc8dad" member_type: VOTER last_known_addr { host: "127.22.167.65" port: 46669 } } peers { permanent_uuid: "3202926ff8c141af80ac121604f0dce3" member_type: VOTER last_known_addr { host: "127.22.167.66" port: 39673 } } peers { permanent_uuid: "086ccfa1dd66486bbbcc563857db108b" member_type: VOTER last_known_addr { host: "127.22.167.67" port: 43761 } } }
I20260810 04:41:13.138118 26938 tablet_replica.cc:442] T 5903b9663bd54231a8fa17891266352e P 3202926ff8c141af80ac121604f0dce3: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 04:41:13.140236 26930 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:41:13.137714 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49790 (local address 127.22.167.65:46669)
0810 04:41:13.137758 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0810 04:41:13.137761 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 04:41:13.137768 (+ 7us) server_negotiation.cc:407] Connection header received
0810 04:41:13.137788 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:41:13.137788 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:41:13.137797 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:41:13.137839 (+ 42us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 04:41:13.138042 (+ 203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:41:13.138686 (+ 644us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:41:13.139633 (+ 947us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:41:13.139825 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:41:13.139876 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:41:13.140022 (+ 146us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 04:41:13.140056 (+ 34us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 04:41:13.140093 (+ 37us) server_negotiation.cc:1035] Waiting for connection context
0810 04:41:13.140144 (+ 51us) server_negotiation.cc:299] Negotiation successful
0810 04:41:13.140171 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260810 04:41:13.145263 26930 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:41:13.142700 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.167.65:51275 (local address 127.22.167.65:46669)
0810 04:41:13.142739 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0810 04:41:13.142741 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 04:41:13.142872 (+ 131us) server_negotiation.cc:407] Connection header received
0810 04:41:13.142920 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:41:13.142921 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:41:13.142928 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:41:13.142983 (+ 55us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 04:41:13.143121 (+ 138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:41:13.143607 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:41:13.144659 (+ 1052us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:41:13.144834 (+ 175us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:41:13.144877 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:41:13.145054 (+ 177us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 04:41:13.145077 (+ 23us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 04:41:13.145105 (+ 28us) server_negotiation.cc:1035] Waiting for connection context
0810 04:41:13.145172 (+ 67us) server_negotiation.cc:299] Negotiation successful
0810 04:41:13.145187 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260810 04:41:13.145488 26988 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:41:13.142661 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.65:46669 (local address 127.22.167.65:51275)
0810 04:41:13.142797 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0810 04:41:13.142808 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0810 04:41:13.142876 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 04:41:13.142988 (+ 112us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 04:41:13.142988 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 04:41:13.142990 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0810 04:41:13.143072 (+ 82us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:41:13.143074 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:41:13.143673 (+ 599us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:41:13.143675 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:41:13.144583 (+ 908us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:41:13.144585 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:41:13.144890 (+ 305us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:41:13.144891 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:41:13.144956 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:41:13.144972 (+ 16us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0810 04:41:13.145118 (+ 146us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0810 04:41:13.145119 (+ 1us) client_negotiation.cc:769] Sending connection context
0810 04:41:13.145165 (+ 46us) client_negotiation.cc:240] Negotiation successful
0810 04:41:13.145205 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":101,"thread_start_us":77,"threads_started":1}
I20260810 04:41:13.147606 26817 raft_consensus.cc:1261] T 0d97a94e9e7d44258b7821928b2ca453 P 086ccfa1dd66486bbbcc563857db108b [term 1 FOLLOWER]: Refusing update from remote peer 3b3f370bee5b43f4882dbdcfbabc8dad: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 04:41:13.147645 26567 raft_consensus.cc:1261] T 0d97a94e9e7d44258b7821928b2ca453 P 3202926ff8c141af80ac121604f0dce3 [term 1 FOLLOWER]: Refusing update from remote peer 3b3f370bee5b43f4882dbdcfbabc8dad: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 04:41:13.147843 26941 consensus_queue.cc:1035] T 0d97a94e9e7d44258b7821928b2ca453 P 3b3f370bee5b43f4882dbdcfbabc8dad [LEADER]: Connected to new peer: Peer: permanent_uuid: "3202926ff8c141af80ac121604f0dce3" member_type: VOTER last_known_addr { host: "127.22.167.66" port: 39673 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 04:41:13.147955 26984 consensus_queue.cc:1035] T 0d97a94e9e7d44258b7821928b2ca453 P 3b3f370bee5b43f4882dbdcfbabc8dad [LEADER]: Connected to new peer: Peer: permanent_uuid: "086ccfa1dd66486bbbcc563857db108b" member_type: VOTER last_known_addr { host: "127.22.167.67" port: 43761 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 04:41:13.149055 26974 mvcc.cc:205] Tried to move back new op lower bound from 7316835832411402240 to 7316835830761115648. Current Snapshot: MvccSnapshot[applied={T|T < 7316835832411402240}]
I20260810 04:41:13.149808 26975 mvcc.cc:205] Tried to move back new op lower bound from 7316835832411402240 to 7316835830761115648. Current Snapshot: MvccSnapshot[applied={T|T < 7316835832411402240}]
I20260810 04:41:13.161592 26050 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41384:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 04:41:13.161705 26050 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41384:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 04:41:13.162444 26050 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 5352e0da235343aaa3a83857cadf08c7: Sending DeleteTablet for 3 replicas of tablet 0d97a94e9e7d44258b7821928b2ca453
I20260810 04:41:13.162751 26525 tablet_service.cc:1514] Processing DeleteTablet for tablet 0d97a94e9e7d44258b7821928b2ca453 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 04:41:13 UTC) from {username='kudu', principal='kudu/127.22.167.126@KRBTEST.COM'} at 127.0.0.1:42740
I20260810 04:41:13.162770 26286 tablet_service.cc:1514] Processing DeleteTablet for tablet 0d97a94e9e7d44258b7821928b2ca453 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 04:41:13 UTC) from {username='kudu', principal='kudu/127.22.167.126@KRBTEST.COM'} at 127.0.0.1:49766
I20260810 04:41:13.162768 26775 tablet_service.cc:1514] Processing DeleteTablet for tablet 0d97a94e9e7d44258b7821928b2ca453 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 04:41:13 UTC) from {username='kudu', principal='kudu/127.22.167.126@KRBTEST.COM'} at 127.0.0.1:41078
I20260810 04:41:13.162982 27000 tablet_replica.cc:330] T 0d97a94e9e7d44258b7821928b2ca453 P 3202926ff8c141af80ac121604f0dce3: stopping tablet replica
I20260810 04:41:13.163086 27000 raft_consensus.cc:2229] T 0d97a94e9e7d44258b7821928b2ca453 P 3202926ff8c141af80ac121604f0dce3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 04:41:13.163147 27000 raft_consensus.cc:2258] T 0d97a94e9e7d44258b7821928b2ca453 P 3202926ff8c141af80ac121604f0dce3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 04:41:13.163296 26997 tablet_replica.cc:330] T 0d97a94e9e7d44258b7821928b2ca453 P 086ccfa1dd66486bbbcc563857db108b: stopping tablet replica
I20260810 04:41:13.163303 26999 tablet_replica.cc:330] T 0d97a94e9e7d44258b7821928b2ca453 P 3b3f370bee5b43f4882dbdcfbabc8dad: stopping tablet replica
I20260810 04:41:13.163394 26999 raft_consensus.cc:2229] T 0d97a94e9e7d44258b7821928b2ca453 P 3b3f370bee5b43f4882dbdcfbabc8dad [term 1 LEADER]: Raft consensus shutting down.
I20260810 04:41:13.163327 26050 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41384:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260810 04:41:13.163401 26997 raft_consensus.cc:2229] T 0d97a94e9e7d44258b7821928b2ca453 P 086ccfa1dd66486bbbcc563857db108b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 04:41:13.163453 26997 raft_consensus.cc:2258] T 0d97a94e9e7d44258b7821928b2ca453 P 086ccfa1dd66486bbbcc563857db108b [term 1 FOLLOWER]: Raft consensus is shut down!
W20260810 04:41:13.163455 26050 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 04:41:13.163475 26999 raft_consensus.cc:2258] T 0d97a94e9e7d44258b7821928b2ca453 P 3b3f370bee5b43f4882dbdcfbabc8dad [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 04:41:13.163892 26997 ts_tablet_manager.cc:1905] T 0d97a94e9e7d44258b7821928b2ca453 P 086ccfa1dd66486bbbcc563857db108b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 04:41:13.164505 26525 tablet_service.cc:1467] Processing CreateTablet for tablet ad4d84976ef4439983dc5e278b9fe217 (DEFAULT_TABLE table=test-table [id=d5ed2ddcbbd440cfac106234c8558bdd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 04:41:13.164551 26999 ts_tablet_manager.cc:1905] T 0d97a94e9e7d44258b7821928b2ca453 P 3b3f370bee5b43f4882dbdcfbabc8dad: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 04:41:13.164618 26525 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ad4d84976ef4439983dc5e278b9fe217. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 04:41:13.165638 26997 ts_tablet_manager.cc:1918] T 0d97a94e9e7d44258b7821928b2ca453 P 086ccfa1dd66486bbbcc563857db108b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260810 04:41:13.165701 26997 log.cc:1196] T 0d97a94e9e7d44258b7821928b2ca453 P 086ccfa1dd66486bbbcc563857db108b: Deleting WAL directory at /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786336863388938-23197-0/minicluster-data/ts-2/wal/wals/0d97a94e9e7d44258b7821928b2ca453
I20260810 04:41:13.165951 26997 ts_tablet_manager.cc:1939] T 0d97a94e9e7d44258b7821928b2ca453 P 086ccfa1dd66486bbbcc563857db108b: Deleting consensus metadata
I20260810 04:41:13.165956 26931 tablet_bootstrap.cc:492] T ad4d84976ef4439983dc5e278b9fe217 P 3202926ff8c141af80ac121604f0dce3: Bootstrap starting.
I20260810 04:41:13.166370 26931 tablet_bootstrap.cc:654] T ad4d84976ef4439983dc5e278b9fe217 P 3202926ff8c141af80ac121604f0dce3: Neither blocks nor log segments found. Creating new log.
I20260810 04:41:13.166385 26999 ts_tablet_manager.cc:1918] T 0d97a94e9e7d44258b7821928b2ca453 P 3b3f370bee5b43f4882dbdcfbabc8dad: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260810 04:41:13.166445 26999 log.cc:1196] T 0d97a94e9e7d44258b7821928b2ca453 P 3b3f370bee5b43f4882dbdcfbabc8dad: Deleting WAL directory at /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786336863388938-23197-0/minicluster-data/ts-0/wal/wals/0d97a94e9e7d44258b7821928b2ca453
I20260810 04:41:13.166706 26999 ts_tablet_manager.cc:1939] T 0d97a94e9e7d44258b7821928b2ca453 P 3b3f370bee5b43f4882dbdcfbabc8dad: Deleting consensus metadata
I20260810 04:41:13.166873 26030 catalog_manager.cc:4841] TS 086ccfa1dd66486bbbcc563857db108b (127.22.167.67:43761): tablet 0d97a94e9e7d44258b7821928b2ca453 (table test-table [id=e583b6d014bf4b6eb133fcbfb430cf5a]) successfully deleted
I20260810 04:41:13.167078 26775 tablet_service.cc:1467] Processing CreateTablet for tablet ad4d84976ef4439983dc5e278b9fe217 (DEFAULT_TABLE table=test-table [id=d5ed2ddcbbd440cfac106234c8558bdd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 04:41:13.167188 26775 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ad4d84976ef4439983dc5e278b9fe217. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 04:41:13.167246 26030 catalog_manager.cc:4841] TS 3b3f370bee5b43f4882dbdcfbabc8dad (127.22.167.65:46669): tablet 0d97a94e9e7d44258b7821928b2ca453 (table test-table [id=e583b6d014bf4b6eb133fcbfb430cf5a]) successfully deleted
I20260810 04:41:13.167277 26286 tablet_service.cc:1467] Processing CreateTablet for tablet ad4d84976ef4439983dc5e278b9fe217 (DEFAULT_TABLE table=test-table [id=d5ed2ddcbbd440cfac106234c8558bdd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 04:41:13.167366 26286 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ad4d84976ef4439983dc5e278b9fe217. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 04:41:13.167526 26931 tablet_bootstrap.cc:492] T ad4d84976ef4439983dc5e278b9fe217 P 3202926ff8c141af80ac121604f0dce3: No bootstrap required, opened a new log
I20260810 04:41:13.167567 26931 ts_tablet_manager.cc:1397] T ad4d84976ef4439983dc5e278b9fe217 P 3202926ff8c141af80ac121604f0dce3: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260810 04:41:13.167694 26931 raft_consensus.cc:348] T ad4d84976ef4439983dc5e278b9fe217 P 3202926ff8c141af80ac121604f0dce3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b3f370bee5b43f4882dbdcfbabc8dad" member_type: VOTER last_known_addr { host: "127.22.167.65" port: 46669 } } peers { permanent_uuid: "3202926ff8c141af80ac121604f0dce3" member_type: VOTER last_known_addr { host: "127.22.167.66" port: 39673 } } peers { permanent_uuid: "086ccfa1dd66486bbbcc563857db108b" member_type: VOTER last_known_addr { host: "127.22.167.67" port: 43761 } }
I20260810 04:41:13.167762 26931 raft_consensus.cc:374] T ad4d84976ef4439983dc5e278b9fe217 P 3202926ff8c141af80ac121604f0dce3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 04:41:13.167783 26931 raft_consensus.cc:729] T ad4d84976ef4439983dc5e278b9fe217 P 3202926ff8c141af80ac121604f0dce3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3202926ff8c141af80ac121604f0dce3, State: Initialized, Role: FOLLOWER
I20260810 04:41:13.167824 26931 consensus_queue.cc:260] T ad4d84976ef4439983dc5e278b9fe217 P 3202926ff8c141af80ac121604f0dce3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b3f370bee5b43f4882dbdcfbabc8dad" member_type: VOTER last_known_addr { host: "127.22.167.65" port: 46669 } } peers { permanent_uuid: "3202926ff8c141af80ac121604f0dce3" member_type: VOTER last_known_addr { host: "127.22.167.66" port: 39673 } } peers { permanent_uuid: "086ccfa1dd66486bbbcc563857db108b" member_type: VOTER last_known_addr { host: "127.22.167.67" port: 43761 } }
I20260810 04:41:13.167904 26931 ts_tablet_manager.cc:1428] T ad4d84976ef4439983dc5e278b9fe217 P 3202926ff8c141af80ac121604f0dce3: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260810 04:41:13.168375 26936 tablet_bootstrap.cc:492] T ad4d84976ef4439983dc5e278b9fe217 P 086ccfa1dd66486bbbcc563857db108b: Bootstrap starting.
I20260810 04:41:13.168569 26932 tablet_bootstrap.cc:492] T ad4d84976ef4439983dc5e278b9fe217 P 3b3f370bee5b43f4882dbdcfbabc8dad: Bootstrap starting.
I20260810 04:41:13.168761 26936 tablet_bootstrap.cc:654] T ad4d84976ef4439983dc5e278b9fe217 P 086ccfa1dd66486bbbcc563857db108b: Neither blocks nor log segments found. Creating new log.
I20260810 04:41:13.168917 26932 tablet_bootstrap.cc:654] T ad4d84976ef4439983dc5e278b9fe217 P 3b3f370bee5b43f4882dbdcfbabc8dad: Neither blocks nor log segments found. Creating new log.
I20260810 04:41:13.169373 26936 tablet_bootstrap.cc:492] T ad4d84976ef4439983dc5e278b9fe217 P 086ccfa1dd66486bbbcc563857db108b: No bootstrap required, opened a new log
I20260810 04:41:13.169425 26936 ts_tablet_manager.cc:1397] T ad4d84976ef4439983dc5e278b9fe217 P 086ccfa1dd66486bbbcc563857db108b: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260810 04:41:13.169555 26936 raft_consensus.cc:348] T ad4d84976ef4439983dc5e278b9fe217 P 086ccfa1dd66486bbbcc563857db108b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b3f370bee5b43f4882dbdcfbabc8dad" member_type: VOTER last_known_addr { host: "127.22.167.65" port: 46669 } } peers { permanent_uuid: "3202926ff8c141af80ac121604f0dce3" member_type: VOTER last_known_addr { host: "127.22.167.66" port: 39673 } } peers { permanent_uuid: "086ccfa1dd66486bbbcc563857db108b" member_type: VOTER last_known_addr { host: "127.22.167.67" port: 43761 } }
I20260810 04:41:13.169607 26936 raft_consensus.cc:374] T ad4d84976ef4439983dc5e278b9fe217 P 086ccfa1dd66486bbbcc563857db108b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 04:41:13.169620 26936 raft_consensus.cc:729] T ad4d84976ef4439983dc5e278b9fe217 P 086ccfa1dd66486bbbcc563857db108b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 086ccfa1dd66486bbbcc563857db108b, State: Initialized, Role: FOLLOWER
I20260810 04:41:13.169648 26936 consensus_queue.cc:260] T ad4d84976ef4439983dc5e278b9fe217 P 086ccfa1dd66486bbbcc563857db108b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b3f370bee5b43f4882dbdcfbabc8dad" member_type: VOTER last_known_addr { host: "127.22.167.65" port: 46669 } } peers { permanent_uuid: "3202926ff8c141af80ac121604f0dce3" member_type: VOTER last_known_addr { host: "127.22.167.66" port: 39673 } } peers { permanent_uuid: "086ccfa1dd66486bbbcc563857db108b" member_type: VOTER last_known_addr { host: "127.22.167.67" port: 43761 } }
I20260810 04:41:13.169745 26936 ts_tablet_manager.cc:1428] T ad4d84976ef4439983dc5e278b9fe217 P 086ccfa1dd66486bbbcc563857db108b: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260810 04:41:13.170042 26932 tablet_bootstrap.cc:492] T ad4d84976ef4439983dc5e278b9fe217 P 3b3f370bee5b43f4882dbdcfbabc8dad: No bootstrap required, opened a new log
I20260810 04:41:13.170094 26932 ts_tablet_manager.cc:1397] T ad4d84976ef4439983dc5e278b9fe217 P 3b3f370bee5b43f4882dbdcfbabc8dad: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260810 04:41:13.170238 26932 raft_consensus.cc:348] T ad4d84976ef4439983dc5e278b9fe217 P 3b3f370bee5b43f4882dbdcfbabc8dad [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b3f370bee5b43f4882dbdcfbabc8dad" member_type: VOTER last_known_addr { host: "127.22.167.65" port: 46669 } } peers { permanent_uuid: "3202926ff8c141af80ac121604f0dce3" member_type: VOTER last_known_addr { host: "127.22.167.66" port: 39673 } } peers { permanent_uuid: "086ccfa1dd66486bbbcc563857db108b" member_type: VOTER last_known_addr { host: "127.22.167.67" port: 43761 } }
I20260810 04:41:13.170293 26932 raft_consensus.cc:374] T ad4d84976ef4439983dc5e278b9fe217 P 3b3f370bee5b43f4882dbdcfbabc8dad [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 04:41:13.170305 26932 raft_consensus.cc:729] T ad4d84976ef4439983dc5e278b9fe217 P 3b3f370bee5b43f4882dbdcfbabc8dad [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3b3f370bee5b43f4882dbdcfbabc8dad, State: Initialized, Role: FOLLOWER
I20260810 04:41:13.170338 26932 consensus_queue.cc:260] T ad4d84976ef4439983dc5e278b9fe217 P 3b3f370bee5b43f4882dbdcfbabc8dad [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b3f370bee5b43f4882dbdcfbabc8dad" member_type: VOTER last_known_addr { host: "127.22.167.65" port: 46669 } } peers { permanent_uuid: "3202926ff8c141af80ac121604f0dce3" member_type: VOTER last_known_addr { host: "127.22.167.66" port: 39673 } } peers { permanent_uuid: "086ccfa1dd66486bbbcc563857db108b" member_type: VOTER last_known_addr { host: "127.22.167.67" port: 43761 } }
I20260810 04:41:13.170415 26932 ts_tablet_manager.cc:1428] T ad4d84976ef4439983dc5e278b9fe217 P 3b3f370bee5b43f4882dbdcfbabc8dad: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260810 04:41:13.174563 27000 ts_tablet_manager.cc:1905] T 0d97a94e9e7d44258b7821928b2ca453 P 3202926ff8c141af80ac121604f0dce3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 04:41:13.175899 27000 ts_tablet_manager.cc:1918] T 0d97a94e9e7d44258b7821928b2ca453 P 3202926ff8c141af80ac121604f0dce3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260810 04:41:13.175966 27000 log.cc:1196] T 0d97a94e9e7d44258b7821928b2ca453 P 3202926ff8c141af80ac121604f0dce3: Deleting WAL directory at /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786336863388938-23197-0/minicluster-data/ts-1/wal/wals/0d97a94e9e7d44258b7821928b2ca453
I20260810 04:41:13.176164 27000 ts_tablet_manager.cc:1939] T 0d97a94e9e7d44258b7821928b2ca453 P 3202926ff8c141af80ac121604f0dce3: Deleting consensus metadata
I20260810 04:41:13.176558 26026 catalog_manager.cc:4841] TS 3202926ff8c141af80ac121604f0dce3 (127.22.167.66:39673): tablet 0d97a94e9e7d44258b7821928b2ca453 (table test-table [id=e583b6d014bf4b6eb133fcbfb430cf5a]) successfully deleted
I20260810 04:41:13.371490 26942 raft_consensus.cc:482] T ad4d84976ef4439983dc5e278b9fe217 P 086ccfa1dd66486bbbcc563857db108b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 04:41:13.371591 26942 raft_consensus.cc:504] T ad4d84976ef4439983dc5e278b9fe217 P 086ccfa1dd66486bbbcc563857db108b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b3f370bee5b43f4882dbdcfbabc8dad" member_type: VOTER last_known_addr { host: "127.22.167.65" port: 46669 } } peers { permanent_uuid: "3202926ff8c141af80ac121604f0dce3" member_type: VOTER last_known_addr { host: "127.22.167.66" port: 39673 } } peers { permanent_uuid: "086ccfa1dd66486bbbcc563857db108b" member_type: VOTER last_known_addr { host: "127.22.167.67" port: 43761 } }
I20260810 04:41:13.371941 26942 leader_election.cc:288] T ad4d84976ef4439983dc5e278b9fe217 P 086ccfa1dd66486bbbcc563857db108b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3b3f370bee5b43f4882dbdcfbabc8dad (127.22.167.65:46669), 3202926ff8c141af80ac121604f0dce3 (127.22.167.66:39673)
I20260810 04:41:13.374805 26930 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:41:13.372221 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.167.67:52139 (local address 127.22.167.65:46669)
0810 04:41:13.372271 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0810 04:41:13.372273 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 04:41:13.372361 (+ 88us) server_negotiation.cc:407] Connection header received
0810 04:41:13.372418 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:41:13.372418 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:41:13.372428 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:41:13.372472 (+ 44us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 04:41:13.372641 (+ 169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:41:13.373111 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:41:13.374118 (+ 1007us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:41:13.374503 (+ 385us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:41:13.374568 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:41:13.374578 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0810 04:41:13.374710 (+ 132us) server_negotiation.cc:299] Negotiation successful
0810 04:41:13.374743 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260810 04:41:13.374822 27007 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:41:13.372124 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.66:39673 (local address 127.22.167.67:53419)
0810 04:41:13.372266 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0810 04:41:13.372276 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0810 04:41:13.372331 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 04:41:13.372498 (+ 167us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 04:41:13.372498 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 04:41:13.372501 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 04:41:13.372590 (+ 89us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:41:13.372591 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:41:13.373228 (+ 637us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:41:13.373229 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:41:13.374017 (+ 788us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:41:13.374019 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:41:13.374562 (+ 543us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:41:13.374563 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:41:13.374633 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:41:13.374636 (+ 3us) client_negotiation.cc:769] Sending connection context
0810 04:41:13.374677 (+ 41us) client_negotiation.cc:240] Negotiation successful
0810 04:41:13.374717 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":128,"mutex_wait_us":25,"thread_start_us":77,"threads_started":1}
I20260810 04:41:13.374817 27008 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:41:13.372205 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.167.67:53419 (local address 127.22.167.66:39673)
0810 04:41:13.372306 (+ 101us) server_negotiation.cc:206] Beginning negotiation
0810 04:41:13.372308 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 04:41:13.372319 (+ 11us) server_negotiation.cc:407] Connection header received
0810 04:41:13.372400 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:41:13.372400 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:41:13.372413 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:41:13.372490 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 04:41:13.372655 (+ 165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:41:13.373144 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:41:13.374100 (+ 956us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:41:13.374485 (+ 385us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:41:13.374552 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:41:13.374561 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0810 04:41:13.374693 (+ 132us) server_negotiation.cc:299] Negotiation successful
0810 04:41:13.374736 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82,"thread_start_us":50,"threads_started":1}
I20260810 04:41:13.374835 27006 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:41:13.372134 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.65:46669 (local address 127.22.167.67:52139)
0810 04:41:13.372278 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0810 04:41:13.372289 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0810 04:41:13.372361 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 04:41:13.372491 (+ 130us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 04:41:13.372492 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 04:41:13.372494 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 04:41:13.372574 (+ 80us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:41:13.372577 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:41:13.373187 (+ 610us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:41:13.373188 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:41:13.374024 (+ 836us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:41:13.374026 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:41:13.374584 (+ 558us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:41:13.374584 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:41:13.374645 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:41:13.374649 (+ 4us) client_negotiation.cc:769] Sending connection context
0810 04:41:13.374698 (+ 49us) client_negotiation.cc:240] Negotiation successful
0810 04:41:13.374737 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":127,"mutex_wait_us":30,"thread_start_us":68,"threads_started":1}
I20260810 04:41:13.375069 26567 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ad4d84976ef4439983dc5e278b9fe217" candidate_uuid: "086ccfa1dd66486bbbcc563857db108b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3202926ff8c141af80ac121604f0dce3" is_pre_election: true
I20260810 04:41:13.375174 26567 raft_consensus.cc:2454] T ad4d84976ef4439983dc5e278b9fe217 P 3202926ff8c141af80ac121604f0dce3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 086ccfa1dd66486bbbcc563857db108b in term 0.
I20260810 04:41:13.375259 26324 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ad4d84976ef4439983dc5e278b9fe217" candidate_uuid: "086ccfa1dd66486bbbcc563857db108b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3b3f370bee5b43f4882dbdcfbabc8dad" is_pre_election: true
I20260810 04:41:13.375341 26695 leader_election.cc:302] T ad4d84976ef4439983dc5e278b9fe217 P 086ccfa1dd66486bbbcc563857db108b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 086ccfa1dd66486bbbcc563857db108b, 3202926ff8c141af80ac121604f0dce3; no voters:
I20260810 04:41:13.375401 26324 raft_consensus.cc:2454] T ad4d84976ef4439983dc5e278b9fe217 P 3b3f370bee5b43f4882dbdcfbabc8dad [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 086ccfa1dd66486bbbcc563857db108b in term 0.
I20260810 04:41:13.375435 26942 raft_consensus.cc:2781] T ad4d84976ef4439983dc5e278b9fe217 P 086ccfa1dd66486bbbcc563857db108b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 04:41:13.375476 26942 raft_consensus.cc:482] T ad4d84976ef4439983dc5e278b9fe217 P 086ccfa1dd66486bbbcc563857db108b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 04:41:13.375491 26942 raft_consensus.cc:3037] T ad4d84976ef4439983dc5e278b9fe217 P 086ccfa1dd66486bbbcc563857db108b [term 0 FOLLOWER]: Advancing to term 1
I20260810 04:41:13.376118 26942 raft_consensus.cc:504] T ad4d84976ef4439983dc5e278b9fe217 P 086ccfa1dd66486bbbcc563857db108b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b3f370bee5b43f4882dbdcfbabc8dad" member_type: VOTER last_known_addr { host: "127.22.167.65" port: 46669 } } peers { permanent_uuid: "3202926ff8c141af80ac121604f0dce3" member_type: VOTER last_known_addr { host: "127.22.167.66" port: 39673 } } peers { permanent_uuid: "086ccfa1dd66486bbbcc563857db108b" member_type: VOTER last_known_addr { host: "127.22.167.67" port: 43761 } }
I20260810 04:41:13.376230 26942 leader_election.cc:288] T ad4d84976ef4439983dc5e278b9fe217 P 086ccfa1dd66486bbbcc563857db108b [CANDIDATE]: Term 1 election: Requested vote from peers 3b3f370bee5b43f4882dbdcfbabc8dad (127.22.167.65:46669), 3202926ff8c141af80ac121604f0dce3 (127.22.167.66:39673)
I20260810 04:41:13.376406 26567 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ad4d84976ef4439983dc5e278b9fe217" candidate_uuid: "086ccfa1dd66486bbbcc563857db108b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3202926ff8c141af80ac121604f0dce3"
I20260810 04:41:13.376405 26324 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ad4d84976ef4439983dc5e278b9fe217" candidate_uuid: "086ccfa1dd66486bbbcc563857db108b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3b3f370bee5b43f4882dbdcfbabc8dad"
I20260810 04:41:13.376478 26567 raft_consensus.cc:3037] T ad4d84976ef4439983dc5e278b9fe217 P 3202926ff8c141af80ac121604f0dce3 [term 0 FOLLOWER]: Advancing to term 1
I20260810 04:41:13.376479 26324 raft_consensus.cc:3037] T ad4d84976ef4439983dc5e278b9fe217 P 3b3f370bee5b43f4882dbdcfbabc8dad [term 0 FOLLOWER]: Advancing to term 1
I20260810 04:41:13.377063 26567 raft_consensus.cc:2454] T ad4d84976ef4439983dc5e278b9fe217 P 3202926ff8c141af80ac121604f0dce3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 086ccfa1dd66486bbbcc563857db108b in term 1.
I20260810 04:41:13.377205 26695 leader_election.cc:302] T ad4d84976ef4439983dc5e278b9fe217 P 086ccfa1dd66486bbbcc563857db108b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 086ccfa1dd66486bbbcc563857db108b, 3202926ff8c141af80ac121604f0dce3; no voters:
I20260810 04:41:13.377280 26942 raft_consensus.cc:2781] T ad4d84976ef4439983dc5e278b9fe217 P 086ccfa1dd66486bbbcc563857db108b [term 1 FOLLOWER]: Leader election won for term 1
I20260810 04:41:13.377288 26324 raft_consensus.cc:2454] T ad4d84976ef4439983dc5e278b9fe217 P 3b3f370bee5b43f4882dbdcfbabc8dad [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 086ccfa1dd66486bbbcc563857db108b in term 1.
I20260810 04:41:13.377377 26942 raft_consensus.cc:686] T ad4d84976ef4439983dc5e278b9fe217 P 086ccfa1dd66486bbbcc563857db108b [term 1 LEADER]: Becoming Leader. State: Replica: 086ccfa1dd66486bbbcc563857db108b, State: Running, Role: LEADER
I20260810 04:41:13.377444 26942 consensus_queue.cc:237] T ad4d84976ef4439983dc5e278b9fe217 P 086ccfa1dd66486bbbcc563857db108b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b3f370bee5b43f4882dbdcfbabc8dad" member_type: VOTER last_known_addr { host: "127.22.167.65" port: 46669 } } peers { permanent_uuid: "3202926ff8c141af80ac121604f0dce3" member_type: VOTER last_known_addr { host: "127.22.167.66" port: 39673 } } peers { permanent_uuid: "086ccfa1dd66486bbbcc563857db108b" member_type: VOTER last_known_addr { host: "127.22.167.67" port: 43761 } }
I20260810 04:41:13.378139 26051 catalog_manager.cc:5495] T ad4d84976ef4439983dc5e278b9fe217 P 086ccfa1dd66486bbbcc563857db108b reported cstate change: term changed from 0 to 1, leader changed from <none> to 086ccfa1dd66486bbbcc563857db108b (127.22.167.67). New cstate: current_term: 1 leader_uuid: "086ccfa1dd66486bbbcc563857db108b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b3f370bee5b43f4882dbdcfbabc8dad" member_type: VOTER last_known_addr { host: "127.22.167.65" port: 46669 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3202926ff8c141af80ac121604f0dce3" member_type: VOTER last_known_addr { host: "127.22.167.66" port: 39673 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "086ccfa1dd66486bbbcc563857db108b" member_type: VOTER last_known_addr { host: "127.22.167.67" port: 43761 } health_report { overall_health: HEALTHY } } }
I20260810 04:41:13.396215 27015 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:41:13.394019 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41094 (local address 127.22.167.67:43761)
0810 04:41:13.394205 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0810 04:41:13.394207 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 04:41:13.394214 (+ 7us) server_negotiation.cc:407] Connection header received
0810 04:41:13.394246 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:41:13.394246 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:41:13.394264 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:41:13.394382 (+ 118us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 04:41:13.394491 (+ 109us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:41:13.394978 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:41:13.395621 (+ 643us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:41:13.395792 (+ 171us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:41:13.395834 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:41:13.395926 (+ 92us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 04:41:13.396027 (+ 101us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 04:41:13.396091 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0810 04:41:13.396104 (+ 13us) server_negotiation.cc:299] Negotiation successful
0810 04:41:13.396120 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":159,"thread_start_us":121,"threads_started":1}
I20260810 04:41:13.400679 26930 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:41:13.398499 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.167.67:43437 (local address 127.22.167.65:46669)
0810 04:41:13.398553 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0810 04:41:13.398581 (+ 28us) server_negotiation.cc:399] Waiting for connection header
0810 04:41:13.398692 (+ 111us) server_negotiation.cc:407] Connection header received
0810 04:41:13.398756 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:41:13.398757 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:41:13.398766 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:41:13.398806 (+ 40us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 04:41:13.398913 (+ 107us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:41:13.399358 (+ 445us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:41:13.400078 (+ 720us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:41:13.400264 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:41:13.400318 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:41:13.400460 (+ 142us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 04:41:13.400488 (+ 28us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 04:41:13.400528 (+ 40us) server_negotiation.cc:1035] Waiting for connection context
0810 04:41:13.400580 (+ 52us) server_negotiation.cc:299] Negotiation successful
0810 04:41:13.400599 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260810 04:41:13.400725 27018 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:41:13.398423 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.65:46669 (local address 127.22.167.67:43437)
0810 04:41:13.398576 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0810 04:41:13.398587 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0810 04:41:13.398696 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 04:41:13.398814 (+ 118us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 04:41:13.398815 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 04:41:13.398816 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0810 04:41:13.398867 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:41:13.398868 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:41:13.399423 (+ 555us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:41:13.399424 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:41:13.400001 (+ 577us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:41:13.400002 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:41:13.400321 (+ 319us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:41:13.400322 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:41:13.400376 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:41:13.400390 (+ 14us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0810 04:41:13.400535 (+ 145us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0810 04:41:13.400536 (+ 1us) client_negotiation.cc:769] Sending connection context
0810 04:41:13.400577 (+ 41us) client_negotiation.cc:240] Negotiation successful
0810 04:41:13.400619 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":75,"threads_started":1}
I20260810 04:41:13.402809 26567 raft_consensus.cc:1261] T ad4d84976ef4439983dc5e278b9fe217 P 3202926ff8c141af80ac121604f0dce3 [term 1 FOLLOWER]: Refusing update from remote peer 086ccfa1dd66486bbbcc563857db108b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 04:41:13.402839 26324 raft_consensus.cc:1261] T ad4d84976ef4439983dc5e278b9fe217 P 3b3f370bee5b43f4882dbdcfbabc8dad [term 1 FOLLOWER]: Refusing update from remote peer 086ccfa1dd66486bbbcc563857db108b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 04:41:13.403075 26942 consensus_queue.cc:1035] T ad4d84976ef4439983dc5e278b9fe217 P 086ccfa1dd66486bbbcc563857db108b [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b3f370bee5b43f4882dbdcfbabc8dad" member_type: VOTER last_known_addr { host: "127.22.167.65" port: 46669 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 04:41:13.403226 27010 consensus_queue.cc:1035] T ad4d84976ef4439983dc5e278b9fe217 P 086ccfa1dd66486bbbcc563857db108b [LEADER]: Connected to new peer: Peer: permanent_uuid: "3202926ff8c141af80ac121604f0dce3" member_type: VOTER last_known_addr { host: "127.22.167.66" port: 39673 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 04:41:13.410007 27015 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:41:13.407961 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.167.65:38283 (local address 127.22.167.67:43761)
0810 04:41:13.408001 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0810 04:41:13.408003 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 04:41:13.408011 (+ 8us) server_negotiation.cc:407] Connection header received
0810 04:41:13.408030 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:41:13.408030 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:41:13.408040 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:41:13.408101 (+ 61us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 04:41:13.408213 (+ 112us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:41:13.408701 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:41:13.409343 (+ 642us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:41:13.409609 (+ 266us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:41:13.409674 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:41:13.409798 (+ 124us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 04:41:13.409831 (+ 33us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 04:41:13.409871 (+ 40us) server_negotiation.cc:1035] Waiting for connection context
0810 04:41:13.409923 (+ 52us) server_negotiation.cc:299] Negotiation successful
0810 04:41:13.409944 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260810 04:41:13.410025 26988 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:41:13.407832 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.67:43761 (local address 127.22.167.65:38283)
0810 04:41:13.407875 (+ 43us) negotiation.cc:107] Waiting for socket to connect
0810 04:41:13.407884 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0810 04:41:13.407915 (+ 31us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 04:41:13.408113 (+ 198us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 04:41:13.408114 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 04:41:13.408115 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0810 04:41:13.408156 (+ 41us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:41:13.408157 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:41:13.408759 (+ 602us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:41:13.408760 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:41:13.409291 (+ 531us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:41:13.409292 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:41:13.409674 (+ 382us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:41:13.409675 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:41:13.409735 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:41:13.409741 (+ 6us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0810 04:41:13.409872 (+ 131us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0810 04:41:13.409874 (+ 2us) client_negotiation.cc:769] Sending connection context
0810 04:41:13.409920 (+ 46us) client_negotiation.cc:240] Negotiation successful
0810 04:41:13.409963 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":26}
W20260810 04:41:13.410362 26975 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260810 04:41:14.929023 26050 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41384:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 04:41:14.929134 26050 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41384:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 04:41:14.929917 26050 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 5352e0da235343aaa3a83857cadf08c7: Sending DeleteTablet for 3 replicas of tablet ad4d84976ef4439983dc5e278b9fe217
I20260810 04:41:14.930189 26286 tablet_service.cc:1514] Processing DeleteTablet for tablet ad4d84976ef4439983dc5e278b9fe217 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 04:41:14 UTC) from {username='kudu', principal='kudu/127.22.167.126@KRBTEST.COM'} at 127.0.0.1:49766
I20260810 04:41:14.930240 26775 tablet_service.cc:1514] Processing DeleteTablet for tablet ad4d84976ef4439983dc5e278b9fe217 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 04:41:14 UTC) from {username='kudu', principal='kudu/127.22.167.126@KRBTEST.COM'} at 127.0.0.1:41078
I20260810 04:41:14.930401 27075 tablet_replica.cc:330] T ad4d84976ef4439983dc5e278b9fe217 P 086ccfa1dd66486bbbcc563857db108b: stopping tablet replica
I20260810 04:41:14.930452 27075 raft_consensus.cc:2229] T ad4d84976ef4439983dc5e278b9fe217 P 086ccfa1dd66486bbbcc563857db108b [term 1 LEADER]: Raft consensus shutting down.
I20260810 04:41:14.930546 27075 raft_consensus.cc:2258] T ad4d84976ef4439983dc5e278b9fe217 P 086ccfa1dd66486bbbcc563857db108b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 04:41:14.930775 27075 ts_tablet_manager.cc:1905] T ad4d84976ef4439983dc5e278b9fe217 P 086ccfa1dd66486bbbcc563857db108b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 04:41:14.931986 26525 tablet_service.cc:1514] Processing DeleteTablet for tablet ad4d84976ef4439983dc5e278b9fe217 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 04:41:14 UTC) from {username='kudu', principal='kudu/127.22.167.126@KRBTEST.COM'} at 127.0.0.1:42740
I20260810 04:41:14.932215 27081 tablet_replica.cc:330] T ad4d84976ef4439983dc5e278b9fe217 P 3202926ff8c141af80ac121604f0dce3: stopping tablet replica
I20260810 04:41:14.932379 27075 ts_tablet_manager.cc:1918] T ad4d84976ef4439983dc5e278b9fe217 P 086ccfa1dd66486bbbcc563857db108b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260810 04:41:14.932430 27075 log.cc:1196] T ad4d84976ef4439983dc5e278b9fe217 P 086ccfa1dd66486bbbcc563857db108b: Deleting WAL directory at /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786336863388938-23197-0/minicluster-data/ts-2/wal/wals/ad4d84976ef4439983dc5e278b9fe217
I20260810 04:41:14.932660 27075 ts_tablet_manager.cc:1939] T ad4d84976ef4439983dc5e278b9fe217 P 086ccfa1dd66486bbbcc563857db108b: Deleting consensus metadata
I20260810 04:41:14.932862 27081 raft_consensus.cc:2229] T ad4d84976ef4439983dc5e278b9fe217 P 3202926ff8c141af80ac121604f0dce3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 04:41:14.932946 27081 raft_consensus.cc:2258] T ad4d84976ef4439983dc5e278b9fe217 P 3202926ff8c141af80ac121604f0dce3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 04:41:14.932951 26030 catalog_manager.cc:4841] TS 086ccfa1dd66486bbbcc563857db108b (127.22.167.67:43761): tablet ad4d84976ef4439983dc5e278b9fe217 (table test-table [id=d5ed2ddcbbd440cfac106234c8558bdd]) successfully deleted
I20260810 04:41:14.933187 27081 ts_tablet_manager.cc:1905] T ad4d84976ef4439983dc5e278b9fe217 P 3202926ff8c141af80ac121604f0dce3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 04:41:14.933430 27074 tablet_replica.cc:330] T ad4d84976ef4439983dc5e278b9fe217 P 3b3f370bee5b43f4882dbdcfbabc8dad: stopping tablet replica
I20260810 04:41:14.933497 27074 raft_consensus.cc:2229] T ad4d84976ef4439983dc5e278b9fe217 P 3b3f370bee5b43f4882dbdcfbabc8dad [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 04:41:14.933560 27074 raft_consensus.cc:2258] T ad4d84976ef4439983dc5e278b9fe217 P 3b3f370bee5b43f4882dbdcfbabc8dad [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 04:41:14.933800 27074 ts_tablet_manager.cc:1905] T ad4d84976ef4439983dc5e278b9fe217 P 3b3f370bee5b43f4882dbdcfbabc8dad: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 04:41:14.934681 27081 ts_tablet_manager.cc:1918] T ad4d84976ef4439983dc5e278b9fe217 P 3202926ff8c141af80ac121604f0dce3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260810 04:41:14.934737 27081 log.cc:1196] T ad4d84976ef4439983dc5e278b9fe217 P 3202926ff8c141af80ac121604f0dce3: Deleting WAL directory at /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786336863388938-23197-0/minicluster-data/ts-1/wal/wals/ad4d84976ef4439983dc5e278b9fe217
I20260810 04:41:14.934959 27081 ts_tablet_manager.cc:1939] T ad4d84976ef4439983dc5e278b9fe217 P 3202926ff8c141af80ac121604f0dce3: Deleting consensus metadata
I20260810 04:41:14.935042 27074 ts_tablet_manager.cc:1918] T ad4d84976ef4439983dc5e278b9fe217 P 3b3f370bee5b43f4882dbdcfbabc8dad: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260810 04:41:14.935104 27074 log.cc:1196] T ad4d84976ef4439983dc5e278b9fe217 P 3b3f370bee5b43f4882dbdcfbabc8dad: Deleting WAL directory at /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786336863388938-23197-0/minicluster-data/ts-0/wal/wals/ad4d84976ef4439983dc5e278b9fe217
I20260810 04:41:14.935322 27074 ts_tablet_manager.cc:1939] T ad4d84976ef4439983dc5e278b9fe217 P 3b3f370bee5b43f4882dbdcfbabc8dad: Deleting consensus metadata
I20260810 04:41:14.935317 26026 catalog_manager.cc:4841] TS 3202926ff8c141af80ac121604f0dce3 (127.22.167.66:39673): tablet ad4d84976ef4439983dc5e278b9fe217 (table test-table [id=d5ed2ddcbbd440cfac106234c8558bdd]) successfully deleted
I20260810 04:41:14.935647 26030 catalog_manager.cc:4841] TS 3b3f370bee5b43f4882dbdcfbabc8dad (127.22.167.65:46669): tablet ad4d84976ef4439983dc5e278b9fe217 (table test-table [id=d5ed2ddcbbd440cfac106234c8558bdd]) successfully deleted
Aug 10 04:41:14 dist-test-slave-bqcl krb5kdc[25977](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786336874, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 10 04:41:14 dist-test-slave-bqcl krb5kdc[25977](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786336874, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.22.167.126@KRBTEST.COM
I20260810 04:41:14.957628 27092 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:41:14.947774 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41402 (local address 127.22.167.126:37755)
0810 04:41:14.947917 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0810 04:41:14.947919 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 04:41:14.947927 (+ 8us) server_negotiation.cc:407] Connection header received
0810 04:41:14.947951 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:41:14.947951 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:41:14.947962 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:41:14.948038 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0810 04:41:14.948529 (+ 491us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:41:14.949066 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:41:14.949753 (+ 687us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:41:14.949945 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:41:14.950002 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:41:14.955382 (+ 5380us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 04:41:14.955384 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 04:41:14.955384 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 04:41:14.955392 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0810 04:41:14.956764 (+ 1372us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 04:41:14.957063 (+ 299us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 04:41:14.957064 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 04:41:14.957064 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 04:41:14.957121 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 04:41:14.957246 (+ 125us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 04:41:14.957246 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 04:41:14.957247 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 04:41:14.957349 (+ 102us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 04:41:14.957423 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0810 04:41:14.957489 (+ 66us) server_negotiation.cc:299] Negotiation successful
0810 04:41:14.957555 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":112,"thread_start_us":78,"threads_started":1}
I20260810 04:41:14.959453 26050 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:41402:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 04:41:14.959645 26050 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 04:41:14.961010 26525 tablet_service.cc:1467] Processing CreateTablet for tablet 6c1d701c0fd84895a289e4d25b7ae7e2 (DEFAULT_TABLE table=test-table [id=53c06075f86542f684d6116dc2ff00f6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 04:41:14.961009 26286 tablet_service.cc:1467] Processing CreateTablet for tablet 6c1d701c0fd84895a289e4d25b7ae7e2 (DEFAULT_TABLE table=test-table [id=53c06075f86542f684d6116dc2ff00f6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 04:41:14.961004 26775 tablet_service.cc:1467] Processing CreateTablet for tablet 6c1d701c0fd84895a289e4d25b7ae7e2 (DEFAULT_TABLE table=test-table [id=53c06075f86542f684d6116dc2ff00f6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 04:41:14.961151 26775 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6c1d701c0fd84895a289e4d25b7ae7e2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 04:41:14.961153 26286 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6c1d701c0fd84895a289e4d25b7ae7e2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 04:41:14.961153 26525 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6c1d701c0fd84895a289e4d25b7ae7e2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 04:41:14.962474 27095 tablet_bootstrap.cc:492] T 6c1d701c0fd84895a289e4d25b7ae7e2 P 086ccfa1dd66486bbbcc563857db108b: Bootstrap starting.
I20260810 04:41:14.962590 27096 tablet_bootstrap.cc:492] T 6c1d701c0fd84895a289e4d25b7ae7e2 P 3202926ff8c141af80ac121604f0dce3: Bootstrap starting.
I20260810 04:41:14.962774 27098 tablet_bootstrap.cc:492] T 6c1d701c0fd84895a289e4d25b7ae7e2 P 3b3f370bee5b43f4882dbdcfbabc8dad: Bootstrap starting.
I20260810 04:41:14.962888 27095 tablet_bootstrap.cc:654] T 6c1d701c0fd84895a289e4d25b7ae7e2 P 086ccfa1dd66486bbbcc563857db108b: Neither blocks nor log segments found. Creating new log.
I20260810 04:41:14.962965 27096 tablet_bootstrap.cc:654] T 6c1d701c0fd84895a289e4d25b7ae7e2 P 3202926ff8c141af80ac121604f0dce3: Neither blocks nor log segments found. Creating new log.
I20260810 04:41:14.963155 27098 tablet_bootstrap.cc:654] T 6c1d701c0fd84895a289e4d25b7ae7e2 P 3b3f370bee5b43f4882dbdcfbabc8dad: Neither blocks nor log segments found. Creating new log.
I20260810 04:41:14.963544 27096 tablet_bootstrap.cc:492] T 6c1d701c0fd84895a289e4d25b7ae7e2 P 3202926ff8c141af80ac121604f0dce3: No bootstrap required, opened a new log
I20260810 04:41:14.963546 27095 tablet_bootstrap.cc:492] T 6c1d701c0fd84895a289e4d25b7ae7e2 P 086ccfa1dd66486bbbcc563857db108b: No bootstrap required, opened a new log
I20260810 04:41:14.963608 27095 ts_tablet_manager.cc:1397] T 6c1d701c0fd84895a289e4d25b7ae7e2 P 086ccfa1dd66486bbbcc563857db108b: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260810 04:41:14.963608 27096 ts_tablet_manager.cc:1397] T 6c1d701c0fd84895a289e4d25b7ae7e2 P 3202926ff8c141af80ac121604f0dce3: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260810 04:41:14.963735 27095 raft_consensus.cc:348] T 6c1d701c0fd84895a289e4d25b7ae7e2 P 086ccfa1dd66486bbbcc563857db108b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b3f370bee5b43f4882dbdcfbabc8dad" member_type: VOTER last_known_addr { host: "127.22.167.65" port: 46669 } } peers { permanent_uuid: "3202926ff8c141af80ac121604f0dce3" member_type: VOTER last_known_addr { host: "127.22.167.66" port: 39673 } } peers { permanent_uuid: "086ccfa1dd66486bbbcc563857db108b" member_type: VOTER last_known_addr { host: "127.22.167.67" port: 43761 } }
I20260810 04:41:14.963735 27096 raft_consensus.cc:348] T 6c1d701c0fd84895a289e4d25b7ae7e2 P 3202926ff8c141af80ac121604f0dce3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b3f370bee5b43f4882dbdcfbabc8dad" member_type: VOTER last_known_addr { host: "127.22.167.65" port: 46669 } } peers { permanent_uuid: "3202926ff8c141af80ac121604f0dce3" member_type: VOTER last_known_addr { host: "127.22.167.66" port: 39673 } } peers { permanent_uuid: "086ccfa1dd66486bbbcc563857db108b" member_type: VOTER last_known_addr { host: "127.22.167.67" port: 43761 } }
I20260810 04:41:14.963781 27098 tablet_bootstrap.cc:492] T 6c1d701c0fd84895a289e4d25b7ae7e2 P 3b3f370bee5b43f4882dbdcfbabc8dad: No bootstrap required, opened a new log
I20260810 04:41:14.963804 27095 raft_consensus.cc:374] T 6c1d701c0fd84895a289e4d25b7ae7e2 P 086ccfa1dd66486bbbcc563857db108b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 04:41:14.963804 27096 raft_consensus.cc:374] T 6c1d701c0fd84895a289e4d25b7ae7e2 P 3202926ff8c141af80ac121604f0dce3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 04:41:14.963819 27098 ts_tablet_manager.cc:1397] T 6c1d701c0fd84895a289e4d25b7ae7e2 P 3b3f370bee5b43f4882dbdcfbabc8dad: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260810 04:41:14.963827 27095 raft_consensus.cc:729] T 6c1d701c0fd84895a289e4d25b7ae7e2 P 086ccfa1dd66486bbbcc563857db108b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 086ccfa1dd66486bbbcc563857db108b, State: Initialized, Role: FOLLOWER
I20260810 04:41:14.963827 27096 raft_consensus.cc:729] T 6c1d701c0fd84895a289e4d25b7ae7e2 P 3202926ff8c141af80ac121604f0dce3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3202926ff8c141af80ac121604f0dce3, State: Initialized, Role: FOLLOWER
I20260810 04:41:14.963879 27096 consensus_queue.cc:260] T 6c1d701c0fd84895a289e4d25b7ae7e2 P 3202926ff8c141af80ac121604f0dce3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b3f370bee5b43f4882dbdcfbabc8dad" member_type: VOTER last_known_addr { host: "127.22.167.65" port: 46669 } } peers { permanent_uuid: "3202926ff8c141af80ac121604f0dce3" member_type: VOTER last_known_addr { host: "127.22.167.66" port: 39673 } } peers { permanent_uuid: "086ccfa1dd66486bbbcc563857db108b" member_type: VOTER last_known_addr { host: "127.22.167.67" port: 43761 } }
I20260810 04:41:14.963883 27095 consensus_queue.cc:260] T 6c1d701c0fd84895a289e4d25b7ae7e2 P 086ccfa1dd66486bbbcc563857db108b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b3f370bee5b43f4882dbdcfbabc8dad" member_type: VOTER last_known_addr { host: "127.22.167.65" port: 46669 } } peers { permanent_uuid: "3202926ff8c141af80ac121604f0dce3" member_type: VOTER last_known_addr { host: "127.22.167.66" port: 39673 } } peers { permanent_uuid: "086ccfa1dd66486bbbcc563857db108b" member_type: VOTER last_known_addr { host: "127.22.167.67" port: 43761 } }
I20260810 04:41:14.963908 27098 raft_consensus.cc:348] T 6c1d701c0fd84895a289e4d25b7ae7e2 P 3b3f370bee5b43f4882dbdcfbabc8dad [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b3f370bee5b43f4882dbdcfbabc8dad" member_type: VOTER last_known_addr { host: "127.22.167.65" port: 46669 } } peers { permanent_uuid: "3202926ff8c141af80ac121604f0dce3" member_type: VOTER last_known_addr { host: "127.22.167.66" port: 39673 } } peers { permanent_uuid: "086ccfa1dd66486bbbcc563857db108b" member_type: VOTER last_known_addr { host: "127.22.167.67" port: 43761 } }
I20260810 04:41:14.963969 27098 raft_consensus.cc:374] T 6c1d701c0fd84895a289e4d25b7ae7e2 P 3b3f370bee5b43f4882dbdcfbabc8dad [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 04:41:14.963986 27098 raft_consensus.cc:729] T 6c1d701c0fd84895a289e4d25b7ae7e2 P 3b3f370bee5b43f4882dbdcfbabc8dad [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3b3f370bee5b43f4882dbdcfbabc8dad, State: Initialized, Role: FOLLOWER
I20260810 04:41:14.963990 27095 ts_tablet_manager.cc:1428] T 6c1d701c0fd84895a289e4d25b7ae7e2 P 086ccfa1dd66486bbbcc563857db108b: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260810 04:41:14.964046 27098 consensus_queue.cc:260] T 6c1d701c0fd84895a289e4d25b7ae7e2 P 3b3f370bee5b43f4882dbdcfbabc8dad [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b3f370bee5b43f4882dbdcfbabc8dad" member_type: VOTER last_known_addr { host: "127.22.167.65" port: 46669 } } peers { permanent_uuid: "3202926ff8c141af80ac121604f0dce3" member_type: VOTER last_known_addr { host: "127.22.167.66" port: 39673 } } peers { permanent_uuid: "086ccfa1dd66486bbbcc563857db108b" member_type: VOTER last_known_addr { host: "127.22.167.67" port: 43761 } }
I20260810 04:41:14.964128 27098 ts_tablet_manager.cc:1428] T 6c1d701c0fd84895a289e4d25b7ae7e2 P 3b3f370bee5b43f4882dbdcfbabc8dad: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260810 04:41:14.964242 27096 ts_tablet_manager.cc:1428] T 6c1d701c0fd84895a289e4d25b7ae7e2 P 3202926ff8c141af80ac121604f0dce3: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260810 04:41:14.999189 27062 raft_consensus.cc:482] T 6c1d701c0fd84895a289e4d25b7ae7e2 P 086ccfa1dd66486bbbcc563857db108b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 04:41:14.999298 27062 raft_consensus.cc:504] T 6c1d701c0fd84895a289e4d25b7ae7e2 P 086ccfa1dd66486bbbcc563857db108b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b3f370bee5b43f4882dbdcfbabc8dad" member_type: VOTER last_known_addr { host: "127.22.167.65" port: 46669 } } peers { permanent_uuid: "3202926ff8c141af80ac121604f0dce3" member_type: VOTER last_known_addr { host: "127.22.167.66" port: 39673 } } peers { permanent_uuid: "086ccfa1dd66486bbbcc563857db108b" member_type: VOTER last_known_addr { host: "127.22.167.67" port: 43761 } }
I20260810 04:41:14.999467 27062 leader_election.cc:288] T 6c1d701c0fd84895a289e4d25b7ae7e2 P 086ccfa1dd66486bbbcc563857db108b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3b3f370bee5b43f4882dbdcfbabc8dad (127.22.167.65:46669), 3202926ff8c141af80ac121604f0dce3 (127.22.167.66:39673)
I20260810 04:41:14.999663 26567 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6c1d701c0fd84895a289e4d25b7ae7e2" candidate_uuid: "086ccfa1dd66486bbbcc563857db108b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3202926ff8c141af80ac121604f0dce3" is_pre_election: true
I20260810 04:41:14.999686 26324 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6c1d701c0fd84895a289e4d25b7ae7e2" candidate_uuid: "086ccfa1dd66486bbbcc563857db108b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3b3f370bee5b43f4882dbdcfbabc8dad" is_pre_election: true
I20260810 04:41:14.999773 26324 raft_consensus.cc:2454] T 6c1d701c0fd84895a289e4d25b7ae7e2 P 3b3f370bee5b43f4882dbdcfbabc8dad [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 086ccfa1dd66486bbbcc563857db108b in term 0.
I20260810 04:41:14.999773 26567 raft_consensus.cc:2454] T 6c1d701c0fd84895a289e4d25b7ae7e2 P 3202926ff8c141af80ac121604f0dce3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 086ccfa1dd66486bbbcc563857db108b in term 0.
I20260810 04:41:14.999925 26695 leader_election.cc:302] T 6c1d701c0fd84895a289e4d25b7ae7e2 P 086ccfa1dd66486bbbcc563857db108b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 086ccfa1dd66486bbbcc563857db108b, 3202926ff8c141af80ac121604f0dce3; no voters:
I20260810 04:41:15.000015 27062 raft_consensus.cc:2781] T 6c1d701c0fd84895a289e4d25b7ae7e2 P 086ccfa1dd66486bbbcc563857db108b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 04:41:15.000069 27062 raft_consensus.cc:482] T 6c1d701c0fd84895a289e4d25b7ae7e2 P 086ccfa1dd66486bbbcc563857db108b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 04:41:15.000090 27062 raft_consensus.cc:3037] T 6c1d701c0fd84895a289e4d25b7ae7e2 P 086ccfa1dd66486bbbcc563857db108b [term 0 FOLLOWER]: Advancing to term 1
I20260810 04:41:15.000799 27062 raft_consensus.cc:504] T 6c1d701c0fd84895a289e4d25b7ae7e2 P 086ccfa1dd66486bbbcc563857db108b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b3f370bee5b43f4882dbdcfbabc8dad" member_type: VOTER last_known_addr { host: "127.22.167.65" port: 46669 } } peers { permanent_uuid: "3202926ff8c141af80ac121604f0dce3" member_type: VOTER last_known_addr { host: "127.22.167.66" port: 39673 } } peers { permanent_uuid: "086ccfa1dd66486bbbcc563857db108b" member_type: VOTER last_known_addr { host: "127.22.167.67" port: 43761 } }
I20260810 04:41:15.000910 27062 leader_election.cc:288] T 6c1d701c0fd84895a289e4d25b7ae7e2 P 086ccfa1dd66486bbbcc563857db108b [CANDIDATE]: Term 1 election: Requested vote from peers 3b3f370bee5b43f4882dbdcfbabc8dad (127.22.167.65:46669), 3202926ff8c141af80ac121604f0dce3 (127.22.167.66:39673)
I20260810 04:41:15.001044 26324 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6c1d701c0fd84895a289e4d25b7ae7e2" candidate_uuid: "086ccfa1dd66486bbbcc563857db108b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3b3f370bee5b43f4882dbdcfbabc8dad"
I20260810 04:41:15.001128 26324 raft_consensus.cc:3037] T 6c1d701c0fd84895a289e4d25b7ae7e2 P 3b3f370bee5b43f4882dbdcfbabc8dad [term 0 FOLLOWER]: Advancing to term 1
I20260810 04:41:15.001080 26567 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6c1d701c0fd84895a289e4d25b7ae7e2" candidate_uuid: "086ccfa1dd66486bbbcc563857db108b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3202926ff8c141af80ac121604f0dce3"
I20260810 04:41:15.001163 26567 raft_consensus.cc:3037] T 6c1d701c0fd84895a289e4d25b7ae7e2 P 3202926ff8c141af80ac121604f0dce3 [term 0 FOLLOWER]: Advancing to term 1
I20260810 04:41:15.001742 26324 raft_consensus.cc:2454] T 6c1d701c0fd84895a289e4d25b7ae7e2 P 3b3f370bee5b43f4882dbdcfbabc8dad [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 086ccfa1dd66486bbbcc563857db108b in term 1.
I20260810 04:41:15.001760 26567 raft_consensus.cc:2454] T 6c1d701c0fd84895a289e4d25b7ae7e2 P 3202926ff8c141af80ac121604f0dce3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 086ccfa1dd66486bbbcc563857db108b in term 1.
I20260810 04:41:15.001919 26695 leader_election.cc:302] T 6c1d701c0fd84895a289e4d25b7ae7e2 P 086ccfa1dd66486bbbcc563857db108b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 086ccfa1dd66486bbbcc563857db108b, 3202926ff8c141af80ac121604f0dce3; no voters:
I20260810 04:41:15.002043 27062 raft_consensus.cc:2781] T 6c1d701c0fd84895a289e4d25b7ae7e2 P 086ccfa1dd66486bbbcc563857db108b [term 1 FOLLOWER]: Leader election won for term 1
I20260810 04:41:15.002115 27062 raft_consensus.cc:686] T 6c1d701c0fd84895a289e4d25b7ae7e2 P 086ccfa1dd66486bbbcc563857db108b [term 1 LEADER]: Becoming Leader. State: Replica: 086ccfa1dd66486bbbcc563857db108b, State: Running, Role: LEADER
I20260810 04:41:15.002199 27062 consensus_queue.cc:237] T 6c1d701c0fd84895a289e4d25b7ae7e2 P 086ccfa1dd66486bbbcc563857db108b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b3f370bee5b43f4882dbdcfbabc8dad" member_type: VOTER last_known_addr { host: "127.22.167.65" port: 46669 } } peers { permanent_uuid: "3202926ff8c141af80ac121604f0dce3" member_type: VOTER last_known_addr { host: "127.22.167.66" port: 39673 } } peers { permanent_uuid: "086ccfa1dd66486bbbcc563857db108b" member_type: VOTER last_known_addr { host: "127.22.167.67" port: 43761 } }
I20260810 04:41:15.002753 26051 catalog_manager.cc:5495] T 6c1d701c0fd84895a289e4d25b7ae7e2 P 086ccfa1dd66486bbbcc563857db108b reported cstate change: term changed from 0 to 1, leader changed from <none> to 086ccfa1dd66486bbbcc563857db108b (127.22.167.67). New cstate: current_term: 1 leader_uuid: "086ccfa1dd66486bbbcc563857db108b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b3f370bee5b43f4882dbdcfbabc8dad" member_type: VOTER last_known_addr { host: "127.22.167.65" port: 46669 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3202926ff8c141af80ac121604f0dce3" member_type: VOTER last_known_addr { host: "127.22.167.66" port: 39673 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "086ccfa1dd66486bbbcc563857db108b" member_type: VOTER last_known_addr { host: "127.22.167.67" port: 43761 } health_report { overall_health: HEALTHY } } }
I20260810 04:41:15.012437 27107 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:41:15.010078 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41106 (local address 127.22.167.67:43761)
0810 04:41:15.010274 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0810 04:41:15.010275 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0810 04:41:15.010282 (+ 7us) server_negotiation.cc:407] Connection header received
0810 04:41:15.010299 (+ 17us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:41:15.010299 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:41:15.010309 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:41:15.010479 (+ 170us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 04:41:15.010609 (+ 130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:41:15.011087 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:41:15.011826 (+ 739us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:41:15.012025 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:41:15.012079 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:41:15.012216 (+ 137us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 04:41:15.012246 (+ 30us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 04:41:15.012287 (+ 41us) server_negotiation.cc:1035] Waiting for connection context
0810 04:41:15.012335 (+ 48us) server_negotiation.cc:299] Negotiation successful
0810 04:41:15.012357 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":168,"thread_start_us":129,"threads_started":1}
I20260810 04:41:15.014269 26324 raft_consensus.cc:1261] T 6c1d701c0fd84895a289e4d25b7ae7e2 P 3b3f370bee5b43f4882dbdcfbabc8dad [term 1 FOLLOWER]: Refusing update from remote peer 086ccfa1dd66486bbbcc563857db108b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 04:41:15.014297 26567 raft_consensus.cc:1261] T 6c1d701c0fd84895a289e4d25b7ae7e2 P 3202926ff8c141af80ac121604f0dce3 [term 1 FOLLOWER]: Refusing update from remote peer 086ccfa1dd66486bbbcc563857db108b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 04:41:15.014469 27010 consensus_queue.cc:1035] T 6c1d701c0fd84895a289e4d25b7ae7e2 P 086ccfa1dd66486bbbcc563857db108b [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b3f370bee5b43f4882dbdcfbabc8dad" member_type: VOTER last_known_addr { host: "127.22.167.65" port: 46669 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 04:41:15.014586 27062 consensus_queue.cc:1035] T 6c1d701c0fd84895a289e4d25b7ae7e2 P 086ccfa1dd66486bbbcc563857db108b [LEADER]: Connected to new peer: Peer: permanent_uuid: "3202926ff8c141af80ac121604f0dce3" member_type: VOTER last_known_addr { host: "127.22.167.66" port: 39673 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 04:41:15.023751 26051 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:41402:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 04:41:15.023845 26051 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:41402:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 04:41:15.024379 26051 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 5352e0da235343aaa3a83857cadf08c7: Sending DeleteTablet for 3 replicas of tablet 6c1d701c0fd84895a289e4d25b7ae7e2
I20260810 04:41:15.024587 26286 tablet_service.cc:1514] Processing DeleteTablet for tablet 6c1d701c0fd84895a289e4d25b7ae7e2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 04:41:15 UTC) from {username='kudu', principal='kudu/127.22.167.126@KRBTEST.COM'} at 127.0.0.1:49766
I20260810 04:41:15.024610 26775 tablet_service.cc:1514] Processing DeleteTablet for tablet 6c1d701c0fd84895a289e4d25b7ae7e2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 04:41:15 UTC) from {username='kudu', principal='kudu/127.22.167.126@KRBTEST.COM'} at 127.0.0.1:41078
I20260810 04:41:15.024693 26525 tablet_service.cc:1514] Processing DeleteTablet for tablet 6c1d701c0fd84895a289e4d25b7ae7e2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 04:41:15 UTC) from {username='kudu', principal='kudu/127.22.167.126@KRBTEST.COM'} at 127.0.0.1:42740
I20260810 04:41:15.024708 27075 tablet_replica.cc:330] T 6c1d701c0fd84895a289e4d25b7ae7e2 P 086ccfa1dd66486bbbcc563857db108b: stopping tablet replica
I20260810 04:41:15.024770 27075 raft_consensus.cc:2229] T 6c1d701c0fd84895a289e4d25b7ae7e2 P 086ccfa1dd66486bbbcc563857db108b [term 1 LEADER]: Raft consensus shutting down.
I20260810 04:41:15.024783 27074 tablet_replica.cc:330] T 6c1d701c0fd84895a289e4d25b7ae7e2 P 3b3f370bee5b43f4882dbdcfbabc8dad: stopping tablet replica
I20260810 04:41:15.024782 27081 tablet_replica.cc:330] T 6c1d701c0fd84895a289e4d25b7ae7e2 P 3202926ff8c141af80ac121604f0dce3: stopping tablet replica
I20260810 04:41:15.024825 27075 raft_consensus.cc:2258] T 6c1d701c0fd84895a289e4d25b7ae7e2 P 086ccfa1dd66486bbbcc563857db108b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 04:41:15.024835 27081 raft_consensus.cc:2229] T 6c1d701c0fd84895a289e4d25b7ae7e2 P 3202926ff8c141af80ac121604f0dce3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 04:41:15.024839 27074 raft_consensus.cc:2229] T 6c1d701c0fd84895a289e4d25b7ae7e2 P 3b3f370bee5b43f4882dbdcfbabc8dad [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 04:41:15.024869 27081 raft_consensus.cc:2258] T 6c1d701c0fd84895a289e4d25b7ae7e2 P 3202926ff8c141af80ac121604f0dce3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 04:41:15.024876 27074 raft_consensus.cc:2258] T 6c1d701c0fd84895a289e4d25b7ae7e2 P 3b3f370bee5b43f4882dbdcfbabc8dad [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 04:41:15.025157 27074 ts_tablet_manager.cc:1905] T 6c1d701c0fd84895a289e4d25b7ae7e2 P 3b3f370bee5b43f4882dbdcfbabc8dad: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 04:41:15.025135 26051 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:41402:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260810 04:41:15.025204 27081 ts_tablet_manager.cc:1905] T 6c1d701c0fd84895a289e4d25b7ae7e2 P 3202926ff8c141af80ac121604f0dce3: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260810 04:41:15.025271 26051 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 04:41:15.026355 26525 tablet_service.cc:1467] Processing CreateTablet for tablet ba02f39d58584f29b8904c72aea5cc60 (DEFAULT_TABLE table=test-table [id=e3fbf3a8c6944627b8e975fa6fdff31d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 04:41:15.026355 26775 tablet_service.cc:1467] Processing CreateTablet for tablet ba02f39d58584f29b8904c72aea5cc60 (DEFAULT_TABLE table=test-table [id=e3fbf3a8c6944627b8e975fa6fdff31d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 04:41:15.026463 26525 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ba02f39d58584f29b8904c72aea5cc60. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 04:41:15.026463 26775 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ba02f39d58584f29b8904c72aea5cc60. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 04:41:15.026794 27075 ts_tablet_manager.cc:1905] T 6c1d701c0fd84895a289e4d25b7ae7e2 P 086ccfa1dd66486bbbcc563857db108b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 04:41:15.027244 27081 ts_tablet_manager.cc:1918] T 6c1d701c0fd84895a289e4d25b7ae7e2 P 3202926ff8c141af80ac121604f0dce3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260810 04:41:15.027294 27081 log.cc:1196] T 6c1d701c0fd84895a289e4d25b7ae7e2 P 3202926ff8c141af80ac121604f0dce3: Deleting WAL directory at /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786336863388938-23197-0/minicluster-data/ts-1/wal/wals/6c1d701c0fd84895a289e4d25b7ae7e2
I20260810 04:41:15.027494 27081 ts_tablet_manager.cc:1939] T 6c1d701c0fd84895a289e4d25b7ae7e2 P 3202926ff8c141af80ac121604f0dce3: Deleting consensus metadata
I20260810 04:41:15.027789 27095 tablet_bootstrap.cc:492] T ba02f39d58584f29b8904c72aea5cc60 P 086ccfa1dd66486bbbcc563857db108b: Bootstrap starting.
I20260810 04:41:15.027877 27075 ts_tablet_manager.cc:1918] T 6c1d701c0fd84895a289e4d25b7ae7e2 P 086ccfa1dd66486bbbcc563857db108b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260810 04:41:15.027892 26286 tablet_service.cc:1467] Processing CreateTablet for tablet ba02f39d58584f29b8904c72aea5cc60 (DEFAULT_TABLE table=test-table [id=e3fbf3a8c6944627b8e975fa6fdff31d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 04:41:15.027926 27075 log.cc:1196] T 6c1d701c0fd84895a289e4d25b7ae7e2 P 086ccfa1dd66486bbbcc563857db108b: Deleting WAL directory at /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786336863388938-23197-0/minicluster-data/ts-2/wal/wals/6c1d701c0fd84895a289e4d25b7ae7e2
I20260810 04:41:15.028100 27095 tablet_bootstrap.cc:654] T ba02f39d58584f29b8904c72aea5cc60 P 086ccfa1dd66486bbbcc563857db108b: Neither blocks nor log segments found. Creating new log.
I20260810 04:41:15.028100 27075 ts_tablet_manager.cc:1939] T 6c1d701c0fd84895a289e4d25b7ae7e2 P 086ccfa1dd66486bbbcc563857db108b: Deleting consensus metadata
I20260810 04:41:15.028496 26030 catalog_manager.cc:4841] TS 086ccfa1dd66486bbbcc563857db108b (127.22.167.67:43761): tablet 6c1d701c0fd84895a289e4d25b7ae7e2 (table test-table [id=53c06075f86542f684d6116dc2ff00f6]) successfully deleted
I20260810 04:41:15.028733 27095 tablet_bootstrap.cc:492] T ba02f39d58584f29b8904c72aea5cc60 P 086ccfa1dd66486bbbcc563857db108b: No bootstrap required, opened a new log
I20260810 04:41:15.028775 27095 ts_tablet_manager.cc:1397] T ba02f39d58584f29b8904c72aea5cc60 P 086ccfa1dd66486bbbcc563857db108b: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260810 04:41:15.028865 27095 raft_consensus.cc:348] T ba02f39d58584f29b8904c72aea5cc60 P 086ccfa1dd66486bbbcc563857db108b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b3f370bee5b43f4882dbdcfbabc8dad" member_type: VOTER last_known_addr { host: "127.22.167.65" port: 46669 } } peers { permanent_uuid: "3202926ff8c141af80ac121604f0dce3" member_type: VOTER last_known_addr { host: "127.22.167.66" port: 39673 } } peers { permanent_uuid: "086ccfa1dd66486bbbcc563857db108b" member_type: VOTER last_known_addr { host: "127.22.167.67" port: 43761 } }
I20260810 04:41:15.028924 27095 raft_consensus.cc:374] T ba02f39d58584f29b8904c72aea5cc60 P 086ccfa1dd66486bbbcc563857db108b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 04:41:15.028941 27095 raft_consensus.cc:729] T ba02f39d58584f29b8904c72aea5cc60 P 086ccfa1dd66486bbbcc563857db108b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 086ccfa1dd66486bbbcc563857db108b, State: Initialized, Role: FOLLOWER
I20260810 04:41:15.028959 27096 tablet_bootstrap.cc:492] T ba02f39d58584f29b8904c72aea5cc60 P 3202926ff8c141af80ac121604f0dce3: Bootstrap starting.
I20260810 04:41:15.028978 27095 consensus_queue.cc:260] T ba02f39d58584f29b8904c72aea5cc60 P 086ccfa1dd66486bbbcc563857db108b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b3f370bee5b43f4882dbdcfbabc8dad" member_type: VOTER last_known_addr { host: "127.22.167.65" port: 46669 } } peers { permanent_uuid: "3202926ff8c141af80ac121604f0dce3" member_type: VOTER last_known_addr { host: "127.22.167.66" port: 39673 } } peers { permanent_uuid: "086ccfa1dd66486bbbcc563857db108b" member_type: VOTER last_known_addr { host: "127.22.167.67" port: 43761 } }
I20260810 04:41:15.029065 27095 ts_tablet_manager.cc:1428] T ba02f39d58584f29b8904c72aea5cc60 P 086ccfa1dd66486bbbcc563857db108b: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260810 04:41:15.029048 26026 catalog_manager.cc:4841] TS 3202926ff8c141af80ac121604f0dce3 (127.22.167.66:39673): tablet 6c1d701c0fd84895a289e4d25b7ae7e2 (table test-table [id=53c06075f86542f684d6116dc2ff00f6]) successfully deleted
I20260810 04:41:15.029217 27074 ts_tablet_manager.cc:1918] T 6c1d701c0fd84895a289e4d25b7ae7e2 P 3b3f370bee5b43f4882dbdcfbabc8dad: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260810 04:41:15.029258 27074 log.cc:1196] T 6c1d701c0fd84895a289e4d25b7ae7e2 P 3b3f370bee5b43f4882dbdcfbabc8dad: Deleting WAL directory at /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786336863388938-23197-0/minicluster-data/ts-0/wal/wals/6c1d701c0fd84895a289e4d25b7ae7e2
I20260810 04:41:15.029340 27096 tablet_bootstrap.cc:654] T ba02f39d58584f29b8904c72aea5cc60 P 3202926ff8c141af80ac121604f0dce3: Neither blocks nor log segments found. Creating new log.
I20260810 04:41:15.029452 27074 ts_tablet_manager.cc:1939] T 6c1d701c0fd84895a289e4d25b7ae7e2 P 3b3f370bee5b43f4882dbdcfbabc8dad: Deleting consensus metadata
I20260810 04:41:15.029582 26286 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ba02f39d58584f29b8904c72aea5cc60. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 04:41:15.029779 26030 catalog_manager.cc:4841] TS 3b3f370bee5b43f4882dbdcfbabc8dad (127.22.167.65:46669): tablet 6c1d701c0fd84895a289e4d25b7ae7e2 (table test-table [id=53c06075f86542f684d6116dc2ff00f6]) successfully deleted
I20260810 04:41:15.030161 27096 tablet_bootstrap.cc:492] T ba02f39d58584f29b8904c72aea5cc60 P 3202926ff8c141af80ac121604f0dce3: No bootstrap required, opened a new log
I20260810 04:41:15.030215 27096 ts_tablet_manager.cc:1397] T ba02f39d58584f29b8904c72aea5cc60 P 3202926ff8c141af80ac121604f0dce3: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260810 04:41:15.030347 27096 raft_consensus.cc:348] T ba02f39d58584f29b8904c72aea5cc60 P 3202926ff8c141af80ac121604f0dce3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b3f370bee5b43f4882dbdcfbabc8dad" member_type: VOTER last_known_addr { host: "127.22.167.65" port: 46669 } } peers { permanent_uuid: "3202926ff8c141af80ac121604f0dce3" member_type: VOTER last_known_addr { host: "127.22.167.66" port: 39673 } } peers { permanent_uuid: "086ccfa1dd66486bbbcc563857db108b" member_type: VOTER last_known_addr { host: "127.22.167.67" port: 43761 } }
I20260810 04:41:15.030400 27096 raft_consensus.cc:374] T ba02f39d58584f29b8904c72aea5cc60 P 3202926ff8c141af80ac121604f0dce3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 04:41:15.030412 27096 raft_consensus.cc:729] T ba02f39d58584f29b8904c72aea5cc60 P 3202926ff8c141af80ac121604f0dce3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3202926ff8c141af80ac121604f0dce3, State: Initialized, Role: FOLLOWER
I20260810 04:41:15.030442 27096 consensus_queue.cc:260] T ba02f39d58584f29b8904c72aea5cc60 P 3202926ff8c141af80ac121604f0dce3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b3f370bee5b43f4882dbdcfbabc8dad" member_type: VOTER last_known_addr { host: "127.22.167.65" port: 46669 } } peers { permanent_uuid: "3202926ff8c141af80ac121604f0dce3" member_type: VOTER last_known_addr { host: "127.22.167.66" port: 39673 } } peers { permanent_uuid: "086ccfa1dd66486bbbcc563857db108b" member_type: VOTER last_known_addr { host: "127.22.167.67" port: 43761 } }
I20260810 04:41:15.030527 27096 ts_tablet_manager.cc:1428] T ba02f39d58584f29b8904c72aea5cc60 P 3202926ff8c141af80ac121604f0dce3: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260810 04:41:15.030717 27098 tablet_bootstrap.cc:492] T ba02f39d58584f29b8904c72aea5cc60 P 3b3f370bee5b43f4882dbdcfbabc8dad: Bootstrap starting.
I20260810 04:41:15.031036 27098 tablet_bootstrap.cc:654] T ba02f39d58584f29b8904c72aea5cc60 P 3b3f370bee5b43f4882dbdcfbabc8dad: Neither blocks nor log segments found. Creating new log.
I20260810 04:41:15.031544 27098 tablet_bootstrap.cc:492] T ba02f39d58584f29b8904c72aea5cc60 P 3b3f370bee5b43f4882dbdcfbabc8dad: No bootstrap required, opened a new log
I20260810 04:41:15.031589 27098 ts_tablet_manager.cc:1397] T ba02f39d58584f29b8904c72aea5cc60 P 3b3f370bee5b43f4882dbdcfbabc8dad: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260810 04:41:15.031687 27098 raft_consensus.cc:348] T ba02f39d58584f29b8904c72aea5cc60 P 3b3f370bee5b43f4882dbdcfbabc8dad [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b3f370bee5b43f4882dbdcfbabc8dad" member_type: VOTER last_known_addr { host: "127.22.167.65" port: 46669 } } peers { permanent_uuid: "3202926ff8c141af80ac121604f0dce3" member_type: VOTER last_known_addr { host: "127.22.167.66" port: 39673 } } peers { permanent_uuid: "086ccfa1dd66486bbbcc563857db108b" member_type: VOTER last_known_addr { host: "127.22.167.67" port: 43761 } }
I20260810 04:41:15.031744 27098 raft_consensus.cc:374] T ba02f39d58584f29b8904c72aea5cc60 P 3b3f370bee5b43f4882dbdcfbabc8dad [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 04:41:15.031764 27098 raft_consensus.cc:729] T ba02f39d58584f29b8904c72aea5cc60 P 3b3f370bee5b43f4882dbdcfbabc8dad [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3b3f370bee5b43f4882dbdcfbabc8dad, State: Initialized, Role: FOLLOWER
I20260810 04:41:15.031808 27098 consensus_queue.cc:260] T ba02f39d58584f29b8904c72aea5cc60 P 3b3f370bee5b43f4882dbdcfbabc8dad [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b3f370bee5b43f4882dbdcfbabc8dad" member_type: VOTER last_known_addr { host: "127.22.167.65" port: 46669 } } peers { permanent_uuid: "3202926ff8c141af80ac121604f0dce3" member_type: VOTER last_known_addr { host: "127.22.167.66" port: 39673 } } peers { permanent_uuid: "086ccfa1dd66486bbbcc563857db108b" member_type: VOTER last_known_addr { host: "127.22.167.67" port: 43761 } }
I20260810 04:41:15.031898 27098 ts_tablet_manager.cc:1428] T ba02f39d58584f29b8904c72aea5cc60 P 3b3f370bee5b43f4882dbdcfbabc8dad: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260810 04:41:15.229265 27062 raft_consensus.cc:482] T ba02f39d58584f29b8904c72aea5cc60 P 086ccfa1dd66486bbbcc563857db108b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 04:41:15.229377 27062 raft_consensus.cc:504] T ba02f39d58584f29b8904c72aea5cc60 P 086ccfa1dd66486bbbcc563857db108b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b3f370bee5b43f4882dbdcfbabc8dad" member_type: VOTER last_known_addr { host: "127.22.167.65" port: 46669 } } peers { permanent_uuid: "3202926ff8c141af80ac121604f0dce3" member_type: VOTER last_known_addr { host: "127.22.167.66" port: 39673 } } peers { permanent_uuid: "086ccfa1dd66486bbbcc563857db108b" member_type: VOTER last_known_addr { host: "127.22.167.67" port: 43761 } }
I20260810 04:41:15.229519 27062 leader_election.cc:288] T ba02f39d58584f29b8904c72aea5cc60 P 086ccfa1dd66486bbbcc563857db108b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3b3f370bee5b43f4882dbdcfbabc8dad (127.22.167.65:46669), 3202926ff8c141af80ac121604f0dce3 (127.22.167.66:39673)
I20260810 04:41:15.229745 26567 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ba02f39d58584f29b8904c72aea5cc60" candidate_uuid: "086ccfa1dd66486bbbcc563857db108b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3202926ff8c141af80ac121604f0dce3" is_pre_election: true
I20260810 04:41:15.229764 26324 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ba02f39d58584f29b8904c72aea5cc60" candidate_uuid: "086ccfa1dd66486bbbcc563857db108b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3b3f370bee5b43f4882dbdcfbabc8dad" is_pre_election: true
I20260810 04:41:15.229842 26567 raft_consensus.cc:2454] T ba02f39d58584f29b8904c72aea5cc60 P 3202926ff8c141af80ac121604f0dce3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 086ccfa1dd66486bbbcc563857db108b in term 0.
I20260810 04:41:15.229863 26324 raft_consensus.cc:2454] T ba02f39d58584f29b8904c72aea5cc60 P 3b3f370bee5b43f4882dbdcfbabc8dad [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 086ccfa1dd66486bbbcc563857db108b in term 0.
I20260810 04:41:15.230021 26695 leader_election.cc:302] T ba02f39d58584f29b8904c72aea5cc60 P 086ccfa1dd66486bbbcc563857db108b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 086ccfa1dd66486bbbcc563857db108b, 3202926ff8c141af80ac121604f0dce3; no voters:
I20260810 04:41:15.230157 27062 raft_consensus.cc:2781] T ba02f39d58584f29b8904c72aea5cc60 P 086ccfa1dd66486bbbcc563857db108b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 04:41:15.230222 27062 raft_consensus.cc:482] T ba02f39d58584f29b8904c72aea5cc60 P 086ccfa1dd66486bbbcc563857db108b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 04:41:15.230235 27062 raft_consensus.cc:3037] T ba02f39d58584f29b8904c72aea5cc60 P 086ccfa1dd66486bbbcc563857db108b [term 0 FOLLOWER]: Advancing to term 1
I20260810 04:41:15.230922 27062 raft_consensus.cc:504] T ba02f39d58584f29b8904c72aea5cc60 P 086ccfa1dd66486bbbcc563857db108b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b3f370bee5b43f4882dbdcfbabc8dad" member_type: VOTER last_known_addr { host: "127.22.167.65" port: 46669 } } peers { permanent_uuid: "3202926ff8c141af80ac121604f0dce3" member_type: VOTER last_known_addr { host: "127.22.167.66" port: 39673 } } peers { permanent_uuid: "086ccfa1dd66486bbbcc563857db108b" member_type: VOTER last_known_addr { host: "127.22.167.67" port: 43761 } }
I20260810 04:41:15.231048 27062 leader_election.cc:288] T ba02f39d58584f29b8904c72aea5cc60 P 086ccfa1dd66486bbbcc563857db108b [CANDIDATE]: Term 1 election: Requested vote from peers 3b3f370bee5b43f4882dbdcfbabc8dad (127.22.167.65:46669), 3202926ff8c141af80ac121604f0dce3 (127.22.167.66:39673)
I20260810 04:41:15.231210 26324 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ba02f39d58584f29b8904c72aea5cc60" candidate_uuid: "086ccfa1dd66486bbbcc563857db108b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3b3f370bee5b43f4882dbdcfbabc8dad"
I20260810 04:41:15.231249 26567 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ba02f39d58584f29b8904c72aea5cc60" candidate_uuid: "086ccfa1dd66486bbbcc563857db108b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3202926ff8c141af80ac121604f0dce3"
I20260810 04:41:15.231297 26324 raft_consensus.cc:3037] T ba02f39d58584f29b8904c72aea5cc60 P 3b3f370bee5b43f4882dbdcfbabc8dad [term 0 FOLLOWER]: Advancing to term 1
I20260810 04:41:15.231312 26567 raft_consensus.cc:3037] T ba02f39d58584f29b8904c72aea5cc60 P 3202926ff8c141af80ac121604f0dce3 [term 0 FOLLOWER]: Advancing to term 1
I20260810 04:41:15.232034 26567 raft_consensus.cc:2454] T ba02f39d58584f29b8904c72aea5cc60 P 3202926ff8c141af80ac121604f0dce3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 086ccfa1dd66486bbbcc563857db108b in term 1.
I20260810 04:41:15.232069 26324 raft_consensus.cc:2454] T ba02f39d58584f29b8904c72aea5cc60 P 3b3f370bee5b43f4882dbdcfbabc8dad [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 086ccfa1dd66486bbbcc563857db108b in term 1.
I20260810 04:41:15.232209 26701 leader_election.cc:302] T ba02f39d58584f29b8904c72aea5cc60 P 086ccfa1dd66486bbbcc563857db108b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 086ccfa1dd66486bbbcc563857db108b, 3b3f370bee5b43f4882dbdcfbabc8dad; no voters:
I20260810 04:41:15.232342 27062 raft_consensus.cc:2781] T ba02f39d58584f29b8904c72aea5cc60 P 086ccfa1dd66486bbbcc563857db108b [term 1 FOLLOWER]: Leader election won for term 1
I20260810 04:41:15.232406 27062 raft_consensus.cc:686] T ba02f39d58584f29b8904c72aea5cc60 P 086ccfa1dd66486bbbcc563857db108b [term 1 LEADER]: Becoming Leader. State: Replica: 086ccfa1dd66486bbbcc563857db108b, State: Running, Role: LEADER
I20260810 04:41:15.232460 27062 consensus_queue.cc:237] T ba02f39d58584f29b8904c72aea5cc60 P 086ccfa1dd66486bbbcc563857db108b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b3f370bee5b43f4882dbdcfbabc8dad" member_type: VOTER last_known_addr { host: "127.22.167.65" port: 46669 } } peers { permanent_uuid: "3202926ff8c141af80ac121604f0dce3" member_type: VOTER last_known_addr { host: "127.22.167.66" port: 39673 } } peers { permanent_uuid: "086ccfa1dd66486bbbcc563857db108b" member_type: VOTER last_known_addr { host: "127.22.167.67" port: 43761 } }
I20260810 04:41:15.233007 26050 catalog_manager.cc:5495] T ba02f39d58584f29b8904c72aea5cc60 P 086ccfa1dd66486bbbcc563857db108b reported cstate change: term changed from 0 to 1, leader changed from <none> to 086ccfa1dd66486bbbcc563857db108b (127.22.167.67). New cstate: current_term: 1 leader_uuid: "086ccfa1dd66486bbbcc563857db108b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b3f370bee5b43f4882dbdcfbabc8dad" member_type: VOTER last_known_addr { host: "127.22.167.65" port: 46669 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3202926ff8c141af80ac121604f0dce3" member_type: VOTER last_known_addr { host: "127.22.167.66" port: 39673 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "086ccfa1dd66486bbbcc563857db108b" member_type: VOTER last_known_addr { host: "127.22.167.67" port: 43761 } health_report { overall_health: HEALTHY } } }
I20260810 04:41:15.259408 26324 raft_consensus.cc:1261] T ba02f39d58584f29b8904c72aea5cc60 P 3b3f370bee5b43f4882dbdcfbabc8dad [term 1 FOLLOWER]: Refusing update from remote peer 086ccfa1dd66486bbbcc563857db108b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 04:41:15.259423 26567 raft_consensus.cc:1261] T ba02f39d58584f29b8904c72aea5cc60 P 3202926ff8c141af80ac121604f0dce3 [term 1 FOLLOWER]: Refusing update from remote peer 086ccfa1dd66486bbbcc563857db108b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 04:41:15.259635 27062 consensus_queue.cc:1035] T ba02f39d58584f29b8904c72aea5cc60 P 086ccfa1dd66486bbbcc563857db108b [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b3f370bee5b43f4882dbdcfbabc8dad" member_type: VOTER last_known_addr { host: "127.22.167.65" port: 46669 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 04:41:15.259761 27114 consensus_queue.cc:1035] T ba02f39d58584f29b8904c72aea5cc60 P 086ccfa1dd66486bbbcc563857db108b [LEADER]: Connected to new peer: Peer: permanent_uuid: "3202926ff8c141af80ac121604f0dce3" member_type: VOTER last_known_addr { host: "127.22.167.66" port: 39673 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260810 04:41:15.264192 27045 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260810 04:41:16.780756 26048 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:41402:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 04:41:16.780862 26048 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:41402:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 04:41:16.781643 26048 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 5352e0da235343aaa3a83857cadf08c7: Sending DeleteTablet for 3 replicas of tablet ba02f39d58584f29b8904c72aea5cc60
I20260810 04:41:16.781924 26289 tablet_service.cc:1514] Processing DeleteTablet for tablet ba02f39d58584f29b8904c72aea5cc60 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 04:41:16 UTC) from {username='kudu', principal='kudu/127.22.167.126@KRBTEST.COM'} at 127.0.0.1:49766
I20260810 04:41:16.782032 26525 tablet_service.cc:1514] Processing DeleteTablet for tablet ba02f39d58584f29b8904c72aea5cc60 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 04:41:16 UTC) from {username='kudu', principal='kudu/127.22.167.126@KRBTEST.COM'} at 127.0.0.1:42740
I20260810 04:41:16.782117 23197 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskemv_AM/build/release/bin/kudu with pid 26169
I20260810 04:41:16.782116 26775 tablet_service.cc:1514] Processing DeleteTablet for tablet ba02f39d58584f29b8904c72aea5cc60 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 04:41:16 UTC) from {username='kudu', principal='kudu/127.22.167.126@KRBTEST.COM'} at 127.0.0.1:41078
I20260810 04:41:16.785085 27156 tablet_replica.cc:330] T ba02f39d58584f29b8904c72aea5cc60 P 3202926ff8c141af80ac121604f0dce3: stopping tablet replica
I20260810 04:41:16.785156 27156 raft_consensus.cc:2229] T ba02f39d58584f29b8904c72aea5cc60 P 3202926ff8c141af80ac121604f0dce3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 04:41:16.785192 27156 raft_consensus.cc:2258] T ba02f39d58584f29b8904c72aea5cc60 P 3202926ff8c141af80ac121604f0dce3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 04:41:16.785416 27156 ts_tablet_manager.cc:1905] T ba02f39d58584f29b8904c72aea5cc60 P 3202926ff8c141af80ac121604f0dce3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 04:41:16.792271 27159 tablet_replica.cc:330] T ba02f39d58584f29b8904c72aea5cc60 P 086ccfa1dd66486bbbcc563857db108b: stopping tablet replica
I20260810 04:41:16.792351 27159 raft_consensus.cc:2229] T ba02f39d58584f29b8904c72aea5cc60 P 086ccfa1dd66486bbbcc563857db108b [term 1 LEADER]: Raft consensus shutting down.
I20260810 04:41:16.792412 27159 raft_consensus.cc:2258] T ba02f39d58584f29b8904c72aea5cc60 P 086ccfa1dd66486bbbcc563857db108b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 04:41:16.792689 27159 ts_tablet_manager.cc:1905] T ba02f39d58584f29b8904c72aea5cc60 P 086ccfa1dd66486bbbcc563857db108b: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260810 04:41:16.797809 26030 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.22.167.65:46669 (error 108)
I20260810 04:41:16.798085 23197 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskemv_AM/build/release/bin/kudu with pid 26421
I20260810 04:41:16.800432 27159 ts_tablet_manager.cc:1918] T ba02f39d58584f29b8904c72aea5cc60 P 086ccfa1dd66486bbbcc563857db108b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260810 04:41:16.800490 27159 log.cc:1196] T ba02f39d58584f29b8904c72aea5cc60 P 086ccfa1dd66486bbbcc563857db108b: Deleting WAL directory at /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786336863388938-23197-0/minicluster-data/ts-2/wal/wals/ba02f39d58584f29b8904c72aea5cc60
I20260810 04:41:16.800725 27159 ts_tablet_manager.cc:1939] T ba02f39d58584f29b8904c72aea5cc60 P 086ccfa1dd66486bbbcc563857db108b: Deleting consensus metadata
I20260810 04:41:16.802088 27163 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:41:16.798290 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.65:46669 (local address 127.0.0.1:53952)
0810 04:41:16.801984 (+ 3694us) negotiation.cc:107] Waiting for socket to connect
0810 04:41:16.802047 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.167.65:46669: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":3667,"thread_start_us":68,"threads_started":1}
W20260810 04:41:16.802248 26030 catalog_manager.cc:4584] TS 3b3f370bee5b43f4882dbdcfbabc8dad (127.22.167.65:46669): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet ba02f39d58584f29b8904c72aea5cc60: Network error: Client connection negotiation failed: client connection to 127.22.167.65:46669: connect: Connection refused (error 111)
I20260810 04:41:16.811290 26030 catalog_manager.cc:4841] TS 086ccfa1dd66486bbbcc563857db108b (127.22.167.67:43761): tablet ba02f39d58584f29b8904c72aea5cc60 (table test-table [id=e3fbf3a8c6944627b8e975fa6fdff31d]) successfully deleted
I20260810 04:41:16.813817 27163 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:41:16.813697 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.66:39673 (local address 127.0.0.1:33782)
0810 04:41:16.813747 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0810 04:41:16.813784 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.167.66:39673: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
I20260810 04:41:16.813814 23197 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskemv_AM/build/release/bin/kudu with pid 26664
I20260810 04:41:16.821977 23197 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskemv_AM/build/release/bin/kudu with pid 26004
2026-08-10T04:41:16Z chronyd exiting
[ OK ] SecurityITest.TxnSmokeWithDifferentUserTypes (6108 ms)
[ RUN ] SecurityITest.TestNoKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786336863388938-23197-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 04:41:17 dist-test-slave-bqcl krb5kdc[27167](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 04:41:17 dist-test-slave-bqcl krb5kdc[27167](info): set up 2 sockets
Aug 10 04:41:17 dist-test-slave-bqcl krb5kdc[27167](info): commencing operation
krb5kdc: starting...
W20260810 04:41:19.071936 23197 mini_kdc.cc:121] Time spent starting KDC: real 2.222s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786336863388938-23197-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786336863388938-23197-0/krb5kdc/test-user.keytab.
Aug 10 04:41:19 dist-test-slave-bqcl krb5kdc[27167](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786336879, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T04:41:19Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T04:41:19Z Disabled control of system clock
WARNING: no policy specified for kudu/127.22.167.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.167.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.167.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786336863388938-23197-0/krb5kdc/kudu_127.22.167.126.keytab.
Entry for principal kudu/127.22.167.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786336863388938-23197-0/krb5kdc/kudu_127.22.167.126.keytab.
I20260810 04:41:19.186553 23197 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskemv_AM/build/release/bin/kudu
/tmp/dist-test-taskemv_AM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786336863388938-23197-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786336863388938-23197-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786336863388938-23197-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786336863388938-23197-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.167.126:42075
--webserver_interface=127.22.167.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786336863388938-23197-0/krb5kdc/kudu_127.22.167.126.keytab
--principal=kudu/127.22.167.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.167.84:34579
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.167.126:42075
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786336863388938-23197-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786336863388938-23197-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786336863388938-23197-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 04:41:19.268623 27188 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 04:41:19.268832 27188 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 04:41:19.268854 27188 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 04:41:19.270006 27188 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 04:41:19.270053 27188 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 04:41:19.270069 27188 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 04:41:19.270082 27188 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 04:41:19.270094 27188 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 04:41:19.271616 27188 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.167.84:34579
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786336863388938-23197-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786336863388938-23197-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.22.167.126:42075
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786336863388938-23197-0/krb5kdc/kudu_127.22.167.126.keytab
--principal=kudu/127.22.167.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.167.126:42075
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786336863388938-23197-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.167.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27188
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786336863388938-23197-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 03:54:01 UTC on ccycloud.quasar-icijmt.root.comops.site
build id 4577009
I20260810 04:41:19.271816 27188 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 04:41:19.271994 27188 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260810 04:41:19.276764 27188 server_base.cc:1034] running on GCE node
W20260810 04:41:19.276894 27198 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 04:41:19.276834 27199 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 04:41:19.276852 27203 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 04:41:19.277302 27188 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 04:41:19.277575 27188 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 04:41:19.278738 27188 hybrid_clock.cc:648] HybridClock initialized: now 1786336879278716 us; error 31 us; skew 500 ppm
Aug 10 04:41:19 dist-test-slave-bqcl krb5kdc[27167](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786336879, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.167.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 04:41:19.284325 27188 init.cc:377] Logged in from keytab as kudu/127.22.167.126@KRBTEST.COM (short username kudu)
I20260810 04:41:19.284765 27188 webserver.cc:460] Webserver started at http://127.22.167.126:37637/ using document root <none> and password file <none>
I20260810 04:41:19.284940 27188 fs_manager.cc:359] Metadata directory not provided
I20260810 04:41:19.284977 27188 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 04:41:19.285071 27188 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 04:41:19.286053 27188 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786336863388938-23197-0/minicluster-data/master-0/data/instance:
uuid: "425fdb09eaa44070ba385e3d7c3e3c50"
format_stamp: "Formatted at 2026-08-10 04:41:19 on dist-test-slave-bqcl"
server_key: "b31586d3bfa3ea4821c3d31780a45bcb"
server_key_iv: "327d29a43aeaf29bea68980cbcfb881a"
server_key_version: "encryptionkey@0"
I20260810 04:41:19.286397 27188 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786336863388938-23197-0/minicluster-data/master-0/wal/instance:
uuid: "425fdb09eaa44070ba385e3d7c3e3c50"
format_stamp: "Formatted at 2026-08-10 04:41:19 on dist-test-slave-bqcl"
server_key: "b31586d3bfa3ea4821c3d31780a45bcb"
server_key_iv: "327d29a43aeaf29bea68980cbcfb881a"
server_key_version: "encryptionkey@0"
I20260810 04:41:19.287585 27188 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260810 04:41:19.288195 27217 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 04:41:19.288342 27188 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260810 04:41:19.288417 27188 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786336863388938-23197-0/minicluster-data/master-0/data,/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786336863388938-23197-0/minicluster-data/master-0/wal
uuid: "425fdb09eaa44070ba385e3d7c3e3c50"
format_stamp: "Formatted at 2026-08-10 04:41:19 on dist-test-slave-bqcl"
server_key: "b31586d3bfa3ea4821c3d31780a45bcb"
server_key_iv: "327d29a43aeaf29bea68980cbcfb881a"
server_key_version: "encryptionkey@0"
I20260810 04:41:19.288498 27188 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786336863388938-23197-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786336863388938-23197-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786336863388938-23197-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 04:41:19.299104 27188 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 04:41:19.301038 27188 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 04:41:19.301177 27188 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 04:41:19.304845 27188 rpc_server.cc:307] RPC server started. Bound to: 127.22.167.126:42075
I20260810 04:41:19.304903 27307 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.167.126:42075 every 8 connection(s)
I20260810 04:41:19.305193 27188 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786336863388938-23197-0/minicluster-data/master-0/data/info.pb
I20260810 04:41:19.306042 27308 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 04:41:19.308327 27308 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 425fdb09eaa44070ba385e3d7c3e3c50: Bootstrap starting.
I20260810 04:41:19.308877 27308 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 425fdb09eaa44070ba385e3d7c3e3c50: Neither blocks nor log segments found. Creating new log.
I20260810 04:41:19.309145 27308 log.cc:824] T 00000000000000000000000000000000 P 425fdb09eaa44070ba385e3d7c3e3c50: Log is configured to *not* fsync() on all Append() calls
I20260810 04:41:19.309800 27308 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 425fdb09eaa44070ba385e3d7c3e3c50: No bootstrap required, opened a new log
I20260810 04:41:19.310839 23197 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskemv_AM/build/release/bin/kudu as pid 27188
I20260810 04:41:19.310920 23197 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786336863388938-23197-0/minicluster-data/master-0/wal/instance
I20260810 04:41:19.311053 23197 external_mini_cluster.cc:1428] Setting key 993facf99589c0620be9f93daa8e71e1
I20260810 04:41:19.311596 27308 raft_consensus.cc:348] T 00000000000000000000000000000000 P 425fdb09eaa44070ba385e3d7c3e3c50 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "425fdb09eaa44070ba385e3d7c3e3c50" member_type: VOTER last_known_addr { host: "127.22.167.126" port: 42075 } }
I20260810 04:41:19.311774 27308 raft_consensus.cc:374] T 00000000000000000000000000000000 P 425fdb09eaa44070ba385e3d7c3e3c50 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 04:41:19.311810 27308 raft_consensus.cc:729] T 00000000000000000000000000000000 P 425fdb09eaa44070ba385e3d7c3e3c50 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 425fdb09eaa44070ba385e3d7c3e3c50, State: Initialized, Role: FOLLOWER
I20260810 04:41:19.311930 27308 consensus_queue.cc:260] T 00000000000000000000000000000000 P 425fdb09eaa44070ba385e3d7c3e3c50 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "425fdb09eaa44070ba385e3d7c3e3c50" member_type: VOTER last_known_addr { host: "127.22.167.126" port: 42075 } }
I20260810 04:41:19.312007 27308 raft_consensus.cc:388] T 00000000000000000000000000000000 P 425fdb09eaa44070ba385e3d7c3e3c50 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 04:41:19.312045 27308 raft_consensus.cc:482] T 00000000000000000000000000000000 P 425fdb09eaa44070ba385e3d7c3e3c50 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 04:41:19.312095 27308 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 425fdb09eaa44070ba385e3d7c3e3c50 [term 0 FOLLOWER]: Advancing to term 1
I20260810 04:41:19.313104 27308 raft_consensus.cc:504] T 00000000000000000000000000000000 P 425fdb09eaa44070ba385e3d7c3e3c50 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "425fdb09eaa44070ba385e3d7c3e3c50" member_type: VOTER last_known_addr { host: "127.22.167.126" port: 42075 } }
I20260810 04:41:19.313195 27308 leader_election.cc:302] T 00000000000000000000000000000000 P 425fdb09eaa44070ba385e3d7c3e3c50 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 425fdb09eaa44070ba385e3d7c3e3c50; no voters:
I20260810 04:41:19.313352 27308 leader_election.cc:288] T 00000000000000000000000000000000 P 425fdb09eaa44070ba385e3d7c3e3c50 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 04:41:19.313395 27320 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 425fdb09eaa44070ba385e3d7c3e3c50 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 04:41:19.313544 27320 raft_consensus.cc:686] T 00000000000000000000000000000000 P 425fdb09eaa44070ba385e3d7c3e3c50 [term 1 LEADER]: Becoming Leader. State: Replica: 425fdb09eaa44070ba385e3d7c3e3c50, State: Running, Role: LEADER
I20260810 04:41:19.313620 27308 sys_catalog.cc:564] T 00000000000000000000000000000000 P 425fdb09eaa44070ba385e3d7c3e3c50 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 04:41:19.313635 27320 consensus_queue.cc:237] T 00000000000000000000000000000000 P 425fdb09eaa44070ba385e3d7c3e3c50 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "425fdb09eaa44070ba385e3d7c3e3c50" member_type: VOTER last_known_addr { host: "127.22.167.126" port: 42075 } }
I20260810 04:41:19.313959 27320 sys_catalog.cc:455] T 00000000000000000000000000000000 P 425fdb09eaa44070ba385e3d7c3e3c50 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 425fdb09eaa44070ba385e3d7c3e3c50. Latest consensus state: current_term: 1 leader_uuid: "425fdb09eaa44070ba385e3d7c3e3c50" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "425fdb09eaa44070ba385e3d7c3e3c50" member_type: VOTER last_known_addr { host: "127.22.167.126" port: 42075 } } }
I20260810 04:41:19.314240 27320 sys_catalog.cc:458] T 00000000000000000000000000000000 P 425fdb09eaa44070ba385e3d7c3e3c50 [sys.catalog]: This master's current role is: LEADER
I20260810 04:41:19.314167 27321 sys_catalog.cc:455] T 00000000000000000000000000000000 P 425fdb09eaa44070ba385e3d7c3e3c50 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "425fdb09eaa44070ba385e3d7c3e3c50" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "425fdb09eaa44070ba385e3d7c3e3c50" member_type: VOTER last_known_addr { host: "127.22.167.126" port: 42075 } } }
I20260810 04:41:19.314667 27321 sys_catalog.cc:458] T 00000000000000000000000000000000 P 425fdb09eaa44070ba385e3d7c3e3c50 [sys.catalog]: This master's current role is: LEADER
I20260810 04:41:19.314834 27333 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 04:41:19.315347 27333 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 04:41:19.316860 27333 catalog_manager.cc:1347] Generated new cluster ID: 600ad2afd6f94d0eae1f6a91f9351197
I20260810 04:41:19.316907 27333 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 10 04:41:19 dist-test-slave-bqcl krb5kdc[27167](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786336879, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.167.126@KRBTEST.COM
I20260810 04:41:19.321128 27317 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:41:19.311525 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54792 (local address 127.22.167.126:42075)
0810 04:41:19.311727 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0810 04:41:19.311737 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 04:41:19.311772 (+ 35us) server_negotiation.cc:407] Connection header received
0810 04:41:19.311884 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:41:19.311891 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:41:19.311944 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:41:19.312094 (+ 150us) server_negotiation.cc:226] Negotiated authn=SASL
0810 04:41:19.312952 (+ 858us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:41:19.314226 (+ 1274us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:41:19.315285 (+ 1059us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:41:19.315760 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:41:19.315838 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:41:19.318214 (+ 2376us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 04:41:19.318218 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 04:41:19.318218 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 04:41:19.318229 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0810 04:41:19.319743 (+ 1514us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 04:41:19.320064 (+ 321us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 04:41:19.320068 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 04:41:19.320068 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 04:41:19.320163 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 04:41:19.320341 (+ 178us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 04:41:19.320341 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 04:41:19.320342 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 04:41:19.320576 (+ 234us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 04:41:19.320733 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0810 04:41:19.320800 (+ 67us) server_negotiation.cc:299] Negotiation successful
0810 04:41:19.320935 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":155,"thread_start_us":104,"threads_started":1}
I20260810 04:41:19.360489 27333 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 04:41:19.361230 27333 catalog_manager.cc:1504] Loading token signing keys...
I20260810 04:41:19.369889 27333 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 425fdb09eaa44070ba385e3d7c3e3c50: Generated new TSK 0
I20260810 04:41:19.370091 27333 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.22.167.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.167.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.167.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786336863388938-23197-0/krb5kdc/kudu_127.22.167.65.keytab.
Entry for principal kudu/127.22.167.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786336863388938-23197-0/krb5kdc/kudu_127.22.167.65.keytab.
I20260810 04:41:19.411230 23197 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskemv_AM/build/release/bin/kudu
/tmp/dist-test-taskemv_AM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786336863388938-23197-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786336863388938-23197-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786336863388938-23197-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786336863388938-23197-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.167.65:0
--local_ip_for_outbound_sockets=127.22.167.65
--webserver_interface=127.22.167.65
--webserver_port=0
--tserver_master_addrs=127.22.167.126:42075
--keytab_file=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786336863388938-23197-0/krb5kdc/kudu_127.22.167.65.keytab
--principal=kudu/127.22.167.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.167.84:34579
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786336863388938-23197-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786336863388938-23197-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786336863388938-23197-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 04:41:19.516022 27355 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 04:41:19.516219 27355 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 04:41:19.516240 27355 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 04:41:19.517319 27355 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 04:41:19.517352 27355 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 04:41:19.517375 27355 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.167.65
I20260810 04:41:19.518781 27355 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.167.84:34579
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786336863388938-23197-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786336863388938-23197-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786336863388938-23197-0/krb5kdc/kudu_127.22.167.65.keytab
--principal=kudu/127.22.167.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.167.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786336863388938-23197-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.22.167.65
--webserver_port=0
--tserver_master_addrs=127.22.167.126:42075
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27355
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.167.65
--log_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786336863388938-23197-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 03:54:01 UTC on ccycloud.quasar-icijmt.root.comops.site
build id 4577009
I20260810 04:41:19.518973 27355 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 04:41:19.519220 27355 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 04:41:19.524111 27361 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 04:41:19.524111 27364 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 04:41:19.524211 27362 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 04:41:19.524127 27355 server_base.cc:1034] running on GCE node
I20260810 04:41:19.524567 27355 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 04:41:19.524768 27355 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 04:41:19.525929 27355 hybrid_clock.cc:648] HybridClock initialized: now 1786336879525914 us; error 29 us; skew 500 ppm
Aug 10 04:41:19 dist-test-slave-bqcl krb5kdc[27167](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786336879, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.167.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 04:41:19.528435 27355 init.cc:377] Logged in from keytab as kudu/127.22.167.65@KRBTEST.COM (short username kudu)
I20260810 04:41:19.528846 27355 webserver.cc:460] Webserver started at http://127.22.167.65:40653/ using document root <none> and password file <none>
I20260810 04:41:19.529052 27355 fs_manager.cc:359] Metadata directory not provided
I20260810 04:41:19.529103 27355 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 04:41:19.529229 27355 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 04:41:19.530253 27355 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786336863388938-23197-0/minicluster-data/ts-0/data/instance:
uuid: "4d6fff5b0cb648c3a555ed297aecccea"
format_stamp: "Formatted at 2026-08-10 04:41:19 on dist-test-slave-bqcl"
server_key: "fd5f3a77ba913db238ad723e2313c678"
server_key_iv: "78c6ccb053054164c40c6dd963283a5b"
server_key_version: "encryptionkey@0"
I20260810 04:41:19.530586 27355 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786336863388938-23197-0/minicluster-data/ts-0/wal/instance:
uuid: "4d6fff5b0cb648c3a555ed297aecccea"
format_stamp: "Formatted at 2026-08-10 04:41:19 on dist-test-slave-bqcl"
server_key: "fd5f3a77ba913db238ad723e2313c678"
server_key_iv: "78c6ccb053054164c40c6dd963283a5b"
server_key_version: "encryptionkey@0"
I20260810 04:41:19.531774 27355 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260810 04:41:19.532421 27375 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 04:41:19.532583 27355 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260810 04:41:19.532666 27355 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786336863388938-23197-0/minicluster-data/ts-0/data,/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786336863388938-23197-0/minicluster-data/ts-0/wal
uuid: "4d6fff5b0cb648c3a555ed297aecccea"
format_stamp: "Formatted at 2026-08-10 04:41:19 on dist-test-slave-bqcl"
server_key: "fd5f3a77ba913db238ad723e2313c678"
server_key_iv: "78c6ccb053054164c40c6dd963283a5b"
server_key_version: "encryptionkey@0"
I20260810 04:41:19.532725 27355 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786336863388938-23197-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786336863388938-23197-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786336863388938-23197-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 04:41:19.541718 27355 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 04:41:19.543615 27355 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 04:41:19.543745 27355 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 04:41:19.543933 27355 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 04:41:19.544210 27355 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 04:41:19.544245 27355 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 04:41:19.544271 27355 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 04:41:19.544288 27355 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 04:41:19.549714 27355 rpc_server.cc:307] RPC server started. Bound to: 127.22.167.65:33201
I20260810 04:41:19.549774 27571 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.167.65:33201 every 8 connection(s)
I20260810 04:41:19.550061 27355 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786336863388938-23197-0/minicluster-data/ts-0/data/info.pb
Aug 10 04:41:19 dist-test-slave-bqcl krb5kdc[27167](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786336879, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.167.65@KRBTEST.COM for kudu/127.22.167.126@KRBTEST.COM
I20260810 04:41:19.556995 23197 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskemv_AM/build/release/bin/kudu as pid 27355
I20260810 04:41:19.557093 23197 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786336863388938-23197-0/minicluster-data/ts-0/wal/instance
I20260810 04:41:19.557247 23197 external_mini_cluster.cc:1428] Setting key d775105d90bb1798128758140939ec52
I20260810 04:41:19.558866 27317 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:41:19.550799 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.167.65:59993 (local address 127.22.167.126:42075)
0810 04:41:19.550848 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0810 04:41:19.550850 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 04:41:19.551034 (+ 184us) server_negotiation.cc:407] Connection header received
0810 04:41:19.551527 (+ 493us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:41:19.551529 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:41:19.551543 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:41:19.551595 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0810 04:41:19.552537 (+ 942us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:41:19.553039 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:41:19.554014 (+ 975us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:41:19.554279 (+ 265us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:41:19.554333 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:41:19.556251 (+ 1918us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 04:41:19.556252 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 04:41:19.556252 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 04:41:19.556260 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0810 04:41:19.557542 (+ 1282us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 04:41:19.557951 (+ 409us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 04:41:19.557952 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 04:41:19.557952 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 04:41:19.558011 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 04:41:19.558295 (+ 284us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 04:41:19.558296 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 04:41:19.558296 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 04:41:19.558402 (+ 106us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 04:41:19.558450 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0810 04:41:19.558739 (+ 289us) server_negotiation.cc:299] Negotiation successful
0810 04:41:19.558805 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260810 04:41:19.559075 27574 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:41:19.550765 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.126:42075 (local address 127.22.167.65:59993)
0810 04:41:19.550910 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0810 04:41:19.550951 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0810 04:41:19.551426 (+ 475us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 04:41:19.551689 (+ 263us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 04:41:19.551693 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 04:41:19.552252 (+ 559us) client_negotiation.cc:189] Negotiated authn=SASL
0810 04:41:19.552436 (+ 184us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:41:19.552441 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:41:19.553131 (+ 690us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:41:19.553131 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:41:19.553926 (+ 795us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:41:19.553929 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:41:19.554361 (+ 432us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:41:19.554362 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:41:19.554478 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:41:19.554891 (+ 413us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 04:41:19.554899 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0810 04:41:19.556165 (+ 1266us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 04:41:19.557620 (+ 1455us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 04:41:19.557626 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 04:41:19.557628 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 04:41:19.557884 (+ 256us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 04:41:19.558071 (+ 187us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 04:41:19.558072 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 04:41:19.558073 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 04:41:19.558232 (+ 159us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 04:41:19.558454 (+ 222us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 04:41:19.558460 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 04:41:19.558621 (+ 161us) client_negotiation.cc:769] Sending connection context
0810 04:41:19.558726 (+ 105us) client_negotiation.cc:240] Negotiation successful
0810 04:41:19.558874 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":70,"threads_started":1}
I20260810 04:41:19.559717 27572 heartbeater.cc:347] Connected to a master server at 127.22.167.126:42075
I20260810 04:41:19.559804 27572 heartbeater.cc:464] Registering TS with master...
I20260810 04:41:19.559971 27572 heartbeater.cc:511] Master 127.22.167.126:42075 requested a full tablet report, sending...
I20260810 04:41:19.560464 27247 ts_manager.cc:194] Registered new tserver with Master: 4d6fff5b0cb648c3a555ed297aecccea (127.22.167.65:33201)
I20260810 04:41:19.561075 27247 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.167.65@KRBTEST.COM'} at 127.22.167.65:59993
WARNING: no policy specified for kudu/127.22.167.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.167.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.167.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786336863388938-23197-0/krb5kdc/kudu_127.22.167.66.keytab.
Entry for principal kudu/127.22.167.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786336863388938-23197-0/krb5kdc/kudu_127.22.167.66.keytab.
I20260810 04:41:19.582165 23197 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskemv_AM/build/release/bin/kudu
/tmp/dist-test-taskemv_AM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786336863388938-23197-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786336863388938-23197-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786336863388938-23197-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786336863388938-23197-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.167.66:0
--local_ip_for_outbound_sockets=127.22.167.66
--webserver_interface=127.22.167.66
--webserver_port=0
--tserver_master_addrs=127.22.167.126:42075
--keytab_file=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786336863388938-23197-0/krb5kdc/kudu_127.22.167.66.keytab
--principal=kudu/127.22.167.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.167.84:34579
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786336863388938-23197-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786336863388938-23197-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786336863388938-23197-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 04:41:19.661135 27577 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 04:41:19.661332 27577 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 04:41:19.661361 27577 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 04:41:19.662487 27577 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 04:41:19.662524 27577 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 04:41:19.662557 27577 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.167.66
I20260810 04:41:19.663911 27577 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.167.84:34579
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786336863388938-23197-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786336863388938-23197-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786336863388938-23197-0/krb5kdc/kudu_127.22.167.66.keytab
--principal=kudu/127.22.167.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.167.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786336863388938-23197-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.22.167.66
--webserver_port=0
--tserver_master_addrs=127.22.167.126:42075
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27577
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.167.66
--log_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786336863388938-23197-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 03:54:01 UTC on ccycloud.quasar-icijmt.root.comops.site
build id 4577009
I20260810 04:41:19.664137 27577 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 04:41:19.664338 27577 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 04:41:19.669175 27588 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 04:41:19.669216 27577 server_base.cc:1034] running on GCE node
W20260810 04:41:19.669224 27587 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 04:41:19.669188 27591 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 04:41:19.669487 27577 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 04:41:19.669737 27577 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 04:41:19.670882 27577 hybrid_clock.cc:648] HybridClock initialized: now 1786336879670862 us; error 33 us; skew 500 ppm
Aug 10 04:41:19 dist-test-slave-bqcl krb5kdc[27167](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786336879, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.167.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 04:41:19.673223 27577 init.cc:377] Logged in from keytab as kudu/127.22.167.66@KRBTEST.COM (short username kudu)
I20260810 04:41:19.673655 27577 webserver.cc:460] Webserver started at http://127.22.167.66:44979/ using document root <none> and password file <none>
I20260810 04:41:19.673844 27577 fs_manager.cc:359] Metadata directory not provided
I20260810 04:41:19.673887 27577 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 04:41:19.674005 27577 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 04:41:19.674991 27577 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786336863388938-23197-0/minicluster-data/ts-1/data/instance:
uuid: "1a918c18e41d4335b055e2c9c9e0fa68"
format_stamp: "Formatted at 2026-08-10 04:41:19 on dist-test-slave-bqcl"
server_key: "f2cd2156b8210a09e4535385c7c17717"
server_key_iv: "809ff831cd96a1767a22ab4ac9ecbf46"
server_key_version: "encryptionkey@0"
I20260810 04:41:19.675299 27577 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786336863388938-23197-0/minicluster-data/ts-1/wal/instance:
uuid: "1a918c18e41d4335b055e2c9c9e0fa68"
format_stamp: "Formatted at 2026-08-10 04:41:19 on dist-test-slave-bqcl"
server_key: "f2cd2156b8210a09e4535385c7c17717"
server_key_iv: "809ff831cd96a1767a22ab4ac9ecbf46"
server_key_version: "encryptionkey@0"
I20260810 04:41:19.676510 27577 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260810 04:41:19.677119 27602 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 04:41:19.677251 27577 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260810 04:41:19.677332 27577 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786336863388938-23197-0/minicluster-data/ts-1/data,/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786336863388938-23197-0/minicluster-data/ts-1/wal
uuid: "1a918c18e41d4335b055e2c9c9e0fa68"
format_stamp: "Formatted at 2026-08-10 04:41:19 on dist-test-slave-bqcl"
server_key: "f2cd2156b8210a09e4535385c7c17717"
server_key_iv: "809ff831cd96a1767a22ab4ac9ecbf46"
server_key_version: "encryptionkey@0"
I20260810 04:41:19.677389 27577 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786336863388938-23197-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786336863388938-23197-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786336863388938-23197-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 04:41:19.704041 27577 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 04:41:19.705791 27577 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 04:41:19.705924 27577 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 04:41:19.706153 27577 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 04:41:19.706442 27577 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 04:41:19.706475 27577 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 04:41:19.706512 27577 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 04:41:19.706535 27577 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 04:41:19.711877 27577 rpc_server.cc:307] RPC server started. Bound to: 127.22.167.66:33013
I20260810 04:41:19.711917 27814 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.167.66:33013 every 8 connection(s)
I20260810 04:41:19.712205 27577 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786336863388938-23197-0/minicluster-data/ts-1/data/info.pb
I20260810 04:41:19.717772 23197 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskemv_AM/build/release/bin/kudu as pid 27577
I20260810 04:41:19.717872 23197 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786336863388938-23197-0/minicluster-data/ts-1/wal/instance
I20260810 04:41:19.718048 23197 external_mini_cluster.cc:1428] Setting key d8e70b7c920b2023ce7979afedeb5d3d
Aug 10 04:41:19 dist-test-slave-bqcl krb5kdc[27167](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786336879, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.167.66@KRBTEST.COM for kudu/127.22.167.126@KRBTEST.COM
I20260810 04:41:19.722256 27317 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:41:19.712904 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.167.66:41645 (local address 127.22.167.126:42075)
0810 04:41:19.712981 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0810 04:41:19.712983 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 04:41:19.713190 (+ 207us) server_negotiation.cc:407] Connection header received
0810 04:41:19.713651 (+ 461us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:41:19.713652 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:41:19.713663 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:41:19.713716 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0810 04:41:19.714865 (+ 1149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:41:19.715615 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:41:19.717038 (+ 1423us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:41:19.717315 (+ 277us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:41:19.717353 (+ 38us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:41:19.719647 (+ 2294us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 04:41:19.719649 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 04:41:19.719649 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 04:41:19.719657 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0810 04:41:19.721027 (+ 1370us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 04:41:19.721392 (+ 365us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 04:41:19.721393 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 04:41:19.721394 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 04:41:19.721455 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 04:41:19.721660 (+ 205us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 04:41:19.721661 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 04:41:19.721661 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 04:41:19.721788 (+ 127us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 04:41:19.721853 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0810 04:41:19.722061 (+ 208us) server_negotiation.cc:299] Negotiation successful
0810 04:41:19.722163 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260810 04:41:19.722371 27824 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:41:19.712878 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.126:42075 (local address 127.22.167.66:41645)
0810 04:41:19.713058 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0810 04:41:19.713093 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0810 04:41:19.713572 (+ 479us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 04:41:19.713775 (+ 203us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 04:41:19.713778 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 04:41:19.714469 (+ 691us) client_negotiation.cc:189] Negotiated authn=SASL
0810 04:41:19.714754 (+ 285us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:41:19.714761 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:41:19.715738 (+ 977us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:41:19.715739 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:41:19.716939 (+ 1200us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:41:19.716942 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:41:19.717433 (+ 491us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:41:19.717433 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:41:19.717579 (+ 146us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:41:19.718150 (+ 571us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 04:41:19.718159 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0810 04:41:19.719562 (+ 1403us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 04:41:19.721107 (+ 1545us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 04:41:19.721112 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 04:41:19.721114 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 04:41:19.721324 (+ 210us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 04:41:19.721514 (+ 190us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 04:41:19.721515 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 04:41:19.721515 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 04:41:19.721613 (+ 98us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 04:41:19.721844 (+ 231us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 04:41:19.721846 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 04:41:19.721950 (+ 104us) client_negotiation.cc:769] Sending connection context
0810 04:41:19.722023 (+ 73us) client_negotiation.cc:240] Negotiation successful
0810 04:41:19.722188 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":71,"threads_started":1}
I20260810 04:41:19.722774 27820 heartbeater.cc:347] Connected to a master server at 127.22.167.126:42075
I20260810 04:41:19.722877 27820 heartbeater.cc:464] Registering TS with master...
I20260810 04:41:19.723093 27820 heartbeater.cc:511] Master 127.22.167.126:42075 requested a full tablet report, sending...
I20260810 04:41:19.723475 27247 ts_manager.cc:194] Registered new tserver with Master: 1a918c18e41d4335b055e2c9c9e0fa68 (127.22.167.66:33013)
I20260810 04:41:19.723949 27247 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.167.66@KRBTEST.COM'} at 127.22.167.66:41645
WARNING: no policy specified for kudu/127.22.167.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.167.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.167.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786336863388938-23197-0/krb5kdc/kudu_127.22.167.67.keytab.
Entry for principal kudu/127.22.167.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786336863388938-23197-0/krb5kdc/kudu_127.22.167.67.keytab.
I20260810 04:41:19.742709 23197 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskemv_AM/build/release/bin/kudu
/tmp/dist-test-taskemv_AM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786336863388938-23197-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786336863388938-23197-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786336863388938-23197-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786336863388938-23197-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.167.67:0
--local_ip_for_outbound_sockets=127.22.167.67
--webserver_interface=127.22.167.67
--webserver_port=0
--tserver_master_addrs=127.22.167.126:42075
--keytab_file=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786336863388938-23197-0/krb5kdc/kudu_127.22.167.67.keytab
--principal=kudu/127.22.167.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.167.84:34579
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786336863388938-23197-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786336863388938-23197-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786336863388938-23197-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 04:41:19.821828 27829 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 04:41:19.822012 27829 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 04:41:19.822033 27829 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 04:41:19.823136 27829 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 04:41:19.823177 27829 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 04:41:19.823202 27829 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.167.67
I20260810 04:41:19.824523 27829 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.167.84:34579
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786336863388938-23197-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786336863388938-23197-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786336863388938-23197-0/krb5kdc/kudu_127.22.167.67.keytab
--principal=kudu/127.22.167.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.167.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786336863388938-23197-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.22.167.67
--webserver_port=0
--tserver_master_addrs=127.22.167.126:42075
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27829
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.167.67
--log_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786336863388938-23197-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 03:54:01 UTC on ccycloud.quasar-icijmt.root.comops.site
build id 4577009
I20260810 04:41:19.824703 27829 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 04:41:19.824870 27829 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 04:41:19.829474 27835 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 04:41:19.829480 27836 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 04:41:19.829658 27829 server_base.cc:1034] running on GCE node
W20260810 04:41:19.829499 27840 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 04:41:19.829936 27829 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 04:41:19.830194 27829 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 04:41:19.831337 27829 hybrid_clock.cc:648] HybridClock initialized: now 1786336879831322 us; error 29 us; skew 500 ppm
Aug 10 04:41:19 dist-test-slave-bqcl krb5kdc[27167](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786336879, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.167.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 04:41:19.833762 27829 init.cc:377] Logged in from keytab as kudu/127.22.167.67@KRBTEST.COM (short username kudu)
I20260810 04:41:19.834213 27829 webserver.cc:460] Webserver started at http://127.22.167.67:38839/ using document root <none> and password file <none>
I20260810 04:41:19.834425 27829 fs_manager.cc:359] Metadata directory not provided
I20260810 04:41:19.834472 27829 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 04:41:19.834600 27829 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 04:41:19.835628 27829 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786336863388938-23197-0/minicluster-data/ts-2/data/instance:
uuid: "c5da011605d6416eb9012e55c7ec5caf"
format_stamp: "Formatted at 2026-08-10 04:41:19 on dist-test-slave-bqcl"
server_key: "2a73e9a0b3015ddf8ec37d408dec0dab"
server_key_iv: "199d2b0313bdc59a69b98827c77b267d"
server_key_version: "encryptionkey@0"
I20260810 04:41:19.836031 27829 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786336863388938-23197-0/minicluster-data/ts-2/wal/instance:
uuid: "c5da011605d6416eb9012e55c7ec5caf"
format_stamp: "Formatted at 2026-08-10 04:41:19 on dist-test-slave-bqcl"
server_key: "2a73e9a0b3015ddf8ec37d408dec0dab"
server_key_iv: "199d2b0313bdc59a69b98827c77b267d"
server_key_version: "encryptionkey@0"
I20260810 04:41:19.837361 27829 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260810 04:41:19.838019 27855 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 04:41:19.838199 27829 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260810 04:41:19.838281 27829 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786336863388938-23197-0/minicluster-data/ts-2/data,/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786336863388938-23197-0/minicluster-data/ts-2/wal
uuid: "c5da011605d6416eb9012e55c7ec5caf"
format_stamp: "Formatted at 2026-08-10 04:41:19 on dist-test-slave-bqcl"
server_key: "2a73e9a0b3015ddf8ec37d408dec0dab"
server_key_iv: "199d2b0313bdc59a69b98827c77b267d"
server_key_version: "encryptionkey@0"
I20260810 04:41:19.838341 27829 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786336863388938-23197-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786336863388938-23197-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786336863388938-23197-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 04:41:19.856696 27829 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 04:41:19.858783 27829 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 04:41:19.858916 27829 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 04:41:19.859123 27829 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 04:41:19.859403 27829 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 04:41:19.859436 27829 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 04:41:19.859464 27829 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 04:41:19.859483 27829 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 04:41:19.864674 27829 rpc_server.cc:307] RPC server started. Bound to: 127.22.167.67:45873
I20260810 04:41:19.864738 28044 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.167.67:45873 every 8 connection(s)
I20260810 04:41:19.865002 27829 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786336863388938-23197-0/minicluster-data/ts-2/data/info.pb
I20260810 04:41:19.866883 23197 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskemv_AM/build/release/bin/kudu as pid 27829
I20260810 04:41:19.866969 23197 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786336863388938-23197-0/minicluster-data/ts-2/wal/instance
I20260810 04:41:19.867122 23197 external_mini_cluster.cc:1428] Setting key 0059c38a992b77f5a4e9576aa7c62781
Aug 10 04:41:19 dist-test-slave-bqcl krb5kdc[27167](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786336879, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.167.67@KRBTEST.COM for kudu/127.22.167.126@KRBTEST.COM
I20260810 04:41:19.874339 27317 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:41:19.865702 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.167.67:51993 (local address 127.22.167.126:42075)
0810 04:41:19.865847 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0810 04:41:19.865849 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 04:41:19.865919 (+ 70us) server_negotiation.cc:407] Connection header received
0810 04:41:19.866538 (+ 619us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:41:19.866540 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:41:19.866555 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:41:19.866615 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0810 04:41:19.867695 (+ 1080us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:41:19.868257 (+ 562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:41:19.869269 (+ 1012us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:41:19.869472 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:41:19.869539 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:41:19.871866 (+ 2327us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 04:41:19.871867 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 04:41:19.871868 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 04:41:19.871876 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0810 04:41:19.872945 (+ 1069us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 04:41:19.873335 (+ 390us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 04:41:19.873336 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 04:41:19.873336 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 04:41:19.873401 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 04:41:19.873672 (+ 271us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 04:41:19.873672 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 04:41:19.873673 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 04:41:19.873793 (+ 120us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 04:41:19.873857 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0810 04:41:19.874157 (+ 300us) server_negotiation.cc:299] Negotiation successful
0810 04:41:19.874246 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":116}
I20260810 04:41:19.874490 28049 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:41:19.865613 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.126:42075 (local address 127.22.167.67:51993)
0810 04:41:19.865792 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0810 04:41:19.865839 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0810 04:41:19.866403 (+ 564us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 04:41:19.866686 (+ 283us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 04:41:19.866690 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 04:41:19.867339 (+ 649us) client_negotiation.cc:189] Negotiated authn=SASL
0810 04:41:19.867608 (+ 269us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:41:19.867616 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:41:19.868371 (+ 755us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:41:19.868372 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:41:19.869180 (+ 808us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:41:19.869183 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:41:19.869566 (+ 383us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:41:19.869567 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:41:19.869719 (+ 152us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:41:19.870266 (+ 547us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 04:41:19.870274 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0810 04:41:19.871763 (+ 1489us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 04:41:19.873029 (+ 1266us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 04:41:19.873037 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 04:41:19.873039 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 04:41:19.873260 (+ 221us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 04:41:19.873462 (+ 202us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 04:41:19.873463 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 04:41:19.873463 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 04:41:19.873601 (+ 138us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 04:41:19.873853 (+ 252us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 04:41:19.873859 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 04:41:19.874010 (+ 151us) client_negotiation.cc:769] Sending connection context
0810 04:41:19.874105 (+ 95us) client_negotiation.cc:240] Negotiation successful
0810 04:41:19.874281 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":87,"threads_started":1}
I20260810 04:41:19.874938 28045 heartbeater.cc:347] Connected to a master server at 127.22.167.126:42075
I20260810 04:41:19.875026 28045 heartbeater.cc:464] Registering TS with master...
I20260810 04:41:19.875196 28045 heartbeater.cc:511] Master 127.22.167.126:42075 requested a full tablet report, sending...
I20260810 04:41:19.875535 27247 ts_manager.cc:194] Registered new tserver with Master: c5da011605d6416eb9012e55c7ec5caf (127.22.167.67:45873)
I20260810 04:41:19.875897 27247 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.167.67@KRBTEST.COM'} at 127.22.167.67:51993
I20260810 04:41:19.878682 23197 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260810 04:41:19.891620 27317 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 04:41:19.890856 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54796 (local address 127.22.167.126:42075)
0810 04:41:19.890920 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0810 04:41:19.890922 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 04:41:19.891005 (+ 83us) server_negotiation.cc:407] Connection header received
0810 04:41:19.891053 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:41:19.891054 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:41:19.891062 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:41:19.891125 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0810 04:41:19.891574 (+ 449us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:54796: BlockingRecv error: recv got EOF from 127.0.0.1:54796 (error 108)
Metrics: {"server-negotiator.queue_time_us":42}
I20260810 04:41:19.892474 23197 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskemv_AM/build/release/bin/kudu with pid 27355
I20260810 04:41:19.908432 23197 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskemv_AM/build/release/bin/kudu with pid 27577
I20260810 04:41:19.925135 23197 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskemv_AM/build/release/bin/kudu with pid 27829
I20260810 04:41:19.936199 23197 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskemv_AM/build/release/bin/kudu with pid 27188
2026-08-10T04:41:19Z chronyd exiting
[ OK ] SecurityITest.TestNoKerberosCredentials (3108 ms)
[ RUN ] SecurityITest.SaslPlainFallback
Loading random data
Initializing database '/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786336863388938-23197-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 04:41:19 dist-test-slave-bqcl krb5kdc[28065](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 04:41:19 dist-test-slave-bqcl krb5kdc[28065](info): set up 2 sockets
Aug 10 04:41:19 dist-test-slave-bqcl krb5kdc[28065](info): commencing operation
krb5kdc: starting...
W20260810 04:41:21.097034 23197 mini_kdc.cc:121] Time spent starting KDC: real 1.138s user 0.001s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786336863388938-23197-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786336863388938-23197-0/krb5kdc/test-user.keytab.
Aug 10 04:41:21 dist-test-slave-bqcl krb5kdc[28065](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786336881, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T04:41:21Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T04:41:21Z Disabled control of system clock
WARNING: no policy specified for kudu/127.22.167.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.167.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.167.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786336863388938-23197-0/krb5kdc/kudu_127.22.167.126.keytab.
Entry for principal kudu/127.22.167.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786336863388938-23197-0/krb5kdc/kudu_127.22.167.126.keytab.
I20260810 04:41:21.212078 23197 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskemv_AM/build/release/bin/kudu
/tmp/dist-test-taskemv_AM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786336863388938-23197-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786336863388938-23197-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786336863388938-23197-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786336863388938-23197-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.167.126:44493
--webserver_interface=127.22.167.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786336863388938-23197-0/krb5kdc/kudu_127.22.167.126.keytab
--principal=kudu/127.22.167.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.167.84:36687
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.167.126:44493
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc-authentication=optional
--user-acl=* with env {KRB5CCNAME=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786336863388938-23197-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786336863388938-23197-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786336863388938-23197-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 04:41:21.292732 28086 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 04:41:21.292934 28086 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 04:41:21.292958 28086 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 04:41:21.294060 28086 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 04:41:21.294104 28086 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 04:41:21.294142 28086 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 04:41:21.294164 28086 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 04:41:21.294185 28086 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 04:41:21.295699 28086 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.167.84:36687
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786336863388938-23197-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786336863388938-23197-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.22.167.126:44493
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786336863388938-23197-0/krb5kdc/kudu_127.22.167.126.keytab
--principal=kudu/127.22.167.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.167.126:44493
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786336863388938-23197-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.167.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28086
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786336863388938-23197-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 03:54:01 UTC on ccycloud.quasar-icijmt.root.comops.site
build id 4577009
I20260810 04:41:21.295890 28086 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 04:41:21.296061 28086 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 04:41:21.300696 28102 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 04:41:21.300733 28096 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 04:41:21.300746 28097 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 04:41:21.300765 28086 server_base.cc:1034] running on GCE node
I20260810 04:41:21.301147 28086 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 04:41:21.301409 28086 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 04:41:21.302549 28086 hybrid_clock.cc:648] HybridClock initialized: now 1786336881302534 us; error 29 us; skew 500 ppm
Aug 10 04:41:21 dist-test-slave-bqcl krb5kdc[28065](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786336881, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.167.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 04:41:21.304894 28086 init.cc:377] Logged in from keytab as kudu/127.22.167.126@KRBTEST.COM (short username kudu)
I20260810 04:41:21.305341 28086 webserver.cc:460] Webserver started at http://127.22.167.126:44967/ using document root <none> and password file <none>
I20260810 04:41:21.305526 28086 fs_manager.cc:359] Metadata directory not provided
I20260810 04:41:21.305583 28086 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 04:41:21.305706 28086 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 04:41:21.306746 28086 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786336863388938-23197-0/minicluster-data/master-0/data/instance:
uuid: "5a0e9f6b525648d2bf320ff6cb9b6220"
format_stamp: "Formatted at 2026-08-10 04:41:21 on dist-test-slave-bqcl"
server_key: "da82a98a1344bd2fc471c46e89ae6d10"
server_key_iv: "e74720caac74094d65407545c1400ec4"
server_key_version: "encryptionkey@0"
I20260810 04:41:21.307058 28086 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786336863388938-23197-0/minicluster-data/master-0/wal/instance:
uuid: "5a0e9f6b525648d2bf320ff6cb9b6220"
format_stamp: "Formatted at 2026-08-10 04:41:21 on dist-test-slave-bqcl"
server_key: "da82a98a1344bd2fc471c46e89ae6d10"
server_key_iv: "e74720caac74094d65407545c1400ec4"
server_key_version: "encryptionkey@0"
I20260810 04:41:21.308310 28086 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260810 04:41:21.308970 28109 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 04:41:21.309115 28086 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260810 04:41:21.309196 28086 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786336863388938-23197-0/minicluster-data/master-0/data,/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786336863388938-23197-0/minicluster-data/master-0/wal
uuid: "5a0e9f6b525648d2bf320ff6cb9b6220"
format_stamp: "Formatted at 2026-08-10 04:41:21 on dist-test-slave-bqcl"
server_key: "da82a98a1344bd2fc471c46e89ae6d10"
server_key_iv: "e74720caac74094d65407545c1400ec4"
server_key_version: "encryptionkey@0"
I20260810 04:41:21.309253 28086 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786336863388938-23197-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786336863388938-23197-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786336863388938-23197-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 04:41:21.324513 28086 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 04:41:21.326342 28086 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 04:41:21.326470 28086 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 04:41:21.330327 28086 rpc_server.cc:307] RPC server started. Bound to: 127.22.167.126:44493
I20260810 04:41:21.330363 28209 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.167.126:44493 every 8 connection(s)
I20260810 04:41:21.330870 28086 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786336863388938-23197-0/minicluster-data/master-0/data/info.pb
I20260810 04:41:21.331502 28211 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 04:41:21.333745 28211 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5a0e9f6b525648d2bf320ff6cb9b6220: Bootstrap starting.
I20260810 04:41:21.334350 28211 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5a0e9f6b525648d2bf320ff6cb9b6220: Neither blocks nor log segments found. Creating new log.
I20260810 04:41:21.334636 28211 log.cc:824] T 00000000000000000000000000000000 P 5a0e9f6b525648d2bf320ff6cb9b6220: Log is configured to *not* fsync() on all Append() calls
I20260810 04:41:21.335275 28211 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5a0e9f6b525648d2bf320ff6cb9b6220: No bootstrap required, opened a new log
I20260810 04:41:21.336305 23197 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskemv_AM/build/release/bin/kudu as pid 28086
I20260810 04:41:21.336390 23197 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786336863388938-23197-0/minicluster-data/master-0/wal/instance
I20260810 04:41:21.336539 23197 external_mini_cluster.cc:1428] Setting key f0a883a0396e9705ee5bee44a384473a
I20260810 04:41:21.336956 28211 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5a0e9f6b525648d2bf320ff6cb9b6220 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5a0e9f6b525648d2bf320ff6cb9b6220" member_type: VOTER last_known_addr { host: "127.22.167.126" port: 44493 } }
I20260810 04:41:21.337172 28211 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5a0e9f6b525648d2bf320ff6cb9b6220 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 04:41:21.337211 28211 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5a0e9f6b525648d2bf320ff6cb9b6220 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5a0e9f6b525648d2bf320ff6cb9b6220, State: Initialized, Role: FOLLOWER
I20260810 04:41:21.337339 28211 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5a0e9f6b525648d2bf320ff6cb9b6220 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5a0e9f6b525648d2bf320ff6cb9b6220" member_type: VOTER last_known_addr { host: "127.22.167.126" port: 44493 } }
I20260810 04:41:21.337405 28211 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5a0e9f6b525648d2bf320ff6cb9b6220 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 04:41:21.337428 28211 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5a0e9f6b525648d2bf320ff6cb9b6220 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 04:41:21.337474 28211 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5a0e9f6b525648d2bf320ff6cb9b6220 [term 0 FOLLOWER]: Advancing to term 1
I20260810 04:41:21.338107 28211 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5a0e9f6b525648d2bf320ff6cb9b6220 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5a0e9f6b525648d2bf320ff6cb9b6220" member_type: VOTER last_known_addr { host: "127.22.167.126" port: 44493 } }
I20260810 04:41:21.338268 28211 leader_election.cc:302] T 00000000000000000000000000000000 P 5a0e9f6b525648d2bf320ff6cb9b6220 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5a0e9f6b525648d2bf320ff6cb9b6220; no voters:
I20260810 04:41:21.338457 28211 leader_election.cc:288] T 00000000000000000000000000000000 P 5a0e9f6b525648d2bf320ff6cb9b6220 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 04:41:21.338511 28219 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5a0e9f6b525648d2bf320ff6cb9b6220 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 04:41:21.338749 28219 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5a0e9f6b525648d2bf320ff6cb9b6220 [term 1 LEADER]: Becoming Leader. State: Replica: 5a0e9f6b525648d2bf320ff6cb9b6220, State: Running, Role: LEADER
I20260810 04:41:21.338781 28211 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5a0e9f6b525648d2bf320ff6cb9b6220 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 04:41:21.338903 28219 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5a0e9f6b525648d2bf320ff6cb9b6220 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5a0e9f6b525648d2bf320ff6cb9b6220" member_type: VOTER last_known_addr { host: "127.22.167.126" port: 44493 } }
I20260810 04:41:21.339798 28222 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5a0e9f6b525648d2bf320ff6cb9b6220 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5a0e9f6b525648d2bf320ff6cb9b6220" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5a0e9f6b525648d2bf320ff6cb9b6220" member_type: VOTER last_known_addr { host: "127.22.167.126" port: 44493 } } }
I20260810 04:41:21.339876 28222 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5a0e9f6b525648d2bf320ff6cb9b6220 [sys.catalog]: This master's current role is: LEADER
I20260810 04:41:21.340116 28221 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5a0e9f6b525648d2bf320ff6cb9b6220 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5a0e9f6b525648d2bf320ff6cb9b6220. Latest consensus state: current_term: 1 leader_uuid: "5a0e9f6b525648d2bf320ff6cb9b6220" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5a0e9f6b525648d2bf320ff6cb9b6220" member_type: VOTER last_known_addr { host: "127.22.167.126" port: 44493 } } }
I20260810 04:41:21.340266 28221 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5a0e9f6b525648d2bf320ff6cb9b6220 [sys.catalog]: This master's current role is: LEADER
I20260810 04:41:21.340485 28236 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 04:41:21.340986 28236 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 04:41:21.342760 28236 catalog_manager.cc:1347] Generated new cluster ID: be16303f93d34eb791e7f3aa1234be9d
I20260810 04:41:21.342809 28236 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 10 04:41:21 dist-test-slave-bqcl krb5kdc[28065](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786336881, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.167.126@KRBTEST.COM
I20260810 04:41:21.347625 28218 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:41:21.337011 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38482 (local address 127.22.167.126:44493)
0810 04:41:21.337554 (+ 543us) server_negotiation.cc:206] Beginning negotiation
0810 04:41:21.337561 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 04:41:21.337591 (+ 30us) server_negotiation.cc:407] Connection header received
0810 04:41:21.337688 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:41:21.337693 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:41:21.337737 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:41:21.337864 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0810 04:41:21.338748 (+ 884us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:41:21.340472 (+ 1724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:41:21.341488 (+ 1016us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:41:21.342099 (+ 611us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:41:21.342229 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:41:21.344341 (+ 2112us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 04:41:21.344345 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 04:41:21.344345 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 04:41:21.344357 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0810 04:41:21.346238 (+ 1881us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 04:41:21.346545 (+ 307us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 04:41:21.346548 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 04:41:21.346548 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 04:41:21.346651 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 04:41:21.346821 (+ 170us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 04:41:21.346821 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 04:41:21.346822 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 04:41:21.347063 (+ 241us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 04:41:21.347235 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0810 04:41:21.347299 (+ 64us) server_negotiation.cc:299] Negotiation successful
0810 04:41:21.347430 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":494,"thread_start_us":469,"threads_started":1}
I20260810 04:41:21.348450 23197 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 04:41:21.358909 28236 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 04:41:21.359593 28236 catalog_manager.cc:1504] Loading token signing keys...
I20260810 04:41:21.364382 28218 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:41:21.361002 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38484 (local address 127.22.167.126:44493)
0810 04:41:21.361041 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0810 04:41:21.361043 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 04:41:21.361178 (+ 135us) server_negotiation.cc:407] Connection header received
0810 04:41:21.361233 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:41:21.361234 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:41:21.361246 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:41:21.361291 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0810 04:41:21.361658 (+ 367us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:41:21.362422 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:41:21.363129 (+ 707us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:41:21.363330 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:41:21.363385 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:41:21.363979 (+ 594us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 04:41:21.363981 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 04:41:21.363981 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 04:41:21.364152 (+ 171us) server_negotiation.cc:922] calling sasl_server_start()
0810 04:41:21.364168 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 04:41:21.364173 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 04:41:21.364225 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0810 04:41:21.364288 (+ 63us) server_negotiation.cc:299] Negotiation successful
0810 04:41:21.364311 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260810 04:41:21.374541 28236 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5a0e9f6b525648d2bf320ff6cb9b6220: Generated new TSK 0
I20260810 04:41:21.374727 28236 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 04:41:21.383495 23197 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskemv_AM/build/release/bin/kudu with pid 28086
2026-08-10T04:41:21Z chronyd exiting
[ OK ] SecurityITest.SaslPlainFallback (1450 ms)
[ RUN ] SecurityITest.TestUnauthorizedClientKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786336863388938-23197-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 04:41:21 dist-test-slave-bqcl krb5kdc[28271](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 04:41:21 dist-test-slave-bqcl krb5kdc[28271](info): set up 2 sockets
Aug 10 04:41:21 dist-test-slave-bqcl krb5kdc[28271](info): commencing operation
krb5kdc: starting...
W20260810 04:41:22.539392 23197 mini_kdc.cc:121] Time spent starting KDC: real 1.130s user 0.001s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786336863388938-23197-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786336863388938-23197-0/krb5kdc/test-user.keytab.
Aug 10 04:41:22 dist-test-slave-bqcl krb5kdc[28271](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786336882, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T04:41:22Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T04:41:22Z Disabled control of system clock
WARNING: no policy specified for kudu/127.22.167.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.167.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.167.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786336863388938-23197-0/krb5kdc/kudu_127.22.167.126.keytab.
Entry for principal kudu/127.22.167.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786336863388938-23197-0/krb5kdc/kudu_127.22.167.126.keytab.
I20260810 04:41:22.653179 23197 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskemv_AM/build/release/bin/kudu
/tmp/dist-test-taskemv_AM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786336863388938-23197-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786336863388938-23197-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786336863388938-23197-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786336863388938-23197-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.167.126:45649
--webserver_interface=127.22.167.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786336863388938-23197-0/krb5kdc/kudu_127.22.167.126.keytab
--principal=kudu/127.22.167.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.167.84:38165
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.167.126:45649
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786336863388938-23197-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786336863388938-23197-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786336863388938-23197-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 04:41:22.732846 28297 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 04:41:22.733034 28297 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 04:41:22.733071 28297 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 04:41:22.734179 28297 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 04:41:22.734217 28297 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 04:41:22.734236 28297 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 04:41:22.734251 28297 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 04:41:22.734265 28297 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 04:41:22.735689 28297 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.167.84:38165
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786336863388938-23197-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786336863388938-23197-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.22.167.126:45649
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786336863388938-23197-0/krb5kdc/kudu_127.22.167.126.keytab
--principal=kudu/127.22.167.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.167.126:45649
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786336863388938-23197-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.167.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28297
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786336863388938-23197-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 03:54:01 UTC on ccycloud.quasar-icijmt.root.comops.site
build id 4577009
I20260810 04:41:22.735911 28297 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 04:41:22.736106 28297 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260810 04:41:22.740592 28297 server_base.cc:1034] running on GCE node
W20260810 04:41:22.740571 28306 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 04:41:22.740654 28308 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 04:41:22.740705 28312 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 04:41:22.741078 28297 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 04:41:22.741353 28297 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 04:41:22.742506 28297 hybrid_clock.cc:648] HybridClock initialized: now 1786336882742486 us; error 40 us; skew 500 ppm
Aug 10 04:41:22 dist-test-slave-bqcl krb5kdc[28271](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786336882, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.167.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 04:41:22.745035 28297 init.cc:377] Logged in from keytab as kudu/127.22.167.126@KRBTEST.COM (short username kudu)
I20260810 04:41:22.745458 28297 webserver.cc:460] Webserver started at http://127.22.167.126:33709/ using document root <none> and password file <none>
I20260810 04:41:22.745631 28297 fs_manager.cc:359] Metadata directory not provided
I20260810 04:41:22.745671 28297 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 04:41:22.745759 28297 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 04:41:22.746711 28297 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786336863388938-23197-0/minicluster-data/master-0/data/instance:
uuid: "154fdb602d5846f5859542e59af2b45e"
format_stamp: "Formatted at 2026-08-10 04:41:22 on dist-test-slave-bqcl"
server_key: "eba2b54cf320737e95fa235a82768e8f"
server_key_iv: "d28e72352e519d331b0ac5e71ba933a9"
server_key_version: "encryptionkey@0"
I20260810 04:41:22.747010 28297 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786336863388938-23197-0/minicluster-data/master-0/wal/instance:
uuid: "154fdb602d5846f5859542e59af2b45e"
format_stamp: "Formatted at 2026-08-10 04:41:22 on dist-test-slave-bqcl"
server_key: "eba2b54cf320737e95fa235a82768e8f"
server_key_iv: "d28e72352e519d331b0ac5e71ba933a9"
server_key_version: "encryptionkey@0"
I20260810 04:41:22.748136 28297 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260810 04:41:22.748747 28327 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 04:41:22.748894 28297 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260810 04:41:22.748972 28297 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786336863388938-23197-0/minicluster-data/master-0/data,/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786336863388938-23197-0/minicluster-data/master-0/wal
uuid: "154fdb602d5846f5859542e59af2b45e"
format_stamp: "Formatted at 2026-08-10 04:41:22 on dist-test-slave-bqcl"
server_key: "eba2b54cf320737e95fa235a82768e8f"
server_key_iv: "d28e72352e519d331b0ac5e71ba933a9"
server_key_version: "encryptionkey@0"
I20260810 04:41:22.749034 28297 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786336863388938-23197-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786336863388938-23197-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786336863388938-23197-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 04:41:22.764348 28297 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 04:41:22.765919 28297 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 04:41:22.766039 28297 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 04:41:22.769851 28297 rpc_server.cc:307] RPC server started. Bound to: 127.22.167.126:45649
I20260810 04:41:22.769882 28423 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.167.126:45649 every 8 connection(s)
I20260810 04:41:22.770195 28297 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786336863388938-23197-0/minicluster-data/master-0/data/info.pb
I20260810 04:41:22.771303 28424 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 04:41:22.773407 28424 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 154fdb602d5846f5859542e59af2b45e: Bootstrap starting.
I20260810 04:41:22.773929 28424 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 154fdb602d5846f5859542e59af2b45e: Neither blocks nor log segments found. Creating new log.
I20260810 04:41:22.774216 28424 log.cc:824] T 00000000000000000000000000000000 P 154fdb602d5846f5859542e59af2b45e: Log is configured to *not* fsync() on all Append() calls
I20260810 04:41:22.774820 28424 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 154fdb602d5846f5859542e59af2b45e: No bootstrap required, opened a new log
I20260810 04:41:22.776465 28424 raft_consensus.cc:348] T 00000000000000000000000000000000 P 154fdb602d5846f5859542e59af2b45e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "154fdb602d5846f5859542e59af2b45e" member_type: VOTER last_known_addr { host: "127.22.167.126" port: 45649 } }
I20260810 04:41:22.776602 28424 raft_consensus.cc:374] T 00000000000000000000000000000000 P 154fdb602d5846f5859542e59af2b45e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 04:41:22.776623 28424 raft_consensus.cc:729] T 00000000000000000000000000000000 P 154fdb602d5846f5859542e59af2b45e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 154fdb602d5846f5859542e59af2b45e, State: Initialized, Role: FOLLOWER
I20260810 04:41:22.776728 28424 consensus_queue.cc:260] T 00000000000000000000000000000000 P 154fdb602d5846f5859542e59af2b45e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "154fdb602d5846f5859542e59af2b45e" member_type: VOTER last_known_addr { host: "127.22.167.126" port: 45649 } }
I20260810 04:41:22.776803 28424 raft_consensus.cc:388] T 00000000000000000000000000000000 P 154fdb602d5846f5859542e59af2b45e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 04:41:22.776830 28424 raft_consensus.cc:482] T 00000000000000000000000000000000 P 154fdb602d5846f5859542e59af2b45e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 04:41:22.776865 28424 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 154fdb602d5846f5859542e59af2b45e [term 0 FOLLOWER]: Advancing to term 1
I20260810 04:41:22.777468 23197 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskemv_AM/build/release/bin/kudu as pid 28297
I20260810 04:41:22.777587 23197 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786336863388938-23197-0/minicluster-data/master-0/wal/instance
I20260810 04:41:22.777705 28424 raft_consensus.cc:504] T 00000000000000000000000000000000 P 154fdb602d5846f5859542e59af2b45e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "154fdb602d5846f5859542e59af2b45e" member_type: VOTER last_known_addr { host: "127.22.167.126" port: 45649 } }
I20260810 04:41:22.777772 23197 external_mini_cluster.cc:1428] Setting key c1889f66d90a5954bfd00970a85ca4a5
I20260810 04:41:22.777810 28424 leader_election.cc:302] T 00000000000000000000000000000000 P 154fdb602d5846f5859542e59af2b45e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 154fdb602d5846f5859542e59af2b45e; no voters:
I20260810 04:41:22.777978 28424 leader_election.cc:288] T 00000000000000000000000000000000 P 154fdb602d5846f5859542e59af2b45e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 04:41:22.778012 28428 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 154fdb602d5846f5859542e59af2b45e [term 1 FOLLOWER]: Leader election won for term 1
I20260810 04:41:22.778142 28428 raft_consensus.cc:686] T 00000000000000000000000000000000 P 154fdb602d5846f5859542e59af2b45e [term 1 LEADER]: Becoming Leader. State: Replica: 154fdb602d5846f5859542e59af2b45e, State: Running, Role: LEADER
I20260810 04:41:22.778282 28428 consensus_queue.cc:237] T 00000000000000000000000000000000 P 154fdb602d5846f5859542e59af2b45e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "154fdb602d5846f5859542e59af2b45e" member_type: VOTER last_known_addr { host: "127.22.167.126" port: 45649 } }
I20260810 04:41:22.778280 28424 sys_catalog.cc:564] T 00000000000000000000000000000000 P 154fdb602d5846f5859542e59af2b45e [sys.catalog]: configured and running, proceeding with master startup.
I20260810 04:41:22.778664 28430 sys_catalog.cc:455] T 00000000000000000000000000000000 P 154fdb602d5846f5859542e59af2b45e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "154fdb602d5846f5859542e59af2b45e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "154fdb602d5846f5859542e59af2b45e" member_type: VOTER last_known_addr { host: "127.22.167.126" port: 45649 } } }
I20260810 04:41:22.778673 28433 sys_catalog.cc:455] T 00000000000000000000000000000000 P 154fdb602d5846f5859542e59af2b45e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 154fdb602d5846f5859542e59af2b45e. Latest consensus state: current_term: 1 leader_uuid: "154fdb602d5846f5859542e59af2b45e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "154fdb602d5846f5859542e59af2b45e" member_type: VOTER last_known_addr { host: "127.22.167.126" port: 45649 } } }
I20260810 04:41:22.778767 28430 sys_catalog.cc:458] T 00000000000000000000000000000000 P 154fdb602d5846f5859542e59af2b45e [sys.catalog]: This master's current role is: LEADER
I20260810 04:41:22.778770 28433 sys_catalog.cc:458] T 00000000000000000000000000000000 P 154fdb602d5846f5859542e59af2b45e [sys.catalog]: This master's current role is: LEADER
I20260810 04:41:22.779173 28439 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 04:41:22.779681 28439 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 04:41:22.781098 28439 catalog_manager.cc:1347] Generated new cluster ID: 15623ae7cb654f499260aad48eb9703a
I20260810 04:41:22.781144 28439 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 10 04:41:22 dist-test-slave-bqcl krb5kdc[28271](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786336882, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.167.126@KRBTEST.COM
I20260810 04:41:22.789254 28436 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:41:22.778381 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55334 (local address 127.22.167.126:45649)
0810 04:41:22.778521 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0810 04:41:22.778530 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 04:41:22.778560 (+ 30us) server_negotiation.cc:407] Connection header received
0810 04:41:22.778759 (+ 199us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:41:22.778784 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:41:22.778903 (+ 119us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:41:22.779834 (+ 931us) server_negotiation.cc:226] Negotiated authn=SASL
0810 04:41:22.779943 (+ 109us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:41:22.781350 (+ 1407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:41:22.782426 (+ 1076us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:41:22.783015 (+ 589us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:41:22.783111 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:41:22.785731 (+ 2620us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 04:41:22.785735 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 04:41:22.785736 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 04:41:22.785748 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0810 04:41:22.787672 (+ 1924us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 04:41:22.788039 (+ 367us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 04:41:22.788042 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 04:41:22.788042 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 04:41:22.788165 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 04:41:22.788344 (+ 179us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 04:41:22.788344 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 04:41:22.788345 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 04:41:22.788672 (+ 327us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 04:41:22.788832 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0810 04:41:22.788897 (+ 65us) server_negotiation.cc:299] Negotiation successful
0810 04:41:22.789041 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":96,"thread_start_us":69,"threads_started":1}
I20260810 04:41:22.794507 28439 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 04:41:22.794963 28439 catalog_manager.cc:1504] Loading token signing keys...
I20260810 04:41:22.796917 28439 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 154fdb602d5846f5859542e59af2b45e: Generated new TSK 0
I20260810 04:41:22.797060 28439 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.22.167.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.167.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.167.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786336863388938-23197-0/krb5kdc/kudu_127.22.167.65.keytab.
Entry for principal kudu/127.22.167.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786336863388938-23197-0/krb5kdc/kudu_127.22.167.65.keytab.
I20260810 04:41:22.824608 23197 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskemv_AM/build/release/bin/kudu
/tmp/dist-test-taskemv_AM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786336863388938-23197-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786336863388938-23197-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786336863388938-23197-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786336863388938-23197-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.167.65:0
--local_ip_for_outbound_sockets=127.22.167.65
--webserver_interface=127.22.167.65
--webserver_port=0
--tserver_master_addrs=127.22.167.126:45649
--keytab_file=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786336863388938-23197-0/krb5kdc/kudu_127.22.167.65.keytab
--principal=kudu/127.22.167.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.167.84:38165
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786336863388938-23197-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786336863388938-23197-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786336863388938-23197-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 04:41:22.905030 28466 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 04:41:22.905323 28466 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 04:41:22.905361 28466 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 04:41:22.906643 28466 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 04:41:22.906687 28466 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 04:41:22.906713 28466 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.167.65
I20260810 04:41:22.908104 28466 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.167.84:38165
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786336863388938-23197-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786336863388938-23197-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786336863388938-23197-0/krb5kdc/kudu_127.22.167.65.keytab
--principal=kudu/127.22.167.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.167.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786336863388938-23197-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.22.167.65
--webserver_port=0
--tserver_master_addrs=127.22.167.126:45649
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28466
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.167.65
--log_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786336863388938-23197-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 03:54:01 UTC on ccycloud.quasar-icijmt.root.comops.site
build id 4577009
I20260810 04:41:22.908317 28466 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 04:41:22.908521 28466 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 04:41:22.913285 28484 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 04:41:22.913348 28482 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 04:41:22.913302 28481 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 04:41:22.913534 28466 server_base.cc:1034] running on GCE node
I20260810 04:41:22.913738 28466 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 04:41:22.913993 28466 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 04:41:22.915160 28466 hybrid_clock.cc:648] HybridClock initialized: now 1786336882915140 us; error 32 us; skew 500 ppm
Aug 10 04:41:22 dist-test-slave-bqcl krb5kdc[28271](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786336882, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.167.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 04:41:22.917659 28466 init.cc:377] Logged in from keytab as kudu/127.22.167.65@KRBTEST.COM (short username kudu)
I20260810 04:41:22.918082 28466 webserver.cc:460] Webserver started at http://127.22.167.65:38823/ using document root <none> and password file <none>
I20260810 04:41:22.918303 28466 fs_manager.cc:359] Metadata directory not provided
I20260810 04:41:22.918351 28466 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 04:41:22.918473 28466 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 04:41:22.919466 28466 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786336863388938-23197-0/minicluster-data/ts-0/data/instance:
uuid: "295c30d7a88e480884991c52ef5fc870"
format_stamp: "Formatted at 2026-08-10 04:41:22 on dist-test-slave-bqcl"
server_key: "67d8a8c3db928696ee6b596cec198ecb"
server_key_iv: "f942928d2fe370d25e0eeb7a9b46aa8e"
server_key_version: "encryptionkey@0"
I20260810 04:41:22.919801 28466 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786336863388938-23197-0/minicluster-data/ts-0/wal/instance:
uuid: "295c30d7a88e480884991c52ef5fc870"
format_stamp: "Formatted at 2026-08-10 04:41:22 on dist-test-slave-bqcl"
server_key: "67d8a8c3db928696ee6b596cec198ecb"
server_key_iv: "f942928d2fe370d25e0eeb7a9b46aa8e"
server_key_version: "encryptionkey@0"
I20260810 04:41:22.921085 28466 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260810 04:41:22.921722 28499 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 04:41:22.921875 28466 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260810 04:41:22.921957 28466 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786336863388938-23197-0/minicluster-data/ts-0/data,/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786336863388938-23197-0/minicluster-data/ts-0/wal
uuid: "295c30d7a88e480884991c52ef5fc870"
format_stamp: "Formatted at 2026-08-10 04:41:22 on dist-test-slave-bqcl"
server_key: "67d8a8c3db928696ee6b596cec198ecb"
server_key_iv: "f942928d2fe370d25e0eeb7a9b46aa8e"
server_key_version: "encryptionkey@0"
I20260810 04:41:22.922021 28466 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786336863388938-23197-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786336863388938-23197-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786336863388938-23197-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 04:41:22.938707 28466 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 04:41:22.940584 28466 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 04:41:22.940730 28466 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 04:41:22.940956 28466 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 04:41:22.941280 28466 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 04:41:22.941319 28466 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 04:41:22.941351 28466 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 04:41:22.941378 28466 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 04:41:22.947515 28466 rpc_server.cc:307] RPC server started. Bound to: 127.22.167.65:35973
I20260810 04:41:22.947551 28705 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.167.65:35973 every 8 connection(s)
I20260810 04:41:22.947858 28466 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786336863388938-23197-0/minicluster-data/ts-0/data/info.pb
I20260810 04:41:22.949012 23197 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskemv_AM/build/release/bin/kudu as pid 28466
I20260810 04:41:22.949126 23197 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786336863388938-23197-0/minicluster-data/ts-0/wal/instance
I20260810 04:41:22.949324 23197 external_mini_cluster.cc:1428] Setting key 4df282e9f1b8acbcc4417346c633a4e1
Aug 10 04:41:22 dist-test-slave-bqcl krb5kdc[28271](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786336882, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.167.65@KRBTEST.COM for kudu/127.22.167.126@KRBTEST.COM
I20260810 04:41:22.957886 28436 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:41:22.948538 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.167.65:38665 (local address 127.22.167.126:45649)
0810 04:41:22.948599 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0810 04:41:22.948601 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 04:41:22.948787 (+ 186us) server_negotiation.cc:407] Connection header received
0810 04:41:22.949411 (+ 624us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:41:22.949412 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:41:22.949428 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:41:22.949514 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0810 04:41:22.950754 (+ 1240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:41:22.951255 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:41:22.952433 (+ 1178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:41:22.952820 (+ 387us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:41:22.952883 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:41:22.955196 (+ 2313us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 04:41:22.955197 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 04:41:22.955198 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 04:41:22.955206 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0810 04:41:22.956556 (+ 1350us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 04:41:22.956949 (+ 393us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 04:41:22.956950 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 04:41:22.956951 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 04:41:22.957014 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 04:41:22.957249 (+ 235us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 04:41:22.957250 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 04:41:22.957250 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 04:41:22.957366 (+ 116us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 04:41:22.957430 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0810 04:41:22.957698 (+ 268us) server_negotiation.cc:299] Negotiation successful
0810 04:41:22.957790 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
WARNING: no policy specified for kudu/127.22.167.66@KRBTEST.COM; defaulting to no policy
I20260810 04:41:22.958042 28710 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:41:22.948500 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.126:45649 (local address 127.22.167.65:38665)
0810 04:41:22.948670 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0810 04:41:22.948711 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0810 04:41:22.949307 (+ 596us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 04:41:22.949667 (+ 360us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 04:41:22.949672 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 04:41:22.950384 (+ 712us) client_negotiation.cc:189] Negotiated authn=SASL
0810 04:41:22.950658 (+ 274us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:41:22.950665 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:41:22.951342 (+ 677us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:41:22.951343 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:41:22.952349 (+ 1006us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:41:22.952352 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:41:22.952914 (+ 562us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:41:22.952915 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:41:22.953047 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:41:22.953502 (+ 455us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 04:41:22.953511 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0810 04:41:22.955101 (+ 1590us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 04:41:22.956640 (+ 1539us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 04:41:22.956647 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 04:41:22.956649 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 04:41:22.956874 (+ 225us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 04:41:22.957075 (+ 201us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 04:41:22.957076 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 04:41:22.957076 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 04:41:22.957187 (+ 111us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 04:41:22.957428 (+ 241us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 04:41:22.957435 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 04:41:22.957571 (+ 136us) client_negotiation.cc:769] Sending connection context
0810 04:41:22.957666 (+ 95us) client_negotiation.cc:240] Negotiation successful
0810 04:41:22.957820 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":78,"threads_started":1}
I20260810 04:41:22.958559 28706 heartbeater.cc:347] Connected to a master server at 127.22.167.126:45649
I20260810 04:41:22.958662 28706 heartbeater.cc:464] Registering TS with master...
I20260810 04:41:22.958827 28706 heartbeater.cc:511] Master 127.22.167.126:45649 requested a full tablet report, sending...
I20260810 04:41:22.959378 28368 ts_manager.cc:194] Registered new tserver with Master: 295c30d7a88e480884991c52ef5fc870 (127.22.167.65:35973)
I20260810 04:41:22.959959 28368 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.167.65@KRBTEST.COM'} at 127.22.167.65:38665
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.167.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.167.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786336863388938-23197-0/krb5kdc/kudu_127.22.167.66.keytab.
Entry for principal kudu/127.22.167.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786336863388938-23197-0/krb5kdc/kudu_127.22.167.66.keytab.
I20260810 04:41:22.975059 23197 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskemv_AM/build/release/bin/kudu
/tmp/dist-test-taskemv_AM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786336863388938-23197-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786336863388938-23197-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786336863388938-23197-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786336863388938-23197-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.167.66:0
--local_ip_for_outbound_sockets=127.22.167.66
--webserver_interface=127.22.167.66
--webserver_port=0
--tserver_master_addrs=127.22.167.126:45649
--keytab_file=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786336863388938-23197-0/krb5kdc/kudu_127.22.167.66.keytab
--principal=kudu/127.22.167.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.167.84:38165
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786336863388938-23197-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786336863388938-23197-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786336863388938-23197-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 04:41:23.054157 28715 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 04:41:23.054349 28715 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 04:41:23.054370 28715 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 04:41:23.055465 28715 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 04:41:23.055497 28715 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 04:41:23.055529 28715 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.167.66
I20260810 04:41:23.056903 28715 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.167.84:38165
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786336863388938-23197-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786336863388938-23197-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786336863388938-23197-0/krb5kdc/kudu_127.22.167.66.keytab
--principal=kudu/127.22.167.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.167.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786336863388938-23197-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.22.167.66
--webserver_port=0
--tserver_master_addrs=127.22.167.126:45649
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28715
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.167.66
--log_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786336863388938-23197-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 03:54:01 UTC on ccycloud.quasar-icijmt.root.comops.site
build id 4577009
I20260810 04:41:23.057078 28715 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 04:41:23.057240 28715 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 04:41:23.061987 28722 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 04:41:23.062000 28728 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 04:41:23.062063 28715 server_base.cc:1034] running on GCE node
W20260810 04:41:23.062000 28720 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 04:41:23.062371 28715 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 04:41:23.062614 28715 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 04:41:23.063773 28715 hybrid_clock.cc:648] HybridClock initialized: now 1786336883063751 us; error 33 us; skew 500 ppm
Aug 10 04:41:23 dist-test-slave-bqcl krb5kdc[28271](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786336883, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.167.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 04:41:23.066334 28715 init.cc:377] Logged in from keytab as kudu/127.22.167.66@KRBTEST.COM (short username kudu)
I20260810 04:41:23.066713 28715 webserver.cc:460] Webserver started at http://127.22.167.66:45641/ using document root <none> and password file <none>
I20260810 04:41:23.066902 28715 fs_manager.cc:359] Metadata directory not provided
I20260810 04:41:23.066939 28715 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 04:41:23.067025 28715 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 04:41:23.067961 28715 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786336863388938-23197-0/minicluster-data/ts-1/data/instance:
uuid: "9adb472eb3894453b628cbc3bbbe505f"
format_stamp: "Formatted at 2026-08-10 04:41:23 on dist-test-slave-bqcl"
server_key: "982bf287b0a797a2503be0143d6aeefc"
server_key_iv: "b95ef7d94a638884314a29f152af8db0"
server_key_version: "encryptionkey@0"
I20260810 04:41:23.068249 28715 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786336863388938-23197-0/minicluster-data/ts-1/wal/instance:
uuid: "9adb472eb3894453b628cbc3bbbe505f"
format_stamp: "Formatted at 2026-08-10 04:41:23 on dist-test-slave-bqcl"
server_key: "982bf287b0a797a2503be0143d6aeefc"
server_key_iv: "b95ef7d94a638884314a29f152af8db0"
server_key_version: "encryptionkey@0"
I20260810 04:41:23.069442 28715 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260810 04:41:23.070160 28741 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 04:41:23.070303 28715 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260810 04:41:23.070381 28715 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786336863388938-23197-0/minicluster-data/ts-1/data,/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786336863388938-23197-0/minicluster-data/ts-1/wal
uuid: "9adb472eb3894453b628cbc3bbbe505f"
format_stamp: "Formatted at 2026-08-10 04:41:23 on dist-test-slave-bqcl"
server_key: "982bf287b0a797a2503be0143d6aeefc"
server_key_iv: "b95ef7d94a638884314a29f152af8db0"
server_key_version: "encryptionkey@0"
I20260810 04:41:23.070451 28715 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786336863388938-23197-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786336863388938-23197-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786336863388938-23197-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 04:41:23.096058 28715 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 04:41:23.098083 28715 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 04:41:23.098232 28715 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 04:41:23.098438 28715 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 04:41:23.098734 28715 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 04:41:23.098767 28715 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 04:41:23.098794 28715 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 04:41:23.098811 28715 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 04:41:23.104040 28715 rpc_server.cc:307] RPC server started. Bound to: 127.22.167.66:40155
I20260810 04:41:23.104110 28943 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.167.66:40155 every 8 connection(s)
I20260810 04:41:23.104365 28715 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786336863388938-23197-0/minicluster-data/ts-1/data/info.pb
I20260810 04:41:23.109524 23197 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskemv_AM/build/release/bin/kudu as pid 28715
I20260810 04:41:23.109651 23197 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786336863388938-23197-0/minicluster-data/ts-1/wal/instance
I20260810 04:41:23.109850 23197 external_mini_cluster.cc:1428] Setting key b201d8ad9a8dbd887a11ca3e1740c4d6
Aug 10 04:41:23 dist-test-slave-bqcl krb5kdc[28271](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786336883, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.167.66@KRBTEST.COM for kudu/127.22.167.126@KRBTEST.COM
I20260810 04:41:23.113706 28436 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:41:23.105020 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.167.66:55649 (local address 127.22.167.126:45649)
0810 04:41:23.105080 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0810 04:41:23.105083 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 04:41:23.105259 (+ 176us) server_negotiation.cc:407] Connection header received
0810 04:41:23.105750 (+ 491us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:41:23.105752 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:41:23.105764 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:41:23.105812 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0810 04:41:23.106754 (+ 942us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:41:23.107256 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:41:23.108224 (+ 968us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:41:23.108445 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:41:23.108502 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:41:23.110753 (+ 2251us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 04:41:23.110754 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 04:41:23.110755 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 04:41:23.110762 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 04:41:23.112333 (+ 1571us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 04:41:23.112836 (+ 503us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 04:41:23.112837 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 04:41:23.112838 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 04:41:23.112893 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 04:41:23.113141 (+ 248us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 04:41:23.113142 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 04:41:23.113142 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 04:41:23.113237 (+ 95us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 04:41:23.113280 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0810 04:41:23.113571 (+ 291us) server_negotiation.cc:299] Negotiation successful
0810 04:41:23.113643 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260810 04:41:23.113931 28947 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:41:23.104973 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.126:45649 (local address 127.22.167.66:55649)
0810 04:41:23.105139 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0810 04:41:23.105179 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0810 04:41:23.105657 (+ 478us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 04:41:23.105892 (+ 235us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 04:41:23.105897 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 04:41:23.106478 (+ 581us) client_negotiation.cc:189] Negotiated authn=SASL
0810 04:41:23.106663 (+ 185us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:41:23.106668 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:41:23.107337 (+ 669us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:41:23.107338 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:41:23.108142 (+ 804us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:41:23.108144 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:41:23.108513 (+ 369us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:41:23.108514 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:41:23.108632 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:41:23.109010 (+ 378us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 04:41:23.109018 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0810 04:41:23.110666 (+ 1648us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 04:41:23.112417 (+ 1751us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 04:41:23.112424 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 04:41:23.112426 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 04:41:23.112749 (+ 323us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 04:41:23.112944 (+ 195us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 04:41:23.112944 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 04:41:23.112945 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 04:41:23.113083 (+ 138us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 04:41:23.113282 (+ 199us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 04:41:23.113287 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 04:41:23.113440 (+ 153us) client_negotiation.cc:769] Sending connection context
0810 04:41:23.113559 (+ 119us) client_negotiation.cc:240] Negotiation successful
0810 04:41:23.113717 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":83,"threads_started":1}
I20260810 04:41:23.114411 28945 heartbeater.cc:347] Connected to a master server at 127.22.167.126:45649
I20260810 04:41:23.114498 28945 heartbeater.cc:464] Registering TS with master...
I20260810 04:41:23.114666 28945 heartbeater.cc:511] Master 127.22.167.126:45649 requested a full tablet report, sending...
I20260810 04:41:23.115041 28368 ts_manager.cc:194] Registered new tserver with Master: 9adb472eb3894453b628cbc3bbbe505f (127.22.167.66:40155)
I20260810 04:41:23.115404 28368 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.167.66@KRBTEST.COM'} at 127.22.167.66:55649
WARNING: no policy specified for kudu/127.22.167.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.167.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.167.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786336863388938-23197-0/krb5kdc/kudu_127.22.167.67.keytab.
Entry for principal kudu/127.22.167.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786336863388938-23197-0/krb5kdc/kudu_127.22.167.67.keytab.
I20260810 04:41:23.135234 23197 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskemv_AM/build/release/bin/kudu
/tmp/dist-test-taskemv_AM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786336863388938-23197-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786336863388938-23197-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786336863388938-23197-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786336863388938-23197-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.167.67:0
--local_ip_for_outbound_sockets=127.22.167.67
--webserver_interface=127.22.167.67
--webserver_port=0
--tserver_master_addrs=127.22.167.126:45649
--keytab_file=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786336863388938-23197-0/krb5kdc/kudu_127.22.167.67.keytab
--principal=kudu/127.22.167.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.167.84:38165
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786336863388938-23197-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786336863388938-23197-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786336863388938-23197-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 04:41:23.214669 28953 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 04:41:23.214859 28953 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 04:41:23.214880 28953 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 04:41:23.215969 28953 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 04:41:23.216009 28953 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 04:41:23.216034 28953 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.167.67
I20260810 04:41:23.217408 28953 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.167.84:38165
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786336863388938-23197-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786336863388938-23197-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786336863388938-23197-0/krb5kdc/kudu_127.22.167.67.keytab
--principal=kudu/127.22.167.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.167.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786336863388938-23197-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.22.167.67
--webserver_port=0
--tserver_master_addrs=127.22.167.126:45649
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28953
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.167.67
--log_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786336863388938-23197-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 03:54:01 UTC on ccycloud.quasar-icijmt.root.comops.site
build id 4577009
I20260810 04:41:23.217613 28953 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 04:41:23.217818 28953 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 04:41:23.222684 28963 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 04:41:23.222760 28968 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 04:41:23.222900 28953 server_base.cc:1034] running on GCE node
W20260810 04:41:23.222760 28962 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 04:41:23.223160 28953 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 04:41:23.223377 28953 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 04:41:23.224524 28953 hybrid_clock.cc:648] HybridClock initialized: now 1786336883224491 us; error 48 us; skew 500 ppm
Aug 10 04:41:23 dist-test-slave-bqcl krb5kdc[28271](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786336883, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.167.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 04:41:23.226979 28953 init.cc:377] Logged in from keytab as kudu/127.22.167.67@KRBTEST.COM (short username kudu)
I20260810 04:41:23.227411 28953 webserver.cc:460] Webserver started at http://127.22.167.67:35717/ using document root <none> and password file <none>
I20260810 04:41:23.227607 28953 fs_manager.cc:359] Metadata directory not provided
I20260810 04:41:23.227651 28953 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 04:41:23.227763 28953 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 04:41:23.228765 28953 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786336863388938-23197-0/minicluster-data/ts-2/data/instance:
uuid: "e1590de01f2c45e08c0a063ff0344fdd"
format_stamp: "Formatted at 2026-08-10 04:41:23 on dist-test-slave-bqcl"
server_key: "ed812d7bf219433455060b9c2d7c6001"
server_key_iv: "c849d7b17200bed4ec5518db923e6488"
server_key_version: "encryptionkey@0"
I20260810 04:41:23.229107 28953 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786336863388938-23197-0/minicluster-data/ts-2/wal/instance:
uuid: "e1590de01f2c45e08c0a063ff0344fdd"
format_stamp: "Formatted at 2026-08-10 04:41:23 on dist-test-slave-bqcl"
server_key: "ed812d7bf219433455060b9c2d7c6001"
server_key_iv: "c849d7b17200bed4ec5518db923e6488"
server_key_version: "encryptionkey@0"
I20260810 04:41:23.230352 28953 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260810 04:41:23.231022 28982 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 04:41:23.231180 28953 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260810 04:41:23.231263 28953 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786336863388938-23197-0/minicluster-data/ts-2/data,/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786336863388938-23197-0/minicluster-data/ts-2/wal
uuid: "e1590de01f2c45e08c0a063ff0344fdd"
format_stamp: "Formatted at 2026-08-10 04:41:23 on dist-test-slave-bqcl"
server_key: "ed812d7bf219433455060b9c2d7c6001"
server_key_iv: "c849d7b17200bed4ec5518db923e6488"
server_key_version: "encryptionkey@0"
I20260810 04:41:23.231321 28953 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786336863388938-23197-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786336863388938-23197-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786336863388938-23197-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 04:41:23.255553 28953 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 04:41:23.257479 28953 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 04:41:23.257614 28953 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 04:41:23.257833 28953 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 04:41:23.258154 28953 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 04:41:23.258199 28953 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 04:41:23.258229 28953 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 04:41:23.258252 28953 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 04:41:23.263620 28953 rpc_server.cc:307] RPC server started. Bound to: 127.22.167.67:35657
I20260810 04:41:23.263677 29179 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.167.67:35657 every 8 connection(s)
I20260810 04:41:23.263963 28953 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786336863388938-23197-0/minicluster-data/ts-2/data/info.pb
Aug 10 04:41:23 dist-test-slave-bqcl krb5kdc[28271](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786336883, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.167.67@KRBTEST.COM for kudu/127.22.167.126@KRBTEST.COM
I20260810 04:41:23.270077 23197 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskemv_AM/build/release/bin/kudu as pid 28953
I20260810 04:41:23.270213 23197 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786336863388938-23197-0/minicluster-data/ts-2/wal/instance
I20260810 04:41:23.270406 23197 external_mini_cluster.cc:1428] Setting key c7ab0751d833691e7f2c21b607564a2b
I20260810 04:41:23.272883 28436 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:41:23.264620 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.167.67:35891 (local address 127.22.167.126:45649)
0810 04:41:23.264696 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0810 04:41:23.264699 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 04:41:23.264840 (+ 141us) server_negotiation.cc:407] Connection header received
0810 04:41:23.265330 (+ 490us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:41:23.265331 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:41:23.265341 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:41:23.265375 (+ 34us) server_negotiation.cc:226] Negotiated authn=SASL
0810 04:41:23.266282 (+ 907us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:41:23.266765 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:41:23.267731 (+ 966us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:41:23.267915 (+ 184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:41:23.267966 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:41:23.270116 (+ 2150us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 04:41:23.270117 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 04:41:23.270118 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 04:41:23.270157 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0810 04:41:23.271612 (+ 1455us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 04:41:23.271970 (+ 358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 04:41:23.271971 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 04:41:23.271971 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 04:41:23.272036 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 04:41:23.272264 (+ 228us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 04:41:23.272265 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 04:41:23.272265 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 04:41:23.272381 (+ 116us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 04:41:23.272452 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0810 04:41:23.272713 (+ 261us) server_negotiation.cc:299] Negotiation successful
0810 04:41:23.272796 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260810 04:41:23.273108 29186 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:41:23.264589 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.126:45649 (local address 127.22.167.67:35891)
0810 04:41:23.264731 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0810 04:41:23.264767 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0810 04:41:23.265243 (+ 476us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 04:41:23.265435 (+ 192us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 04:41:23.265438 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 04:41:23.265995 (+ 557us) client_negotiation.cc:189] Negotiated authn=SASL
0810 04:41:23.266213 (+ 218us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:41:23.266219 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:41:23.266858 (+ 639us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:41:23.266859 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:41:23.267664 (+ 805us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:41:23.267666 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:41:23.268017 (+ 351us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:41:23.268018 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:41:23.268136 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:41:23.268624 (+ 488us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 04:41:23.268632 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0810 04:41:23.270005 (+ 1373us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 04:41:23.271683 (+ 1678us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 04:41:23.271690 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 04:41:23.271692 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 04:41:23.271903 (+ 211us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 04:41:23.272091 (+ 188us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 04:41:23.272092 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 04:41:23.272092 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 04:41:23.272203 (+ 111us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 04:41:23.272441 (+ 238us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 04:41:23.272448 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 04:41:23.272590 (+ 142us) client_negotiation.cc:769] Sending connection context
0810 04:41:23.272678 (+ 88us) client_negotiation.cc:240] Negotiation successful
0810 04:41:23.272832 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":71,"threads_started":1}
I20260810 04:41:23.273514 29182 heartbeater.cc:347] Connected to a master server at 127.22.167.126:45649
I20260810 04:41:23.273617 29182 heartbeater.cc:464] Registering TS with master...
I20260810 04:41:23.273797 29182 heartbeater.cc:511] Master 127.22.167.126:45649 requested a full tablet report, sending...
I20260810 04:41:23.274231 28368 ts_manager.cc:194] Registered new tserver with Master: e1590de01f2c45e08c0a063ff0344fdd (127.22.167.67:35657)
I20260810 04:41:23.274605 28368 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.167.67@KRBTEST.COM'} at 127.22.167.67:35891
I20260810 04:41:23.282037 23197 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 10 04:41:23 dist-test-slave-bqcl krb5kdc[28271](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786336883, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for joe-interloper@KRBTEST.COM:
Aug 10 04:41:23 dist-test-slave-bqcl krb5kdc[28271](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786336883, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.22.167.126@KRBTEST.COM
I20260810 04:41:23.304447 28436 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:41:23.298062 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55338 (local address 127.22.167.126:45649)
0810 04:41:23.298113 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0810 04:41:23.298115 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 04:41:23.298234 (+ 119us) server_negotiation.cc:407] Connection header received
0810 04:41:23.298296 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:41:23.298297 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:41:23.298307 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:41:23.298354 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0810 04:41:23.298890 (+ 536us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:41:23.299380 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:41:23.300068 (+ 688us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:41:23.300253 (+ 185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:41:23.300306 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:41:23.302399 (+ 2093us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 04:41:23.302400 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 04:41:23.302400 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 04:41:23.302407 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 04:41:23.303615 (+ 1208us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 04:41:23.303875 (+ 260us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 04:41:23.303875 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 04:41:23.303875 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 04:41:23.303920 (+ 45us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 04:41:23.304047 (+ 127us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 04:41:23.304048 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 04:41:23.304048 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 04:41:23.304177 (+ 129us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 04:41:23.304249 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0810 04:41:23.304322 (+ 73us) server_negotiation.cc:299] Negotiation successful
0810 04:41:23.304377 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
W20260810 04:41:23.304689 28368 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.ConnectToMaster from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:55338
I20260810 04:41:23.305086 23197 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskemv_AM/build/release/bin/kudu with pid 28466
I20260810 04:41:23.318707 23197 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskemv_AM/build/release/bin/kudu with pid 28715
I20260810 04:41:23.329748 23197 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskemv_AM/build/release/bin/kudu with pid 28953
I20260810 04:41:23.337970 23197 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskemv_AM/build/release/bin/kudu with pid 28297
2026-08-10T04:41:23Z chronyd exiting
[ OK ] SecurityITest.TestUnauthorizedClientKerberosCredentials (1950 ms)
[ RUN ] SecurityITest.TestAuthorizedSuperuser
Loading random data
Initializing database '/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786336863388938-23197-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 04:41:23 dist-test-slave-bqcl krb5kdc[29207](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 04:41:23 dist-test-slave-bqcl krb5kdc[29207](info): set up 2 sockets
Aug 10 04:41:23 dist-test-slave-bqcl krb5kdc[29207](info): commencing operation
krb5kdc: starting...
W20260810 04:41:24.488780 23197 mini_kdc.cc:121] Time spent starting KDC: real 1.129s user 0.002s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786336863388938-23197-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786336863388938-23197-0/krb5kdc/test-user.keytab.
Aug 10 04:41:24 dist-test-slave-bqcl krb5kdc[29207](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786336884, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T04:41:24Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T04:41:24Z Disabled control of system clock
WARNING: no policy specified for kudu/127.22.167.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.167.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.167.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786336863388938-23197-0/krb5kdc/kudu_127.22.167.126.keytab.
Entry for principal kudu/127.22.167.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786336863388938-23197-0/krb5kdc/kudu_127.22.167.126.keytab.
I20260810 04:41:24.600797 23197 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskemv_AM/build/release/bin/kudu
/tmp/dist-test-taskemv_AM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786336863388938-23197-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786336863388938-23197-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786336863388938-23197-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786336863388938-23197-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.167.126:35119
--webserver_interface=127.22.167.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786336863388938-23197-0/krb5kdc/kudu_127.22.167.126.keytab
--principal=kudu/127.22.167.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.167.84:39223
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.167.126:35119
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786336863388938-23197-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786336863388938-23197-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786336863388938-23197-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 04:41:24.679718 29226 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 04:41:24.679939 29226 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 04:41:24.679967 29226 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 04:41:24.681083 29226 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 04:41:24.681123 29226 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 04:41:24.681142 29226 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 04:41:24.681157 29226 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 04:41:24.681172 29226 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 04:41:24.682669 29226 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.167.84:39223
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786336863388938-23197-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786336863388938-23197-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.22.167.126:35119
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786336863388938-23197-0/krb5kdc/kudu_127.22.167.126.keytab
--principal=kudu/127.22.167.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.167.126:35119
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786336863388938-23197-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.167.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29226
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786336863388938-23197-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 03:54:01 UTC on ccycloud.quasar-icijmt.root.comops.site
build id 4577009
I20260810 04:41:24.682907 29226 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 04:41:24.683106 29226 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260810 04:41:24.687741 29226 server_base.cc:1034] running on GCE node
W20260810 04:41:24.687660 29232 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 04:41:24.687667 29236 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 04:41:24.687668 29233 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 04:41:24.688151 29226 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 04:41:24.688421 29226 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 04:41:24.689558 29226 hybrid_clock.cc:648] HybridClock initialized: now 1786336884689540 us; error 31 us; skew 500 ppm
Aug 10 04:41:24 dist-test-slave-bqcl krb5kdc[29207](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786336884, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.167.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 04:41:24.691864 29226 init.cc:377] Logged in from keytab as kudu/127.22.167.126@KRBTEST.COM (short username kudu)
I20260810 04:41:24.692265 29226 webserver.cc:460] Webserver started at http://127.22.167.126:35937/ using document root <none> and password file <none>
I20260810 04:41:24.692454 29226 fs_manager.cc:359] Metadata directory not provided
I20260810 04:41:24.692502 29226 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 04:41:24.692632 29226 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 04:41:24.693590 29226 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786336863388938-23197-0/minicluster-data/master-0/data/instance:
uuid: "fb5984eebdce440196aefb71d446eec3"
format_stamp: "Formatted at 2026-08-10 04:41:24 on dist-test-slave-bqcl"
server_key: "8d03422180e6c737a586072c9a57c94d"
server_key_iv: "97d75906a6eebab1d83e12d299434b72"
server_key_version: "encryptionkey@0"
I20260810 04:41:24.693905 29226 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786336863388938-23197-0/minicluster-data/master-0/wal/instance:
uuid: "fb5984eebdce440196aefb71d446eec3"
format_stamp: "Formatted at 2026-08-10 04:41:24 on dist-test-slave-bqcl"
server_key: "8d03422180e6c737a586072c9a57c94d"
server_key_iv: "97d75906a6eebab1d83e12d299434b72"
server_key_version: "encryptionkey@0"
I20260810 04:41:24.695168 29226 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260810 04:41:24.695794 29249 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 04:41:24.695940 29226 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260810 04:41:24.696036 29226 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786336863388938-23197-0/minicluster-data/master-0/data,/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786336863388938-23197-0/minicluster-data/master-0/wal
uuid: "fb5984eebdce440196aefb71d446eec3"
format_stamp: "Formatted at 2026-08-10 04:41:24 on dist-test-slave-bqcl"
server_key: "8d03422180e6c737a586072c9a57c94d"
server_key_iv: "97d75906a6eebab1d83e12d299434b72"
server_key_version: "encryptionkey@0"
I20260810 04:41:24.696111 29226 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786336863388938-23197-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786336863388938-23197-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786336863388938-23197-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 04:41:24.745795 29226 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 04:41:24.747712 29226 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 04:41:24.747840 29226 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 04:41:24.751693 29226 rpc_server.cc:307] RPC server started. Bound to: 127.22.167.126:35119
I20260810 04:41:24.751732 29348 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.167.126:35119 every 8 connection(s)
I20260810 04:41:24.752200 29226 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786336863388938-23197-0/minicluster-data/master-0/data/info.pb
I20260810 04:41:24.752877 29350 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 04:41:24.755110 29350 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fb5984eebdce440196aefb71d446eec3: Bootstrap starting.
I20260810 04:41:24.755395 23197 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskemv_AM/build/release/bin/kudu as pid 29226
I20260810 04:41:24.755494 23197 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786336863388938-23197-0/minicluster-data/master-0/wal/instance
I20260810 04:41:24.755647 23197 external_mini_cluster.cc:1428] Setting key a729680baacced1d8fac2d06b07de367
I20260810 04:41:24.755733 29350 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P fb5984eebdce440196aefb71d446eec3: Neither blocks nor log segments found. Creating new log.
I20260810 04:41:24.756078 29350 log.cc:824] T 00000000000000000000000000000000 P fb5984eebdce440196aefb71d446eec3: Log is configured to *not* fsync() on all Append() calls
I20260810 04:41:24.756860 29350 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fb5984eebdce440196aefb71d446eec3: No bootstrap required, opened a new log
I20260810 04:41:24.759215 29350 raft_consensus.cc:348] T 00000000000000000000000000000000 P fb5984eebdce440196aefb71d446eec3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fb5984eebdce440196aefb71d446eec3" member_type: VOTER last_known_addr { host: "127.22.167.126" port: 35119 } }
I20260810 04:41:24.759370 29350 raft_consensus.cc:374] T 00000000000000000000000000000000 P fb5984eebdce440196aefb71d446eec3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 04:41:24.759408 29350 raft_consensus.cc:729] T 00000000000000000000000000000000 P fb5984eebdce440196aefb71d446eec3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fb5984eebdce440196aefb71d446eec3, State: Initialized, Role: FOLLOWER
I20260810 04:41:24.759536 29350 consensus_queue.cc:260] T 00000000000000000000000000000000 P fb5984eebdce440196aefb71d446eec3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fb5984eebdce440196aefb71d446eec3" member_type: VOTER last_known_addr { host: "127.22.167.126" port: 35119 } }
I20260810 04:41:24.759608 29350 raft_consensus.cc:388] T 00000000000000000000000000000000 P fb5984eebdce440196aefb71d446eec3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 04:41:24.759637 29350 raft_consensus.cc:482] T 00000000000000000000000000000000 P fb5984eebdce440196aefb71d446eec3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 04:41:24.759684 29350 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fb5984eebdce440196aefb71d446eec3 [term 0 FOLLOWER]: Advancing to term 1
I20260810 04:41:24.760290 29350 raft_consensus.cc:504] T 00000000000000000000000000000000 P fb5984eebdce440196aefb71d446eec3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fb5984eebdce440196aefb71d446eec3" member_type: VOTER last_known_addr { host: "127.22.167.126" port: 35119 } }
I20260810 04:41:24.760398 29350 leader_election.cc:302] T 00000000000000000000000000000000 P fb5984eebdce440196aefb71d446eec3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: fb5984eebdce440196aefb71d446eec3; no voters:
I20260810 04:41:24.760560 29350 leader_election.cc:288] T 00000000000000000000000000000000 P fb5984eebdce440196aefb71d446eec3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 04:41:24.760622 29362 raft_consensus.cc:2781] T 00000000000000000000000000000000 P fb5984eebdce440196aefb71d446eec3 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 04:41:24.760807 29362 raft_consensus.cc:686] T 00000000000000000000000000000000 P fb5984eebdce440196aefb71d446eec3 [term 1 LEADER]: Becoming Leader. State: Replica: fb5984eebdce440196aefb71d446eec3, State: Running, Role: LEADER
I20260810 04:41:24.760812 29350 sys_catalog.cc:564] T 00000000000000000000000000000000 P fb5984eebdce440196aefb71d446eec3 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 04:41:24.760938 29362 consensus_queue.cc:237] T 00000000000000000000000000000000 P fb5984eebdce440196aefb71d446eec3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fb5984eebdce440196aefb71d446eec3" member_type: VOTER last_known_addr { host: "127.22.167.126" port: 35119 } }
I20260810 04:41:24.761265 29362 sys_catalog.cc:455] T 00000000000000000000000000000000 P fb5984eebdce440196aefb71d446eec3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader fb5984eebdce440196aefb71d446eec3. Latest consensus state: current_term: 1 leader_uuid: "fb5984eebdce440196aefb71d446eec3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fb5984eebdce440196aefb71d446eec3" member_type: VOTER last_known_addr { host: "127.22.167.126" port: 35119 } } }
I20260810 04:41:24.761343 29362 sys_catalog.cc:458] T 00000000000000000000000000000000 P fb5984eebdce440196aefb71d446eec3 [sys.catalog]: This master's current role is: LEADER
I20260810 04:41:24.761289 29364 sys_catalog.cc:455] T 00000000000000000000000000000000 P fb5984eebdce440196aefb71d446eec3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "fb5984eebdce440196aefb71d446eec3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fb5984eebdce440196aefb71d446eec3" member_type: VOTER last_known_addr { host: "127.22.167.126" port: 35119 } } }
I20260810 04:41:24.761497 29364 sys_catalog.cc:458] T 00000000000000000000000000000000 P fb5984eebdce440196aefb71d446eec3 [sys.catalog]: This master's current role is: LEADER
I20260810 04:41:24.761655 29372 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 04:41:24.762184 29372 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 04:41:24.763626 29372 catalog_manager.cc:1347] Generated new cluster ID: ab2ab13b396442fd87eeb1c91b7763a2
I20260810 04:41:24.763675 29372 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 10 04:41:24 dist-test-slave-bqcl krb5kdc[29207](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786336884, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.167.126@KRBTEST.COM
I20260810 04:41:24.768662 29359 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:41:24.756124 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47200 (local address 127.22.167.126:35119)
0810 04:41:24.756299 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0810 04:41:24.756309 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 04:41:24.756339 (+ 30us) server_negotiation.cc:407] Connection header received
0810 04:41:24.756857 (+ 518us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:41:24.756865 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:41:24.756927 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:41:24.757084 (+ 157us) server_negotiation.cc:226] Negotiated authn=SASL
0810 04:41:24.757645 (+ 561us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:41:24.759039 (+ 1394us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:41:24.759905 (+ 866us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:41:24.760547 (+ 642us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:41:24.760662 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:41:24.764679 (+ 4017us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 04:41:24.764684 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 04:41:24.764684 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 04:41:24.764696 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0810 04:41:24.766638 (+ 1942us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 04:41:24.767036 (+ 398us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 04:41:24.767040 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 04:41:24.767040 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 04:41:24.767148 (+ 108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 04:41:24.767354 (+ 206us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 04:41:24.767355 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 04:41:24.767356 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 04:41:24.767665 (+ 309us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 04:41:24.767812 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0810 04:41:24.768301 (+ 489us) server_negotiation.cc:299] Negotiation successful
0810 04:41:24.768449 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":123,"thread_start_us":83,"threads_started":1}
I20260810 04:41:24.790311 29372 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 04:41:24.790812 29372 catalog_manager.cc:1504] Loading token signing keys...
I20260810 04:41:24.795751 29372 catalog_manager.cc:5868] T 00000000000000000000000000000000 P fb5984eebdce440196aefb71d446eec3: Generated new TSK 0
I20260810 04:41:24.795910 29372 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.22.167.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.167.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.167.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786336863388938-23197-0/krb5kdc/kudu_127.22.167.65.keytab.
Entry for principal kudu/127.22.167.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786336863388938-23197-0/krb5kdc/kudu_127.22.167.65.keytab.
I20260810 04:41:24.830806 23197 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskemv_AM/build/release/bin/kudu
/tmp/dist-test-taskemv_AM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786336863388938-23197-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786336863388938-23197-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786336863388938-23197-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786336863388938-23197-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.167.65:0
--local_ip_for_outbound_sockets=127.22.167.65
--webserver_interface=127.22.167.65
--webserver_port=0
--tserver_master_addrs=127.22.167.126:35119
--keytab_file=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786336863388938-23197-0/krb5kdc/kudu_127.22.167.65.keytab
--principal=kudu/127.22.167.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.167.84:39223
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786336863388938-23197-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786336863388938-23197-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786336863388938-23197-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 04:41:24.935802 29396 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 04:41:24.935992 29396 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 04:41:24.936015 29396 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 04:41:24.937106 29396 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 04:41:24.937153 29396 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 04:41:24.937179 29396 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.167.65
I20260810 04:41:24.938658 29396 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.167.84:39223
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786336863388938-23197-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786336863388938-23197-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786336863388938-23197-0/krb5kdc/kudu_127.22.167.65.keytab
--principal=kudu/127.22.167.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.167.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786336863388938-23197-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.22.167.65
--webserver_port=0
--tserver_master_addrs=127.22.167.126:35119
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29396
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.167.65
--log_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786336863388938-23197-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 03:54:01 UTC on ccycloud.quasar-icijmt.root.comops.site
build id 4577009
I20260810 04:41:24.938877 29396 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 04:41:24.939079 29396 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 04:41:24.943714 29407 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 04:41:24.943681 29403 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 04:41:24.943678 29404 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 04:41:24.943893 29396 server_base.cc:1034] running on GCE node
I20260810 04:41:24.944097 29396 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 04:41:24.944324 29396 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 04:41:24.945470 29396 hybrid_clock.cc:648] HybridClock initialized: now 1786336884945457 us; error 28 us; skew 500 ppm
Aug 10 04:41:24 dist-test-slave-bqcl krb5kdc[29207](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786336884, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.167.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 04:41:24.947872 29396 init.cc:377] Logged in from keytab as kudu/127.22.167.65@KRBTEST.COM (short username kudu)
I20260810 04:41:24.948307 29396 webserver.cc:460] Webserver started at http://127.22.167.65:38129/ using document root <none> and password file <none>
I20260810 04:41:24.948508 29396 fs_manager.cc:359] Metadata directory not provided
I20260810 04:41:24.948555 29396 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 04:41:24.948673 29396 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 04:41:24.949610 29396 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786336863388938-23197-0/minicluster-data/ts-0/data/instance:
uuid: "1be8a8b7810f4edb85a3c00a55c4b617"
format_stamp: "Formatted at 2026-08-10 04:41:24 on dist-test-slave-bqcl"
server_key: "5d141b7521e1be14e9fd1a3eedcba60c"
server_key_iv: "7fe04580c40b8933f39c0e92e69f29b1"
server_key_version: "encryptionkey@0"
I20260810 04:41:24.949935 29396 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786336863388938-23197-0/minicluster-data/ts-0/wal/instance:
uuid: "1be8a8b7810f4edb85a3c00a55c4b617"
format_stamp: "Formatted at 2026-08-10 04:41:24 on dist-test-slave-bqcl"
server_key: "5d141b7521e1be14e9fd1a3eedcba60c"
server_key_iv: "7fe04580c40b8933f39c0e92e69f29b1"
server_key_version: "encryptionkey@0"
I20260810 04:41:24.951313 29396 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260810 04:41:24.952018 29421 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 04:41:24.952188 29396 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260810 04:41:24.952270 29396 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786336863388938-23197-0/minicluster-data/ts-0/data,/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786336863388938-23197-0/minicluster-data/ts-0/wal
uuid: "1be8a8b7810f4edb85a3c00a55c4b617"
format_stamp: "Formatted at 2026-08-10 04:41:24 on dist-test-slave-bqcl"
server_key: "5d141b7521e1be14e9fd1a3eedcba60c"
server_key_iv: "7fe04580c40b8933f39c0e92e69f29b1"
server_key_version: "encryptionkey@0"
I20260810 04:41:24.952335 29396 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786336863388938-23197-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786336863388938-23197-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786336863388938-23197-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 04:41:24.958796 29396 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 04:41:24.960433 29396 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 04:41:24.960553 29396 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 04:41:24.960748 29396 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 04:41:24.961009 29396 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 04:41:24.961040 29396 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 04:41:24.961064 29396 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 04:41:24.961089 29396 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 04:41:24.966521 29396 rpc_server.cc:307] RPC server started. Bound to: 127.22.167.65:42277
I20260810 04:41:24.966566 29620 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.167.65:42277 every 8 connection(s)
I20260810 04:41:24.966835 29396 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786336863388938-23197-0/minicluster-data/ts-0/data/info.pb
Aug 10 04:41:24 dist-test-slave-bqcl krb5kdc[29207](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786336884, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.167.65@KRBTEST.COM for kudu/127.22.167.126@KRBTEST.COM
I20260810 04:41:24.975209 29359 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:41:24.967449 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.167.65:54705 (local address 127.22.167.126:35119)
0810 04:41:24.967488 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0810 04:41:24.967490 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 04:41:24.967687 (+ 197us) server_negotiation.cc:407] Connection header received
0810 04:41:24.968547 (+ 860us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:41:24.968548 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:41:24.968560 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:41:24.968612 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0810 04:41:24.969170 (+ 558us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:41:24.969708 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:41:24.970681 (+ 973us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:41:24.970891 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:41:24.970948 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:41:24.972833 (+ 1885us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 04:41:24.972834 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 04:41:24.972835 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 04:41:24.972841 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0810 04:41:24.974110 (+ 1269us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 04:41:24.974483 (+ 373us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 04:41:24.974484 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 04:41:24.974484 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 04:41:24.974534 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 04:41:24.974724 (+ 190us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 04:41:24.974724 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 04:41:24.974725 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 04:41:24.974829 (+ 104us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 04:41:24.974875 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0810 04:41:24.975067 (+ 192us) server_negotiation.cc:299] Negotiation successful
0810 04:41:24.975141 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260810 04:41:24.975364 29624 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:41:24.967428 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.126:35119 (local address 127.22.167.65:54705)
0810 04:41:24.967578 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0810 04:41:24.967611 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0810 04:41:24.968430 (+ 819us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 04:41:24.968663 (+ 233us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 04:41:24.968666 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 04:41:24.968901 (+ 235us) client_negotiation.cc:189] Negotiated authn=SASL
0810 04:41:24.969073 (+ 172us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:41:24.969077 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:41:24.969777 (+ 700us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:41:24.969778 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:41:24.970601 (+ 823us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:41:24.970603 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:41:24.970954 (+ 351us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:41:24.970954 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:41:24.971044 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:41:24.971452 (+ 408us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 04:41:24.971459 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0810 04:41:24.972747 (+ 1288us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 04:41:24.974224 (+ 1477us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 04:41:24.974229 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 04:41:24.974231 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 04:41:24.974428 (+ 197us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 04:41:24.974595 (+ 167us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 04:41:24.974596 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 04:41:24.974596 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 04:41:24.974682 (+ 86us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 04:41:24.974870 (+ 188us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 04:41:24.974872 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 04:41:24.974976 (+ 104us) client_negotiation.cc:769] Sending connection context
0810 04:41:24.975048 (+ 72us) client_negotiation.cc:240] Negotiation successful
0810 04:41:24.975172 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":92,"threads_started":1}
I20260810 04:41:24.975750 23197 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskemv_AM/build/release/bin/kudu as pid 29396
I20260810 04:41:24.975867 23197 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786336863388938-23197-0/minicluster-data/ts-0/wal/instance
I20260810 04:41:24.975883 29621 heartbeater.cc:347] Connected to a master server at 127.22.167.126:35119
I20260810 04:41:24.975973 29621 heartbeater.cc:464] Registering TS with master...
I20260810 04:41:24.976056 23197 external_mini_cluster.cc:1428] Setting key 773e315f0bcb943ec3d73014c7e18c26
I20260810 04:41:24.976157 29621 heartbeater.cc:511] Master 127.22.167.126:35119 requested a full tablet report, sending...
I20260810 04:41:24.976699 29274 ts_manager.cc:194] Registered new tserver with Master: 1be8a8b7810f4edb85a3c00a55c4b617 (127.22.167.65:42277)
I20260810 04:41:24.977296 29274 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.167.65@KRBTEST.COM'} at 127.22.167.65:54705
WARNING: no policy specified for kudu/127.22.167.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.167.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.167.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786336863388938-23197-0/krb5kdc/kudu_127.22.167.66.keytab.
Entry for principal kudu/127.22.167.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786336863388938-23197-0/krb5kdc/kudu_127.22.167.66.keytab.
I20260810 04:41:25.000545 23197 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskemv_AM/build/release/bin/kudu
/tmp/dist-test-taskemv_AM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786336863388938-23197-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786336863388938-23197-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786336863388938-23197-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786336863388938-23197-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.167.66:0
--local_ip_for_outbound_sockets=127.22.167.66
--webserver_interface=127.22.167.66
--webserver_port=0
--tserver_master_addrs=127.22.167.126:35119
--keytab_file=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786336863388938-23197-0/krb5kdc/kudu_127.22.167.66.keytab
--principal=kudu/127.22.167.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.167.84:39223
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786336863388938-23197-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786336863388938-23197-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786336863388938-23197-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 04:41:25.079592 29628 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 04:41:25.079787 29628 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 04:41:25.079808 29628 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 04:41:25.080966 29628 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 04:41:25.081015 29628 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 04:41:25.081040 29628 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.167.66
I20260810 04:41:25.082501 29628 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.167.84:39223
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786336863388938-23197-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786336863388938-23197-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786336863388938-23197-0/krb5kdc/kudu_127.22.167.66.keytab
--principal=kudu/127.22.167.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.167.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786336863388938-23197-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.22.167.66
--webserver_port=0
--tserver_master_addrs=127.22.167.126:35119
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29628
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.167.66
--log_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786336863388938-23197-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 03:54:01 UTC on ccycloud.quasar-icijmt.root.comops.site
build id 4577009
I20260810 04:41:25.082710 29628 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 04:41:25.082912 29628 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 04:41:25.087621 29642 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 04:41:25.087631 29637 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 04:41:25.087783 29628 server_base.cc:1034] running on GCE node
W20260810 04:41:25.087630 29635 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 04:41:25.088090 29628 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 04:41:25.088318 29628 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 04:41:25.089455 29628 hybrid_clock.cc:648] HybridClock initialized: now 1786336885089444 us; error 32 us; skew 500 ppm
Aug 10 04:41:25 dist-test-slave-bqcl krb5kdc[29207](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786336885, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.167.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 04:41:25.095129 29628 init.cc:377] Logged in from keytab as kudu/127.22.167.66@KRBTEST.COM (short username kudu)
I20260810 04:41:25.095525 29628 webserver.cc:460] Webserver started at http://127.22.167.66:33473/ using document root <none> and password file <none>
I20260810 04:41:25.095696 29628 fs_manager.cc:359] Metadata directory not provided
I20260810 04:41:25.095732 29628 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 04:41:25.095824 29628 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 04:41:25.096740 29628 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786336863388938-23197-0/minicluster-data/ts-1/data/instance:
uuid: "6bd915922b0e4c27926b0023d35bd9e8"
format_stamp: "Formatted at 2026-08-10 04:41:25 on dist-test-slave-bqcl"
server_key: "b2787aaadf396ca914fd470245e7a740"
server_key_iv: "3787b917538c022f92861b989083b12f"
server_key_version: "encryptionkey@0"
I20260810 04:41:25.097026 29628 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786336863388938-23197-0/minicluster-data/ts-1/wal/instance:
uuid: "6bd915922b0e4c27926b0023d35bd9e8"
format_stamp: "Formatted at 2026-08-10 04:41:25 on dist-test-slave-bqcl"
server_key: "b2787aaadf396ca914fd470245e7a740"
server_key_iv: "3787b917538c022f92861b989083b12f"
server_key_version: "encryptionkey@0"
I20260810 04:41:25.098206 29628 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260810 04:41:25.098985 29652 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 04:41:25.099170 29628 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 04:41:25.099251 29628 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786336863388938-23197-0/minicluster-data/ts-1/data,/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786336863388938-23197-0/minicluster-data/ts-1/wal
uuid: "6bd915922b0e4c27926b0023d35bd9e8"
format_stamp: "Formatted at 2026-08-10 04:41:25 on dist-test-slave-bqcl"
server_key: "b2787aaadf396ca914fd470245e7a740"
server_key_iv: "3787b917538c022f92861b989083b12f"
server_key_version: "encryptionkey@0"
I20260810 04:41:25.099301 29628 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786336863388938-23197-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786336863388938-23197-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786336863388938-23197-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 04:41:25.119871 29628 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 04:41:25.121572 29628 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 04:41:25.121680 29628 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 04:41:25.121870 29628 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 04:41:25.122197 29628 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 04:41:25.122234 29628 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 04:41:25.122255 29628 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 04:41:25.122267 29628 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 04:41:25.127684 29628 rpc_server.cc:307] RPC server started. Bound to: 127.22.167.66:46695
I20260810 04:41:25.127749 29873 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.167.66:46695 every 8 connection(s)
I20260810 04:41:25.128021 29628 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786336863388938-23197-0/minicluster-data/ts-1/data/info.pb
Aug 10 04:41:25 dist-test-slave-bqcl krb5kdc[29207](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786336885, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.167.66@KRBTEST.COM for kudu/127.22.167.126@KRBTEST.COM
I20260810 04:41:25.135130 23197 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskemv_AM/build/release/bin/kudu as pid 29628
I20260810 04:41:25.135245 23197 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786336863388938-23197-0/minicluster-data/ts-1/wal/instance
I20260810 04:41:25.135423 23197 external_mini_cluster.cc:1428] Setting key 98525080f51346833ed76d286fcd8d6a
I20260810 04:41:25.136862 29359 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:41:25.128683 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.167.66:51609 (local address 127.22.167.126:35119)
0810 04:41:25.128758 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0810 04:41:25.128760 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 04:41:25.128944 (+ 184us) server_negotiation.cc:407] Connection header received
0810 04:41:25.129824 (+ 880us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:41:25.129825 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:41:25.129836 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:41:25.129884 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0810 04:41:25.130494 (+ 610us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:41:25.130983 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:41:25.131911 (+ 928us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:41:25.132099 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:41:25.132145 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:41:25.134029 (+ 1884us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 04:41:25.134030 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 04:41:25.134031 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 04:41:25.134038 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 04:41:25.135521 (+ 1483us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 04:41:25.135968 (+ 447us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 04:41:25.135969 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 04:41:25.135969 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 04:41:25.136021 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 04:41:25.136259 (+ 238us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 04:41:25.136259 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 04:41:25.136260 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 04:41:25.136388 (+ 128us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 04:41:25.136455 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0810 04:41:25.136702 (+ 247us) server_negotiation.cc:299] Negotiation successful
0810 04:41:25.136775 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260810 04:41:25.136945 29878 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:41:25.128664 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.126:35119 (local address 127.22.167.66:51609)
0810 04:41:25.128829 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0810 04:41:25.128866 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0810 04:41:25.129740 (+ 874us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 04:41:25.129941 (+ 201us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 04:41:25.129944 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 04:41:25.130236 (+ 292us) client_negotiation.cc:189] Negotiated authn=SASL
0810 04:41:25.130426 (+ 190us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:41:25.130431 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:41:25.131044 (+ 613us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:41:25.131044 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:41:25.131837 (+ 793us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:41:25.131840 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:41:25.132154 (+ 314us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:41:25.132154 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:41:25.132252 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:41:25.132646 (+ 394us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 04:41:25.132654 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0810 04:41:25.133945 (+ 1291us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 04:41:25.135611 (+ 1666us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 04:41:25.135619 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 04:41:25.135621 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 04:41:25.135906 (+ 285us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 04:41:25.136073 (+ 167us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 04:41:25.136074 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 04:41:25.136074 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 04:41:25.136202 (+ 128us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 04:41:25.136449 (+ 247us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 04:41:25.136453 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 04:41:25.136583 (+ 130us) client_negotiation.cc:769] Sending connection context
0810 04:41:25.136666 (+ 83us) client_negotiation.cc:240] Negotiation successful
0810 04:41:25.136781 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":97,"threads_started":1}
I20260810 04:41:25.137355 29875 heartbeater.cc:347] Connected to a master server at 127.22.167.126:35119
I20260810 04:41:25.137440 29875 heartbeater.cc:464] Registering TS with master...
I20260810 04:41:25.137616 29875 heartbeater.cc:511] Master 127.22.167.126:35119 requested a full tablet report, sending...
I20260810 04:41:25.137980 29274 ts_manager.cc:194] Registered new tserver with Master: 6bd915922b0e4c27926b0023d35bd9e8 (127.22.167.66:46695)
I20260810 04:41:25.138378 29274 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.167.66@KRBTEST.COM'} at 127.22.167.66:51609
WARNING: no policy specified for kudu/127.22.167.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.167.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.167.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786336863388938-23197-0/krb5kdc/kudu_127.22.167.67.keytab.
Entry for principal kudu/127.22.167.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786336863388938-23197-0/krb5kdc/kudu_127.22.167.67.keytab.
I20260810 04:41:25.160739 23197 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskemv_AM/build/release/bin/kudu
/tmp/dist-test-taskemv_AM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786336863388938-23197-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786336863388938-23197-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786336863388938-23197-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786336863388938-23197-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.167.67:0
--local_ip_for_outbound_sockets=127.22.167.67
--webserver_interface=127.22.167.67
--webserver_port=0
--tserver_master_addrs=127.22.167.126:35119
--keytab_file=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786336863388938-23197-0/krb5kdc/kudu_127.22.167.67.keytab
--principal=kudu/127.22.167.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.167.84:39223
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786336863388938-23197-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786336863388938-23197-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786336863388938-23197-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 04:41:25.242403 29883 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 04:41:25.242623 29883 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 04:41:25.242656 29883 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 04:41:25.243801 29883 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 04:41:25.243850 29883 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 04:41:25.243887 29883 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.167.67
I20260810 04:41:25.245384 29883 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.167.84:39223
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786336863388938-23197-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786336863388938-23197-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786336863388938-23197-0/krb5kdc/kudu_127.22.167.67.keytab
--principal=kudu/127.22.167.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.167.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786336863388938-23197-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.22.167.67
--webserver_port=0
--tserver_master_addrs=127.22.167.126:35119
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29883
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.167.67
--log_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786336863388938-23197-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 03:54:01 UTC on ccycloud.quasar-icijmt.root.comops.site
build id 4577009
I20260810 04:41:25.245646 29883 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 04:41:25.245863 29883 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 04:41:25.250643 29895 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 04:41:25.250669 29898 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 04:41:25.250775 29883 server_base.cc:1034] running on GCE node
W20260810 04:41:25.250676 29889 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 04:41:25.251087 29883 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 04:41:25.251320 29883 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 04:41:25.252485 29883 hybrid_clock.cc:648] HybridClock initialized: now 1786336885252464 us; error 38 us; skew 500 ppm
Aug 10 04:41:25 dist-test-slave-bqcl krb5kdc[29207](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786336885, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.167.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 04:41:25.254937 29883 init.cc:377] Logged in from keytab as kudu/127.22.167.67@KRBTEST.COM (short username kudu)
I20260810 04:41:25.255405 29883 webserver.cc:460] Webserver started at http://127.22.167.67:42635/ using document root <none> and password file <none>
I20260810 04:41:25.255606 29883 fs_manager.cc:359] Metadata directory not provided
I20260810 04:41:25.255649 29883 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 04:41:25.255760 29883 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 04:41:25.256788 29883 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786336863388938-23197-0/minicluster-data/ts-2/data/instance:
uuid: "54e63d1c503b40b0867c9750c2cfd396"
format_stamp: "Formatted at 2026-08-10 04:41:25 on dist-test-slave-bqcl"
server_key: "9283a9f13f030f6447179ad818e1855c"
server_key_iv: "b6293ac8d9f08767b44c8eb47c094c25"
server_key_version: "encryptionkey@0"
I20260810 04:41:25.257107 29883 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786336863388938-23197-0/minicluster-data/ts-2/wal/instance:
uuid: "54e63d1c503b40b0867c9750c2cfd396"
format_stamp: "Formatted at 2026-08-10 04:41:25 on dist-test-slave-bqcl"
server_key: "9283a9f13f030f6447179ad818e1855c"
server_key_iv: "b6293ac8d9f08767b44c8eb47c094c25"
server_key_version: "encryptionkey@0"
I20260810 04:41:25.258306 29883 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260810 04:41:25.258967 29908 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 04:41:25.259105 29883 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260810 04:41:25.259187 29883 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786336863388938-23197-0/minicluster-data/ts-2/data,/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786336863388938-23197-0/minicluster-data/ts-2/wal
uuid: "54e63d1c503b40b0867c9750c2cfd396"
format_stamp: "Formatted at 2026-08-10 04:41:25 on dist-test-slave-bqcl"
server_key: "9283a9f13f030f6447179ad818e1855c"
server_key_iv: "b6293ac8d9f08767b44c8eb47c094c25"
server_key_version: "encryptionkey@0"
I20260810 04:41:25.259246 29883 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786336863388938-23197-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786336863388938-23197-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786336863388938-23197-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 04:41:25.283689 29883 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 04:41:25.285740 29883 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 04:41:25.285871 29883 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 04:41:25.286083 29883 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 04:41:25.286404 29883 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 04:41:25.286444 29883 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 04:41:25.286474 29883 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 04:41:25.286497 29883 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 04:41:25.291910 29883 rpc_server.cc:307] RPC server started. Bound to: 127.22.167.67:37103
I20260810 04:41:25.291971 30105 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.167.67:37103 every 8 connection(s)
I20260810 04:41:25.292260 29883 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786336863388938-23197-0/minicluster-data/ts-2/data/info.pb
I20260810 04:41:25.295414 23197 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskemv_AM/build/release/bin/kudu as pid 29883
I20260810 04:41:25.295509 23197 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786336863388938-23197-0/minicluster-data/ts-2/wal/instance
I20260810 04:41:25.295688 23197 external_mini_cluster.cc:1428] Setting key b8a983db1529254e6d3db0f232cbaf76
Aug 10 04:41:25 dist-test-slave-bqcl krb5kdc[29207](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786336885, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.167.67@KRBTEST.COM for kudu/127.22.167.126@KRBTEST.COM
I20260810 04:41:25.301717 29359 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:41:25.292941 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.167.67:33661 (local address 127.22.167.126:35119)
0810 04:41:25.293005 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0810 04:41:25.293007 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 04:41:25.293236 (+ 229us) server_negotiation.cc:407] Connection header received
0810 04:41:25.294170 (+ 934us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:41:25.294172 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:41:25.294188 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:41:25.294253 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0810 04:41:25.294862 (+ 609us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:41:25.295379 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:41:25.296725 (+ 1346us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:41:25.296916 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:41:25.296967 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:41:25.298998 (+ 2031us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 04:41:25.298999 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 04:41:25.298999 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 04:41:25.299006 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 04:41:25.300394 (+ 1388us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 04:41:25.300797 (+ 403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 04:41:25.300798 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 04:41:25.300799 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 04:41:25.300860 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 04:41:25.301096 (+ 236us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 04:41:25.301097 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 04:41:25.301097 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 04:41:25.301211 (+ 114us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 04:41:25.301275 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0810 04:41:25.301539 (+ 264us) server_negotiation.cc:299] Negotiation successful
0810 04:41:25.301624 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260810 04:41:25.301911 30111 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:41:25.292902 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.126:35119 (local address 127.22.167.67:33661)
0810 04:41:25.293123 (+ 221us) negotiation.cc:107] Waiting for socket to connect
0810 04:41:25.293159 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0810 04:41:25.294016 (+ 857us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 04:41:25.294317 (+ 301us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 04:41:25.294320 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 04:41:25.294570 (+ 250us) client_negotiation.cc:189] Negotiated authn=SASL
0810 04:41:25.294760 (+ 190us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:41:25.294764 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:41:25.295463 (+ 699us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:41:25.295464 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:41:25.296663 (+ 1199us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:41:25.296666 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:41:25.296974 (+ 308us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:41:25.296975 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:41:25.297071 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:41:25.297437 (+ 366us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 04:41:25.297445 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0810 04:41:25.298886 (+ 1441us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 04:41:25.300477 (+ 1591us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 04:41:25.300492 (+ 15us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 04:41:25.300494 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 04:41:25.300724 (+ 230us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 04:41:25.300919 (+ 195us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 04:41:25.300920 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 04:41:25.300921 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 04:41:25.301036 (+ 115us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 04:41:25.301272 (+ 236us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 04:41:25.301278 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 04:41:25.301413 (+ 135us) client_negotiation.cc:769] Sending connection context
0810 04:41:25.301515 (+ 102us) client_negotiation.cc:240] Negotiation successful
0810 04:41:25.301679 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":182,"thread_start_us":152,"threads_started":1}
I20260810 04:41:25.302380 30106 heartbeater.cc:347] Connected to a master server at 127.22.167.126:35119
I20260810 04:41:25.302469 30106 heartbeater.cc:464] Registering TS with master...
I20260810 04:41:25.302624 30106 heartbeater.cc:511] Master 127.22.167.126:35119 requested a full tablet report, sending...
I20260810 04:41:25.302943 29274 ts_manager.cc:194] Registered new tserver with Master: 54e63d1c503b40b0867c9750c2cfd396 (127.22.167.67:37103)
I20260810 04:41:25.303311 29274 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.167.67@KRBTEST.COM'} at 127.22.167.67:33661
I20260810 04:41:25.307235 23197 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 10 04:41:25 dist-test-slave-bqcl krb5kdc[29207](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786336885, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 10 04:41:25 dist-test-slave-bqcl krb5kdc[29207](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786336885, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.167.65@KRBTEST.COM
I20260810 04:41:25.329895 30121 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:41:25.322782 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42306 (local address 127.22.167.65:42277)
0810 04:41:25.322952 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0810 04:41:25.322958 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 04:41:25.322968 (+ 10us) server_negotiation.cc:407] Connection header received
0810 04:41:25.322993 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:41:25.322995 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:41:25.323010 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:41:25.323090 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0810 04:41:25.323616 (+ 526us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:41:25.324352 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:41:25.325021 (+ 669us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:41:25.325326 (+ 305us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:41:25.325374 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:41:25.327414 (+ 2040us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 04:41:25.327422 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 04:41:25.327422 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 04:41:25.327428 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0810 04:41:25.328905 (+ 1477us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 04:41:25.329221 (+ 316us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 04:41:25.329224 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 04:41:25.329224 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 04:41:25.329296 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 04:41:25.329439 (+ 143us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 04:41:25.329440 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 04:41:25.329440 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 04:41:25.329572 (+ 132us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 04:41:25.329672 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0810 04:41:25.329755 (+ 83us) server_negotiation.cc:299] Negotiation successful
0810 04:41:25.329816 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":132,"thread_start_us":83,"threads_started":1}
Aug 10 04:41:25 dist-test-slave-bqcl krb5kdc[29207](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786336885, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.167.126@KRBTEST.COM
I20260810 04:41:25.337095 29359 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:41:25.331192 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47210 (local address 127.22.167.126:35119)
0810 04:41:25.331256 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0810 04:41:25.331258 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 04:41:25.331266 (+ 8us) server_negotiation.cc:407] Connection header received
0810 04:41:25.331290 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:41:25.331290 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:41:25.331300 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:41:25.331347 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0810 04:41:25.331753 (+ 406us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:41:25.332230 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:41:25.332845 (+ 615us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:41:25.333041 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:41:25.333094 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:41:25.335024 (+ 1930us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 04:41:25.335025 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 04:41:25.335025 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 04:41:25.335033 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0810 04:41:25.336118 (+ 1085us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 04:41:25.336393 (+ 275us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 04:41:25.336394 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 04:41:25.336394 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 04:41:25.336444 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 04:41:25.336610 (+ 166us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 04:41:25.336611 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 04:41:25.336611 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 04:41:25.336752 (+ 141us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 04:41:25.336840 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0810 04:41:25.336935 (+ 95us) server_negotiation.cc:299] Negotiation successful
0810 04:41:25.337008 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
W20260810 04:41:25.337319 29274 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47210
I20260810 04:41:25.337613 23197 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskemv_AM/build/release/bin/kudu with pid 29396
I20260810 04:41:25.348875 23197 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskemv_AM/build/release/bin/kudu with pid 29628
I20260810 04:41:25.373023 23197 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskemv_AM/build/release/bin/kudu with pid 29883
I20260810 04:41:25.382931 23197 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskemv_AM/build/release/bin/kudu with pid 29226
2026-08-10T04:41:25Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizedSuperuser (2048 ms)
[ RUN ] SecurityITest.TestDisableWebUI
Loading random data
Initializing database '/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786336863388938-23197-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 04:41:25 dist-test-slave-bqcl krb5kdc[30136](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 04:41:25 dist-test-slave-bqcl krb5kdc[30136](info): set up 2 sockets
Aug 10 04:41:25 dist-test-slave-bqcl krb5kdc[30136](info): commencing operation
krb5kdc: starting...
W20260810 04:41:26.562639 23197 mini_kdc.cc:121] Time spent starting KDC: real 1.154s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786336863388938-23197-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786336863388938-23197-0/krb5kdc/test-user.keytab.
Aug 10 04:41:26 dist-test-slave-bqcl krb5kdc[30136](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786336886, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T04:41:26Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T04:41:26Z Disabled control of system clock
WARNING: no policy specified for kudu/127.22.167.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.167.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.167.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786336863388938-23197-0/krb5kdc/kudu_127.22.167.126.keytab.
Entry for principal kudu/127.22.167.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786336863388938-23197-0/krb5kdc/kudu_127.22.167.126.keytab.
I20260810 04:41:26.674808 23197 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskemv_AM/build/release/bin/kudu
/tmp/dist-test-taskemv_AM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786336863388938-23197-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786336863388938-23197-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786336863388938-23197-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786336863388938-23197-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.167.126:45899
--webserver_interface=127.22.167.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786336863388938-23197-0/krb5kdc/kudu_127.22.167.126.keytab
--principal=kudu/127.22.167.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.167.84:46193
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.167.126:45899
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786336863388938-23197-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786336863388938-23197-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786336863388938-23197-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 04:41:26.755609 30161 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 04:41:26.755801 30161 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 04:41:26.755822 30161 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 04:41:26.756879 30161 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 04:41:26.756910 30161 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 04:41:26.756923 30161 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 04:41:26.756934 30161 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 04:41:26.756944 30161 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 04:41:26.758351 30161 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.167.84:46193
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786336863388938-23197-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786336863388938-23197-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.22.167.126:45899
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786336863388938-23197-0/krb5kdc/kudu_127.22.167.126.keytab
--principal=kudu/127.22.167.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.167.126:45899
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786336863388938-23197-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.167.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30161
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786336863388938-23197-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 03:54:01 UTC on ccycloud.quasar-icijmt.root.comops.site
build id 4577009
I20260810 04:41:26.758561 30161 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 04:41:26.758769 30161 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 04:41:26.763295 30178 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 04:41:26.763312 30174 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 04:41:26.763317 30176 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 04:41:26.763549 30161 server_base.cc:1034] running on GCE node
I20260810 04:41:26.763798 30161 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 04:41:26.764055 30161 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 04:41:26.765220 30161 hybrid_clock.cc:648] HybridClock initialized: now 1786336886765205 us; error 31 us; skew 500 ppm
Aug 10 04:41:26 dist-test-slave-bqcl krb5kdc[30136](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786336886, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.167.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 04:41:26.767619 30161 init.cc:377] Logged in from keytab as kudu/127.22.167.126@KRBTEST.COM (short username kudu)
I20260810 04:41:26.767923 30161 fs_manager.cc:359] Metadata directory not provided
I20260810 04:41:26.767971 30161 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 04:41:26.768075 30161 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 04:41:26.769052 30161 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786336863388938-23197-0/minicluster-data/master-0/data/instance:
uuid: "001d663cc0da4b09af7cb8f1e4f0eda5"
format_stamp: "Formatted at 2026-08-10 04:41:26 on dist-test-slave-bqcl"
server_key: "338018f81e1c0d61776dc9d3abd900eb"
server_key_iv: "88e36aadcfd164a6a74547ef0c15fb65"
server_key_version: "encryptionkey@0"
I20260810 04:41:26.769335 30161 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786336863388938-23197-0/minicluster-data/master-0/wal/instance:
uuid: "001d663cc0da4b09af7cb8f1e4f0eda5"
format_stamp: "Formatted at 2026-08-10 04:41:26 on dist-test-slave-bqcl"
server_key: "338018f81e1c0d61776dc9d3abd900eb"
server_key_iv: "88e36aadcfd164a6a74547ef0c15fb65"
server_key_version: "encryptionkey@0"
I20260810 04:41:26.770494 30161 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260810 04:41:26.771109 30186 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 04:41:26.771261 30161 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260810 04:41:26.771339 30161 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786336863388938-23197-0/minicluster-data/master-0/data,/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786336863388938-23197-0/minicluster-data/master-0/wal
uuid: "001d663cc0da4b09af7cb8f1e4f0eda5"
format_stamp: "Formatted at 2026-08-10 04:41:26 on dist-test-slave-bqcl"
server_key: "338018f81e1c0d61776dc9d3abd900eb"
server_key_iv: "88e36aadcfd164a6a74547ef0c15fb65"
server_key_version: "encryptionkey@0"
I20260810 04:41:26.771391 30161 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786336863388938-23197-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786336863388938-23197-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786336863388938-23197-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 04:41:26.787672 30161 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 04:41:26.789702 30161 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 04:41:26.789808 30161 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 04:41:26.793543 30161 rpc_server.cc:307] RPC server started. Bound to: 127.22.167.126:45899
I20260810 04:41:26.793582 30299 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.167.126:45899 every 8 connection(s)
I20260810 04:41:26.793819 30161 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786336863388938-23197-0/minicluster-data/master-0/data/info.pb
I20260810 04:41:26.794840 30300 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 04:41:26.796998 30300 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 001d663cc0da4b09af7cb8f1e4f0eda5: Bootstrap starting.
I20260810 04:41:26.797525 30300 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 001d663cc0da4b09af7cb8f1e4f0eda5: Neither blocks nor log segments found. Creating new log.
I20260810 04:41:26.797775 30300 log.cc:824] T 00000000000000000000000000000000 P 001d663cc0da4b09af7cb8f1e4f0eda5: Log is configured to *not* fsync() on all Append() calls
I20260810 04:41:26.798395 30300 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 001d663cc0da4b09af7cb8f1e4f0eda5: No bootstrap required, opened a new log
I20260810 04:41:26.799021 23197 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskemv_AM/build/release/bin/kudu as pid 30161
I20260810 04:41:26.799135 23197 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786336863388938-23197-0/minicluster-data/master-0/wal/instance
I20260810 04:41:26.799285 23197 external_mini_cluster.cc:1428] Setting key 19aa32d23436274b5d47e3f981f32ac1
I20260810 04:41:26.800292 30300 raft_consensus.cc:348] T 00000000000000000000000000000000 P 001d663cc0da4b09af7cb8f1e4f0eda5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "001d663cc0da4b09af7cb8f1e4f0eda5" member_type: VOTER last_known_addr { host: "127.22.167.126" port: 45899 } }
I20260810 04:41:26.800455 30300 raft_consensus.cc:374] T 00000000000000000000000000000000 P 001d663cc0da4b09af7cb8f1e4f0eda5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 04:41:26.800532 30300 raft_consensus.cc:729] T 00000000000000000000000000000000 P 001d663cc0da4b09af7cb8f1e4f0eda5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 001d663cc0da4b09af7cb8f1e4f0eda5, State: Initialized, Role: FOLLOWER
I20260810 04:41:26.800684 30300 consensus_queue.cc:260] T 00000000000000000000000000000000 P 001d663cc0da4b09af7cb8f1e4f0eda5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "001d663cc0da4b09af7cb8f1e4f0eda5" member_type: VOTER last_known_addr { host: "127.22.167.126" port: 45899 } }
I20260810 04:41:26.800769 30300 raft_consensus.cc:388] T 00000000000000000000000000000000 P 001d663cc0da4b09af7cb8f1e4f0eda5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 04:41:26.800828 30300 raft_consensus.cc:482] T 00000000000000000000000000000000 P 001d663cc0da4b09af7cb8f1e4f0eda5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 04:41:26.800993 30300 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 001d663cc0da4b09af7cb8f1e4f0eda5 [term 0 FOLLOWER]: Advancing to term 1
I20260810 04:41:26.801695 30300 raft_consensus.cc:504] T 00000000000000000000000000000000 P 001d663cc0da4b09af7cb8f1e4f0eda5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "001d663cc0da4b09af7cb8f1e4f0eda5" member_type: VOTER last_known_addr { host: "127.22.167.126" port: 45899 } }
I20260810 04:41:26.801810 30300 leader_election.cc:302] T 00000000000000000000000000000000 P 001d663cc0da4b09af7cb8f1e4f0eda5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 001d663cc0da4b09af7cb8f1e4f0eda5; no voters:
I20260810 04:41:26.801977 30300 leader_election.cc:288] T 00000000000000000000000000000000 P 001d663cc0da4b09af7cb8f1e4f0eda5 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 04:41:26.802026 30308 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 001d663cc0da4b09af7cb8f1e4f0eda5 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 04:41:26.802210 30308 raft_consensus.cc:686] T 00000000000000000000000000000000 P 001d663cc0da4b09af7cb8f1e4f0eda5 [term 1 LEADER]: Becoming Leader. State: Replica: 001d663cc0da4b09af7cb8f1e4f0eda5, State: Running, Role: LEADER
I20260810 04:41:26.802286 30300 sys_catalog.cc:564] T 00000000000000000000000000000000 P 001d663cc0da4b09af7cb8f1e4f0eda5 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 04:41:26.802353 30308 consensus_queue.cc:237] T 00000000000000000000000000000000 P 001d663cc0da4b09af7cb8f1e4f0eda5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "001d663cc0da4b09af7cb8f1e4f0eda5" member_type: VOTER last_known_addr { host: "127.22.167.126" port: 45899 } }
I20260810 04:41:26.802691 30308 sys_catalog.cc:455] T 00000000000000000000000000000000 P 001d663cc0da4b09af7cb8f1e4f0eda5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 001d663cc0da4b09af7cb8f1e4f0eda5. Latest consensus state: current_term: 1 leader_uuid: "001d663cc0da4b09af7cb8f1e4f0eda5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "001d663cc0da4b09af7cb8f1e4f0eda5" member_type: VOTER last_known_addr { host: "127.22.167.126" port: 45899 } } }
I20260810 04:41:26.802764 30308 sys_catalog.cc:458] T 00000000000000000000000000000000 P 001d663cc0da4b09af7cb8f1e4f0eda5 [sys.catalog]: This master's current role is: LEADER
I20260810 04:41:26.803267 30313 sys_catalog.cc:455] T 00000000000000000000000000000000 P 001d663cc0da4b09af7cb8f1e4f0eda5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "001d663cc0da4b09af7cb8f1e4f0eda5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "001d663cc0da4b09af7cb8f1e4f0eda5" member_type: VOTER last_known_addr { host: "127.22.167.126" port: 45899 } } }
I20260810 04:41:26.803342 30313 sys_catalog.cc:458] T 00000000000000000000000000000000 P 001d663cc0da4b09af7cb8f1e4f0eda5 [sys.catalog]: This master's current role is: LEADER
I20260810 04:41:26.804284 30317 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 04:41:26.804842 30317 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 04:41:26.806427 30317 catalog_manager.cc:1347] Generated new cluster ID: 4b82ac5172184a818609f290460aa2d4
I20260810 04:41:26.806471 30317 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 10 04:41:26 dist-test-slave-bqcl krb5kdc[30136](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786336886, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.167.126@KRBTEST.COM
I20260810 04:41:26.810492 30307 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:41:26.799731 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38342 (local address 127.22.167.126:45899)
0810 04:41:26.800005 (+ 274us) server_negotiation.cc:206] Beginning negotiation
0810 04:41:26.800016 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0810 04:41:26.800049 (+ 33us) server_negotiation.cc:407] Connection header received
0810 04:41:26.800264 (+ 215us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:41:26.800272 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:41:26.800332 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:41:26.800470 (+ 138us) server_negotiation.cc:226] Negotiated authn=SASL
0810 04:41:26.801288 (+ 818us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:41:26.802782 (+ 1494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:41:26.803866 (+ 1084us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:41:26.804521 (+ 655us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:41:26.804625 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:41:26.807206 (+ 2581us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 04:41:26.807210 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 04:41:26.807210 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 04:41:26.807220 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0810 04:41:26.808974 (+ 1754us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 04:41:26.809363 (+ 389us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 04:41:26.809366 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 04:41:26.809366 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 04:41:26.809489 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 04:41:26.809677 (+ 188us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 04:41:26.809677 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 04:41:26.809677 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 04:41:26.809939 (+ 262us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 04:41:26.810076 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0810 04:41:26.810189 (+ 113us) server_negotiation.cc:299] Negotiation successful
0810 04:41:26.810313 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":218,"thread_start_us":79,"threads_started":1}
I20260810 04:41:26.818426 30317 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 04:41:26.818895 30317 catalog_manager.cc:1504] Loading token signing keys...
I20260810 04:41:26.823681 30317 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 001d663cc0da4b09af7cb8f1e4f0eda5: Generated new TSK 0
I20260810 04:41:26.823833 30317 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.22.167.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.167.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.167.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786336863388938-23197-0/krb5kdc/kudu_127.22.167.65.keytab.
Entry for principal kudu/127.22.167.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786336863388938-23197-0/krb5kdc/kudu_127.22.167.65.keytab.
I20260810 04:41:26.851086 23197 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskemv_AM/build/release/bin/kudu
/tmp/dist-test-taskemv_AM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786336863388938-23197-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786336863388938-23197-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786336863388938-23197-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786336863388938-23197-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.167.65:0
--local_ip_for_outbound_sockets=127.22.167.65
--webserver_interface=127.22.167.65
--webserver_port=0
--tserver_master_addrs=127.22.167.126:45899
--keytab_file=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786336863388938-23197-0/krb5kdc/kudu_127.22.167.65.keytab
--principal=kudu/127.22.167.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.167.84:46193
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786336863388938-23197-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786336863388938-23197-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786336863388938-23197-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 04:41:26.931255 30339 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 04:41:26.931457 30339 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 04:41:26.931479 30339 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 04:41:26.932574 30339 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 04:41:26.932611 30339 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 04:41:26.932642 30339 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.167.65
I20260810 04:41:26.934058 30339 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.167.84:46193
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786336863388938-23197-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786336863388938-23197-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786336863388938-23197-0/krb5kdc/kudu_127.22.167.65.keytab
--principal=kudu/127.22.167.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.167.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786336863388938-23197-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.22.167.65
--webserver_port=0
--tserver_master_addrs=127.22.167.126:45899
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30339
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.167.65
--log_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786336863388938-23197-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 03:54:01 UTC on ccycloud.quasar-icijmt.root.comops.site
build id 4577009
I20260810 04:41:26.934264 30339 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 04:41:26.934443 30339 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 04:41:26.939080 30347 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 04:41:26.939087 30345 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 04:41:26.939131 30339 server_base.cc:1034] running on GCE node
W20260810 04:41:26.939087 30344 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 04:41:26.939460 30339 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 04:41:26.939700 30339 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 04:41:26.940834 30339 hybrid_clock.cc:648] HybridClock initialized: now 1786336886940822 us; error 28 us; skew 500 ppm
Aug 10 04:41:26 dist-test-slave-bqcl krb5kdc[30136](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786336886, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.167.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 04:41:26.943202 30339 init.cc:377] Logged in from keytab as kudu/127.22.167.65@KRBTEST.COM (short username kudu)
I20260810 04:41:26.943517 30339 fs_manager.cc:359] Metadata directory not provided
I20260810 04:41:26.943572 30339 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 04:41:26.943706 30339 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 04:41:26.944669 30339 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786336863388938-23197-0/minicluster-data/ts-0/data/instance:
uuid: "331778d4b11b4b7f8609aefb32a1b264"
format_stamp: "Formatted at 2026-08-10 04:41:26 on dist-test-slave-bqcl"
server_key: "05c809b36e351a3bccda74ed64dbe888"
server_key_iv: "896860ada270940d4f83d5399bbc4f6f"
server_key_version: "encryptionkey@0"
I20260810 04:41:26.944999 30339 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786336863388938-23197-0/minicluster-data/ts-0/wal/instance:
uuid: "331778d4b11b4b7f8609aefb32a1b264"
format_stamp: "Formatted at 2026-08-10 04:41:26 on dist-test-slave-bqcl"
server_key: "05c809b36e351a3bccda74ed64dbe888"
server_key_iv: "896860ada270940d4f83d5399bbc4f6f"
server_key_version: "encryptionkey@0"
I20260810 04:41:26.946216 30339 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260810 04:41:26.946895 30363 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 04:41:26.947046 30339 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260810 04:41:26.947127 30339 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786336863388938-23197-0/minicluster-data/ts-0/data,/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786336863388938-23197-0/minicluster-data/ts-0/wal
uuid: "331778d4b11b4b7f8609aefb32a1b264"
format_stamp: "Formatted at 2026-08-10 04:41:26 on dist-test-slave-bqcl"
server_key: "05c809b36e351a3bccda74ed64dbe888"
server_key_iv: "896860ada270940d4f83d5399bbc4f6f"
server_key_version: "encryptionkey@0"
I20260810 04:41:26.947193 30339 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786336863388938-23197-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786336863388938-23197-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786336863388938-23197-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 04:41:26.971800 30339 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 04:41:26.973649 30339 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 04:41:26.973783 30339 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 04:41:26.974007 30339 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 04:41:26.974331 30339 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 04:41:26.974366 30339 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 04:41:26.974395 30339 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 04:41:26.974419 30339 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 04:41:26.979962 30339 rpc_server.cc:307] RPC server started. Bound to: 127.22.167.65:43541
I20260810 04:41:26.980013 30574 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.167.65:43541 every 8 connection(s)
I20260810 04:41:26.980255 30339 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786336863388938-23197-0/minicluster-data/ts-0/data/info.pb
I20260810 04:41:26.985502 23197 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskemv_AM/build/release/bin/kudu as pid 30339
I20260810 04:41:26.985633 23197 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786336863388938-23197-0/minicluster-data/ts-0/wal/instance
I20260810 04:41:26.985829 23197 external_mini_cluster.cc:1428] Setting key 2fe22399441f3011e6f05ec74ef1c2a2
Aug 10 04:41:26 dist-test-slave-bqcl krb5kdc[30136](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786336886, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.167.65@KRBTEST.COM for kudu/127.22.167.126@KRBTEST.COM
I20260810 04:41:26.989369 30307 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:41:26.980933 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.167.65:33341 (local address 127.22.167.126:45899)
0810 04:41:26.980988 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0810 04:41:26.980990 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 04:41:26.981197 (+ 207us) server_negotiation.cc:407] Connection header received
0810 04:41:26.981687 (+ 490us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:41:26.981688 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:41:26.981704 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:41:26.981756 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0810 04:41:26.982704 (+ 948us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:41:26.983206 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:41:26.984239 (+ 1033us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:41:26.984448 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:41:26.984504 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:41:26.986808 (+ 2304us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 04:41:26.986809 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 04:41:26.986809 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 04:41:26.986817 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0810 04:41:26.988022 (+ 1205us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 04:41:26.988490 (+ 468us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 04:41:26.988491 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 04:41:26.988491 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 04:41:26.988556 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 04:41:26.988769 (+ 213us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 04:41:26.988769 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 04:41:26.988770 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 04:41:26.988896 (+ 126us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 04:41:26.988964 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0810 04:41:26.989189 (+ 225us) server_negotiation.cc:299] Negotiation successful
0810 04:41:26.989279 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260810 04:41:26.989441 30577 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:41:26.980905 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.126:45899 (local address 127.22.167.65:33341)
0810 04:41:26.981078 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0810 04:41:26.981113 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0810 04:41:26.981584 (+ 471us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 04:41:26.981835 (+ 251us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 04:41:26.981839 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 04:41:26.982091 (+ 252us) client_negotiation.cc:189] Negotiated authn=SASL
0810 04:41:26.982611 (+ 520us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:41:26.982617 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:41:26.983287 (+ 670us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:41:26.983288 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:41:26.984161 (+ 873us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:41:26.984164 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:41:26.984518 (+ 354us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:41:26.984519 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:41:26.984632 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:41:26.985053 (+ 421us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 04:41:26.985060 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0810 04:41:26.986716 (+ 1656us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 04:41:26.988106 (+ 1390us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 04:41:26.988116 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 04:41:26.988118 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 04:41:26.988420 (+ 302us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 04:41:26.988616 (+ 196us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 04:41:26.988616 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 04:41:26.988617 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 04:41:26.988717 (+ 100us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 04:41:26.988952 (+ 235us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 04:41:26.988955 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 04:41:26.989077 (+ 122us) client_negotiation.cc:769] Sending connection context
0810 04:41:26.989157 (+ 80us) client_negotiation.cc:240] Negotiation successful
0810 04:41:26.989277 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":92,"threads_started":1}
I20260810 04:41:26.989903 30575 heartbeater.cc:347] Connected to a master server at 127.22.167.126:45899
I20260810 04:41:26.990017 30575 heartbeater.cc:464] Registering TS with master...
I20260810 04:41:26.990198 30575 heartbeater.cc:511] Master 127.22.167.126:45899 requested a full tablet report, sending...
I20260810 04:41:26.990801 30214 ts_manager.cc:194] Registered new tserver with Master: 331778d4b11b4b7f8609aefb32a1b264 (127.22.167.65:43541)
I20260810 04:41:26.991676 30214 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.167.65@KRBTEST.COM'} at 127.22.167.65:33341
WARNING: no policy specified for kudu/127.22.167.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.167.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.167.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786336863388938-23197-0/krb5kdc/kudu_127.22.167.66.keytab.
Entry for principal kudu/127.22.167.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786336863388938-23197-0/krb5kdc/kudu_127.22.167.66.keytab.
I20260810 04:41:27.012401 23197 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskemv_AM/build/release/bin/kudu
/tmp/dist-test-taskemv_AM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786336863388938-23197-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786336863388938-23197-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786336863388938-23197-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786336863388938-23197-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.167.66:0
--local_ip_for_outbound_sockets=127.22.167.66
--webserver_interface=127.22.167.66
--webserver_port=0
--tserver_master_addrs=127.22.167.126:45899
--keytab_file=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786336863388938-23197-0/krb5kdc/kudu_127.22.167.66.keytab
--principal=kudu/127.22.167.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.167.84:46193
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786336863388938-23197-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786336863388938-23197-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786336863388938-23197-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 04:41:27.092145 30582 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 04:41:27.092339 30582 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 04:41:27.092361 30582 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 04:41:27.093463 30582 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 04:41:27.093509 30582 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 04:41:27.093541 30582 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.167.66
I20260810 04:41:27.095019 30582 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.167.84:46193
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786336863388938-23197-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786336863388938-23197-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786336863388938-23197-0/krb5kdc/kudu_127.22.167.66.keytab
--principal=kudu/127.22.167.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.167.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786336863388938-23197-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.22.167.66
--webserver_port=0
--tserver_master_addrs=127.22.167.126:45899
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30582
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.167.66
--log_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786336863388938-23197-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 03:54:01 UTC on ccycloud.quasar-icijmt.root.comops.site
build id 4577009
I20260810 04:41:27.095235 30582 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 04:41:27.095441 30582 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 04:41:27.100240 30591 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 04:41:27.100234 30594 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 04:41:27.100231 30592 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 04:41:27.100446 30582 server_base.cc:1034] running on GCE node
I20260810 04:41:27.100685 30582 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 04:41:27.100915 30582 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 04:41:27.102061 30582 hybrid_clock.cc:648] HybridClock initialized: now 1786336887102043 us; error 28 us; skew 500 ppm
Aug 10 04:41:27 dist-test-slave-bqcl krb5kdc[30136](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786336887, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.167.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 04:41:27.104615 30582 init.cc:377] Logged in from keytab as kudu/127.22.167.66@KRBTEST.COM (short username kudu)
I20260810 04:41:27.104918 30582 fs_manager.cc:359] Metadata directory not provided
I20260810 04:41:27.104969 30582 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 04:41:27.105099 30582 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 04:41:27.106143 30582 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786336863388938-23197-0/minicluster-data/ts-1/data/instance:
uuid: "d603a922a30746c7b47f8d9e9e06d0bd"
format_stamp: "Formatted at 2026-08-10 04:41:27 on dist-test-slave-bqcl"
server_key: "a5697e556f33bdc59358ff8418b02004"
server_key_iv: "25763774c7326b71b199cd79f6bb7e51"
server_key_version: "encryptionkey@0"
I20260810 04:41:27.106506 30582 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786336863388938-23197-0/minicluster-data/ts-1/wal/instance:
uuid: "d603a922a30746c7b47f8d9e9e06d0bd"
format_stamp: "Formatted at 2026-08-10 04:41:27 on dist-test-slave-bqcl"
server_key: "a5697e556f33bdc59358ff8418b02004"
server_key_iv: "25763774c7326b71b199cd79f6bb7e51"
server_key_version: "encryptionkey@0"
I20260810 04:41:27.107818 30582 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260810 04:41:27.108489 30603 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 04:41:27.108644 30582 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260810 04:41:27.108728 30582 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786336863388938-23197-0/minicluster-data/ts-1/data,/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786336863388938-23197-0/minicluster-data/ts-1/wal
uuid: "d603a922a30746c7b47f8d9e9e06d0bd"
format_stamp: "Formatted at 2026-08-10 04:41:27 on dist-test-slave-bqcl"
server_key: "a5697e556f33bdc59358ff8418b02004"
server_key_iv: "25763774c7326b71b199cd79f6bb7e51"
server_key_version: "encryptionkey@0"
I20260810 04:41:27.108793 30582 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786336863388938-23197-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786336863388938-23197-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786336863388938-23197-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 04:41:27.126318 30582 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 04:41:27.128229 30582 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 04:41:27.128365 30582 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 04:41:27.128572 30582 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 04:41:27.128865 30582 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 04:41:27.128898 30582 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 04:41:27.128926 30582 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 04:41:27.128950 30582 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 04:41:27.134214 30582 rpc_server.cc:307] RPC server started. Bound to: 127.22.167.66:41685
I20260810 04:41:27.134277 30823 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.167.66:41685 every 8 connection(s)
I20260810 04:41:27.134501 30582 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786336863388938-23197-0/minicluster-data/ts-1/data/info.pb
I20260810 04:41:27.137051 23197 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskemv_AM/build/release/bin/kudu as pid 30582
I20260810 04:41:27.137146 23197 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786336863388938-23197-0/minicluster-data/ts-1/wal/instance
I20260810 04:41:27.137315 23197 external_mini_cluster.cc:1428] Setting key 8f43547f451997efb972d5ae329a0a2e
Aug 10 04:41:27 dist-test-slave-bqcl krb5kdc[30136](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786336887, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.167.66@KRBTEST.COM for kudu/127.22.167.126@KRBTEST.COM
I20260810 04:41:27.143429 30307 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:41:27.135136 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.167.66:49005 (local address 127.22.167.126:45899)
0810 04:41:27.135182 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0810 04:41:27.135184 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 04:41:27.135364 (+ 180us) server_negotiation.cc:407] Connection header received
0810 04:41:27.135904 (+ 540us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:41:27.135905 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:41:27.135916 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:41:27.135961 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0810 04:41:27.136854 (+ 893us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:41:27.137442 (+ 588us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:41:27.138564 (+ 1122us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:41:27.138828 (+ 264us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:41:27.138898 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:41:27.140877 (+ 1979us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 04:41:27.140878 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 04:41:27.140878 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 04:41:27.140884 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0810 04:41:27.142037 (+ 1153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 04:41:27.142508 (+ 471us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 04:41:27.142509 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 04:41:27.142510 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 04:41:27.142563 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 04:41:27.142832 (+ 269us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 04:41:27.142833 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 04:41:27.142833 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 04:41:27.142928 (+ 95us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 04:41:27.142974 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0810 04:41:27.143289 (+ 315us) server_negotiation.cc:299] Negotiation successful
0810 04:41:27.143360 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260810 04:41:27.143680 30828 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:41:27.135107 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.126:45899 (local address 127.22.167.66:49005)
0810 04:41:27.135245 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0810 04:41:27.135282 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0810 04:41:27.135790 (+ 508us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 04:41:27.136025 (+ 235us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 04:41:27.136028 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 04:41:27.136277 (+ 249us) client_negotiation.cc:189] Negotiated authn=SASL
0810 04:41:27.136787 (+ 510us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:41:27.136793 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:41:27.137516 (+ 723us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:41:27.137517 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:41:27.138482 (+ 965us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:41:27.138486 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:41:27.138912 (+ 426us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:41:27.138913 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:41:27.139037 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:41:27.139473 (+ 436us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 04:41:27.139481 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0810 04:41:27.140796 (+ 1315us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 04:41:27.142115 (+ 1319us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 04:41:27.142143 (+ 28us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 04:41:27.142146 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 04:41:27.142440 (+ 294us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 04:41:27.142624 (+ 184us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 04:41:27.142625 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 04:41:27.142625 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 04:41:27.142763 (+ 138us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 04:41:27.142984 (+ 221us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 04:41:27.142991 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 04:41:27.143165 (+ 174us) client_negotiation.cc:769] Sending connection context
0810 04:41:27.143283 (+ 118us) client_negotiation.cc:240] Negotiation successful
0810 04:41:27.143452 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":98,"thread_start_us":67,"threads_started":1}
I20260810 04:41:27.144222 30824 heartbeater.cc:347] Connected to a master server at 127.22.167.126:45899
I20260810 04:41:27.144310 30824 heartbeater.cc:464] Registering TS with master...
I20260810 04:41:27.144446 30824 heartbeater.cc:511] Master 127.22.167.126:45899 requested a full tablet report, sending...
I20260810 04:41:27.144832 30214 ts_manager.cc:194] Registered new tserver with Master: d603a922a30746c7b47f8d9e9e06d0bd (127.22.167.66:41685)
I20260810 04:41:27.145356 30214 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.167.66@KRBTEST.COM'} at 127.22.167.66:49005
WARNING: no policy specified for kudu/127.22.167.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.167.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.167.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786336863388938-23197-0/krb5kdc/kudu_127.22.167.67.keytab.
Entry for principal kudu/127.22.167.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786336863388938-23197-0/krb5kdc/kudu_127.22.167.67.keytab.
I20260810 04:41:27.162976 23197 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskemv_AM/build/release/bin/kudu
/tmp/dist-test-taskemv_AM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786336863388938-23197-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786336863388938-23197-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786336863388938-23197-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786336863388938-23197-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.167.67:0
--local_ip_for_outbound_sockets=127.22.167.67
--webserver_interface=127.22.167.67
--webserver_port=0
--tserver_master_addrs=127.22.167.126:45899
--keytab_file=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786336863388938-23197-0/krb5kdc/kudu_127.22.167.67.keytab
--principal=kudu/127.22.167.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.167.84:46193
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786336863388938-23197-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786336863388938-23197-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786336863388938-23197-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 04:41:27.242655 30831 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 04:41:27.242851 30831 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 04:41:27.242872 30831 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 04:41:27.243959 30831 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 04:41:27.243992 30831 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 04:41:27.244016 30831 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.167.67
I20260810 04:41:27.245467 30831 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.167.84:46193
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786336863388938-23197-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786336863388938-23197-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786336863388938-23197-0/krb5kdc/kudu_127.22.167.67.keytab
--principal=kudu/127.22.167.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.167.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786336863388938-23197-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.22.167.67
--webserver_port=0
--tserver_master_addrs=127.22.167.126:45899
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30831
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.167.67
--log_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786336863388938-23197-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 03:54:01 UTC on ccycloud.quasar-icijmt.root.comops.site
build id 4577009
I20260810 04:41:27.245653 30831 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 04:41:27.245826 30831 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 04:41:27.250753 30838 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 04:41:27.250815 30831 server_base.cc:1034] running on GCE node
W20260810 04:41:27.250763 30840 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 04:41:27.250944 30844 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 04:41:27.251194 30831 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 04:41:27.251427 30831 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 04:41:27.252570 30831 hybrid_clock.cc:648] HybridClock initialized: now 1786336887252562 us; error 35 us; skew 500 ppm
Aug 10 04:41:27 dist-test-slave-bqcl krb5kdc[30136](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786336887, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.167.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 04:41:27.255066 30831 init.cc:377] Logged in from keytab as kudu/127.22.167.67@KRBTEST.COM (short username kudu)
I20260810 04:41:27.255368 30831 fs_manager.cc:359] Metadata directory not provided
I20260810 04:41:27.255415 30831 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 04:41:27.255509 30831 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 04:41:27.256469 30831 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786336863388938-23197-0/minicluster-data/ts-2/data/instance:
uuid: "bd662df754cb4e8e951e49a0f31e2296"
format_stamp: "Formatted at 2026-08-10 04:41:27 on dist-test-slave-bqcl"
server_key: "aa7fa46a2babbee82a1baf48aa4df58b"
server_key_iv: "5073650ceef7abd3d42d2500c55ec1a2"
server_key_version: "encryptionkey@0"
I20260810 04:41:27.256758 30831 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786336863388938-23197-0/minicluster-data/ts-2/wal/instance:
uuid: "bd662df754cb4e8e951e49a0f31e2296"
format_stamp: "Formatted at 2026-08-10 04:41:27 on dist-test-slave-bqcl"
server_key: "aa7fa46a2babbee82a1baf48aa4df58b"
server_key_iv: "5073650ceef7abd3d42d2500c55ec1a2"
server_key_version: "encryptionkey@0"
I20260810 04:41:27.257836 30831 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260810 04:41:27.258522 30855 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 04:41:27.258648 30831 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260810 04:41:27.258761 30831 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786336863388938-23197-0/minicluster-data/ts-2/data,/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786336863388938-23197-0/minicluster-data/ts-2/wal
uuid: "bd662df754cb4e8e951e49a0f31e2296"
format_stamp: "Formatted at 2026-08-10 04:41:27 on dist-test-slave-bqcl"
server_key: "aa7fa46a2babbee82a1baf48aa4df58b"
server_key_iv: "5073650ceef7abd3d42d2500c55ec1a2"
server_key_version: "encryptionkey@0"
I20260810 04:41:27.258837 30831 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786336863388938-23197-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786336863388938-23197-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786336863388938-23197-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 04:41:27.268803 30831 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 04:41:27.270800 30831 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 04:41:27.270925 30831 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 04:41:27.271157 30831 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 04:41:27.271454 30831 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 04:41:27.271488 30831 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 04:41:27.271517 30831 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 04:41:27.271533 30831 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 04:41:27.276994 30831 rpc_server.cc:307] RPC server started. Bound to: 127.22.167.67:37225
I20260810 04:41:27.277041 31067 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.167.67:37225 every 8 connection(s)
I20260810 04:41:27.277277 30831 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786336863388938-23197-0/minicluster-data/ts-2/data/info.pb
I20260810 04:41:27.277549 23197 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskemv_AM/build/release/bin/kudu as pid 30831
I20260810 04:41:27.277662 23197 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786336863388938-23197-0/minicluster-data/ts-2/wal/instance
I20260810 04:41:27.277832 23197 external_mini_cluster.cc:1428] Setting key 80558e40018194c2003185628067dfa1
Aug 10 04:41:27 dist-test-slave-bqcl krb5kdc[30136](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786336887, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.167.67@KRBTEST.COM for kudu/127.22.167.126@KRBTEST.COM
I20260810 04:41:27.285434 30307 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:41:27.277916 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.167.67:44411 (local address 127.22.167.126:45899)
0810 04:41:27.277969 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0810 04:41:27.277971 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 04:41:27.278223 (+ 252us) server_negotiation.cc:407] Connection header received
0810 04:41:27.278765 (+ 542us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:41:27.278766 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:41:27.278777 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:41:27.278823 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0810 04:41:27.279651 (+ 828us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:41:27.280130 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:41:27.281036 (+ 906us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:41:27.281215 (+ 179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:41:27.281257 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:41:27.283222 (+ 1965us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 04:41:27.283224 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 04:41:27.283224 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 04:41:27.283231 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 04:41:27.284286 (+ 1055us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 04:41:27.284669 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 04:41:27.284669 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 04:41:27.284670 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 04:41:27.284724 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 04:41:27.284891 (+ 167us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 04:41:27.284891 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 04:41:27.284892 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 04:41:27.284980 (+ 88us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 04:41:27.285023 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0810 04:41:27.285316 (+ 293us) server_negotiation.cc:299] Negotiation successful
0810 04:41:27.285373 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260810 04:41:27.285589 31073 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:41:27.277903 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.126:45899 (local address 127.22.167.67:44411)
0810 04:41:27.278056 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0810 04:41:27.278094 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0810 04:41:27.278663 (+ 569us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 04:41:27.278876 (+ 213us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 04:41:27.278879 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 04:41:27.279124 (+ 245us) client_negotiation.cc:189] Negotiated authn=SASL
0810 04:41:27.279585 (+ 461us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:41:27.279590 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:41:27.280192 (+ 602us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:41:27.280193 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:41:27.280968 (+ 775us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:41:27.280970 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:41:27.281267 (+ 297us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:41:27.281268 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:41:27.281360 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:41:27.281794 (+ 434us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 04:41:27.281802 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0810 04:41:27.283139 (+ 1337us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 04:41:27.284374 (+ 1235us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 04:41:27.284381 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 04:41:27.284383 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 04:41:27.284603 (+ 220us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 04:41:27.284768 (+ 165us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 04:41:27.284769 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 04:41:27.284769 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 04:41:27.284856 (+ 87us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 04:41:27.285069 (+ 213us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 04:41:27.285076 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 04:41:27.285192 (+ 116us) client_negotiation.cc:769] Sending connection context
0810 04:41:27.285299 (+ 107us) client_negotiation.cc:240] Negotiation successful
0810 04:41:27.285418 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":80,"threads_started":1}
I20260810 04:41:27.285980 31069 heartbeater.cc:347] Connected to a master server at 127.22.167.126:45899
I20260810 04:41:27.286067 31069 heartbeater.cc:464] Registering TS with master...
I20260810 04:41:27.286222 31069 heartbeater.cc:511] Master 127.22.167.126:45899 requested a full tablet report, sending...
I20260810 04:41:27.286552 30214 ts_manager.cc:194] Registered new tserver with Master: bd662df754cb4e8e951e49a0f31e2296 (127.22.167.67:37225)
I20260810 04:41:27.286875 30214 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.167.67@KRBTEST.COM'} at 127.22.167.67:44411
I20260810 04:41:27.289376 23197 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260810 04:41:27.295410 30307 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:41:27.290536 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38344 (local address 127.22.167.126:45899)
0810 04:41:27.290598 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0810 04:41:27.290599 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0810 04:41:27.290721 (+ 122us) server_negotiation.cc:407] Connection header received
0810 04:41:27.290759 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:41:27.290759 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:41:27.290767 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:41:27.290811 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0810 04:41:27.291311 (+ 500us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:41:27.291745 (+ 434us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:41:27.292373 (+ 628us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:41:27.292534 (+ 161us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:41:27.292572 (+ 38us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:41:27.293558 (+ 986us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 04:41:27.293558 (+ 0us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 04:41:27.293558 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 04:41:27.293562 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0810 04:41:27.294622 (+ 1060us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 04:41:27.294894 (+ 272us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 04:41:27.294895 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 04:41:27.294895 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 04:41:27.294936 (+ 41us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 04:41:27.295056 (+ 120us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 04:41:27.295057 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 04:41:27.295057 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 04:41:27.295176 (+ 119us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 04:41:27.295240 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0810 04:41:27.295307 (+ 67us) server_negotiation.cc:299] Negotiation successful
0810 04:41:27.295357 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260810 04:41:27.296710 30214 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38344:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 04:41:27.298215 30214 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 04:41:27.304148 31087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:41:27.300229 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55492 (local address 127.22.167.66:41685)
0810 04:41:27.300615 (+ 386us) server_negotiation.cc:206] Beginning negotiation
0810 04:41:27.300619 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 04:41:27.300629 (+ 10us) server_negotiation.cc:407] Connection header received
0810 04:41:27.300664 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:41:27.300666 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:41:27.300680 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:41:27.300762 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 04:41:27.300890 (+ 128us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:41:27.301614 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:41:27.303126 (+ 1512us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:41:27.303752 (+ 626us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:41:27.303816 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:41:27.303873 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0810 04:41:27.304029 (+ 156us) server_negotiation.cc:299] Negotiation successful
0810 04:41:27.304066 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":350,"thread_start_us":80,"threads_started":1}
I20260810 04:41:27.304171 31084 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:41:27.300114 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.66:41685 (local address 127.0.0.1:55492)
0810 04:41:27.300252 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0810 04:41:27.300287 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0810 04:41:27.300345 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 04:41:27.300768 (+ 423us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 04:41:27.300769 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 04:41:27.300771 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 04:41:27.300808 (+ 37us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:41:27.300809 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:41:27.301703 (+ 894us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:41:27.301704 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:41:27.303041 (+ 1337us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:41:27.303042 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:41:27.303826 (+ 784us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:41:27.303827 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:41:27.303926 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:41:27.303941 (+ 15us) client_negotiation.cc:769] Sending connection context
0810 04:41:27.304003 (+ 62us) client_negotiation.cc:240] Negotiation successful
0810 04:41:27.304066 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":47,"threads_started":1}
I20260810 04:41:27.304185 31085 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:41:27.300115 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.65:43541 (local address 127.0.0.1:51630)
0810 04:41:27.300368 (+ 253us) negotiation.cc:107] Waiting for socket to connect
0810 04:41:27.300376 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0810 04:41:27.300414 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 04:41:27.300609 (+ 195us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 04:41:27.300612 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 04:41:27.300614 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 04:41:27.300665 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:41:27.300666 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:41:27.301904 (+ 1238us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:41:27.301905 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:41:27.303039 (+ 1134us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:41:27.303042 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:41:27.303933 (+ 891us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:41:27.303934 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:41:27.303998 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:41:27.304002 (+ 4us) client_negotiation.cc:769] Sending connection context
0810 04:41:27.304042 (+ 40us) client_negotiation.cc:240] Negotiation successful
0810 04:41:27.304084 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":242,"mutex_wait_us":1,"thread_start_us":57,"threads_started":1}
I20260810 04:41:27.304474 31086 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:41:27.300229 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51630 (local address 127.22.167.65:43541)
0810 04:41:27.300404 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0810 04:41:27.300409 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 04:41:27.300421 (+ 12us) server_negotiation.cc:407] Connection header received
0810 04:41:27.300468 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:41:27.300470 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:41:27.300485 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:41:27.300578 (+ 93us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 04:41:27.300729 (+ 151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:41:27.301803 (+ 1074us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:41:27.303136 (+ 1333us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:41:27.303818 (+ 682us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:41:27.304262 (+ 444us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:41:27.304333 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0810 04:41:27.304359 (+ 26us) server_negotiation.cc:299] Negotiation successful
0810 04:41:27.304398 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":145,"thread_start_us":73,"threads_started":1}
I20260810 04:41:27.304885 31090 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:41:27.300290 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.67:37225 (local address 127.0.0.1:55118)
0810 04:41:27.300864 (+ 574us) negotiation.cc:107] Waiting for socket to connect
0810 04:41:27.300874 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0810 04:41:27.300925 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 04:41:27.301153 (+ 228us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 04:41:27.301154 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 04:41:27.301156 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 04:41:27.301194 (+ 38us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:41:27.301196 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:41:27.302307 (+ 1111us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:41:27.302308 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:41:27.303364 (+ 1056us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:41:27.303367 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:41:27.304707 (+ 1340us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:41:27.304708 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:41:27.304772 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:41:27.304776 (+ 4us) client_negotiation.cc:769] Sending connection context
0810 04:41:27.304813 (+ 37us) client_negotiation.cc:240] Negotiation successful
0810 04:41:27.304842 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":560,"thread_start_us":37,"threads_started":1}
I20260810 04:41:27.305004 30690 tablet_service.cc:1467] Processing CreateTablet for tablet b237ddd14d3a49be8455cf246e19c55c (DEFAULT_TABLE table=test-table [id=a1af2e3b58e64131bbfe4885174066d8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 04:41:27.305065 30468 tablet_service.cc:1467] Processing CreateTablet for tablet b237ddd14d3a49be8455cf246e19c55c (DEFAULT_TABLE table=test-table [id=a1af2e3b58e64131bbfe4885174066d8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 04:41:27.305191 31091 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:41:27.300429 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55118 (local address 127.22.167.67:37225)
0810 04:41:27.300561 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0810 04:41:27.300566 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 04:41:27.300977 (+ 411us) server_negotiation.cc:407] Connection header received
0810 04:41:27.301013 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:41:27.301015 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:41:27.301027 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:41:27.301088 (+ 61us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 04:41:27.301256 (+ 168us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:41:27.302232 (+ 976us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:41:27.303428 (+ 1196us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:41:27.304131 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:41:27.304186 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:41:27.305073 (+ 887us) server_negotiation.cc:1035] Waiting for connection context
0810 04:41:27.305097 (+ 24us) server_negotiation.cc:299] Negotiation successful
0810 04:41:27.305132 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":104,"thread_start_us":77,"threads_started":1}
I20260810 04:41:27.305372 30468 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b237ddd14d3a49be8455cf246e19c55c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 04:41:27.305372 30690 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b237ddd14d3a49be8455cf246e19c55c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 04:41:27.305707 30955 tablet_service.cc:1467] Processing CreateTablet for tablet b237ddd14d3a49be8455cf246e19c55c (DEFAULT_TABLE table=test-table [id=a1af2e3b58e64131bbfe4885174066d8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 04:41:27.305936 30955 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b237ddd14d3a49be8455cf246e19c55c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 04:41:27.308379 31092 tablet_bootstrap.cc:492] T b237ddd14d3a49be8455cf246e19c55c P bd662df754cb4e8e951e49a0f31e2296: Bootstrap starting.
I20260810 04:41:27.308744 31095 tablet_bootstrap.cc:492] T b237ddd14d3a49be8455cf246e19c55c P d603a922a30746c7b47f8d9e9e06d0bd: Bootstrap starting.
I20260810 04:41:27.309033 31092 tablet_bootstrap.cc:654] T b237ddd14d3a49be8455cf246e19c55c P bd662df754cb4e8e951e49a0f31e2296: Neither blocks nor log segments found. Creating new log.
I20260810 04:41:27.309405 31095 tablet_bootstrap.cc:654] T b237ddd14d3a49be8455cf246e19c55c P d603a922a30746c7b47f8d9e9e06d0bd: Neither blocks nor log segments found. Creating new log.
I20260810 04:41:27.309662 31092 log.cc:824] T b237ddd14d3a49be8455cf246e19c55c P bd662df754cb4e8e951e49a0f31e2296: Log is configured to *not* fsync() on all Append() calls
I20260810 04:41:27.310025 31095 log.cc:824] T b237ddd14d3a49be8455cf246e19c55c P d603a922a30746c7b47f8d9e9e06d0bd: Log is configured to *not* fsync() on all Append() calls
I20260810 04:41:27.310297 31094 tablet_bootstrap.cc:492] T b237ddd14d3a49be8455cf246e19c55c P 331778d4b11b4b7f8609aefb32a1b264: Bootstrap starting.
I20260810 04:41:27.310434 31092 tablet_bootstrap.cc:492] T b237ddd14d3a49be8455cf246e19c55c P bd662df754cb4e8e951e49a0f31e2296: No bootstrap required, opened a new log
I20260810 04:41:27.310503 31092 ts_tablet_manager.cc:1397] T b237ddd14d3a49be8455cf246e19c55c P bd662df754cb4e8e951e49a0f31e2296: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260810 04:41:27.310806 31095 tablet_bootstrap.cc:492] T b237ddd14d3a49be8455cf246e19c55c P d603a922a30746c7b47f8d9e9e06d0bd: No bootstrap required, opened a new log
I20260810 04:41:27.310873 31095 ts_tablet_manager.cc:1397] T b237ddd14d3a49be8455cf246e19c55c P d603a922a30746c7b47f8d9e9e06d0bd: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260810 04:41:27.311103 31094 tablet_bootstrap.cc:654] T b237ddd14d3a49be8455cf246e19c55c P 331778d4b11b4b7f8609aefb32a1b264: Neither blocks nor log segments found. Creating new log.
I20260810 04:41:27.311671 31094 log.cc:824] T b237ddd14d3a49be8455cf246e19c55c P 331778d4b11b4b7f8609aefb32a1b264: Log is configured to *not* fsync() on all Append() calls
I20260810 04:41:27.312318 31094 tablet_bootstrap.cc:492] T b237ddd14d3a49be8455cf246e19c55c P 331778d4b11b4b7f8609aefb32a1b264: No bootstrap required, opened a new log
I20260810 04:41:27.312386 31094 ts_tablet_manager.cc:1397] T b237ddd14d3a49be8455cf246e19c55c P 331778d4b11b4b7f8609aefb32a1b264: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260810 04:41:27.312358 31092 raft_consensus.cc:348] T b237ddd14d3a49be8455cf246e19c55c P bd662df754cb4e8e951e49a0f31e2296 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d603a922a30746c7b47f8d9e9e06d0bd" member_type: VOTER last_known_addr { host: "127.22.167.66" port: 41685 } } peers { permanent_uuid: "bd662df754cb4e8e951e49a0f31e2296" member_type: VOTER last_known_addr { host: "127.22.167.67" port: 37225 } } peers { permanent_uuid: "331778d4b11b4b7f8609aefb32a1b264" member_type: VOTER last_known_addr { host: "127.22.167.65" port: 43541 } }
I20260810 04:41:27.312525 31092 raft_consensus.cc:374] T b237ddd14d3a49be8455cf246e19c55c P bd662df754cb4e8e951e49a0f31e2296 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 04:41:27.312565 31092 raft_consensus.cc:729] T b237ddd14d3a49be8455cf246e19c55c P bd662df754cb4e8e951e49a0f31e2296 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd662df754cb4e8e951e49a0f31e2296, State: Initialized, Role: FOLLOWER
I20260810 04:41:27.312695 31092 consensus_queue.cc:260] T b237ddd14d3a49be8455cf246e19c55c P bd662df754cb4e8e951e49a0f31e2296 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d603a922a30746c7b47f8d9e9e06d0bd" member_type: VOTER last_known_addr { host: "127.22.167.66" port: 41685 } } peers { permanent_uuid: "bd662df754cb4e8e951e49a0f31e2296" member_type: VOTER last_known_addr { host: "127.22.167.67" port: 37225 } } peers { permanent_uuid: "331778d4b11b4b7f8609aefb32a1b264" member_type: VOTER last_known_addr { host: "127.22.167.65" port: 43541 } }
I20260810 04:41:27.312704 31095 raft_consensus.cc:348] T b237ddd14d3a49be8455cf246e19c55c P d603a922a30746c7b47f8d9e9e06d0bd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d603a922a30746c7b47f8d9e9e06d0bd" member_type: VOTER last_known_addr { host: "127.22.167.66" port: 41685 } } peers { permanent_uuid: "bd662df754cb4e8e951e49a0f31e2296" member_type: VOTER last_known_addr { host: "127.22.167.67" port: 37225 } } peers { permanent_uuid: "331778d4b11b4b7f8609aefb32a1b264" member_type: VOTER last_known_addr { host: "127.22.167.65" port: 43541 } }
I20260810 04:41:27.312844 31095 raft_consensus.cc:374] T b237ddd14d3a49be8455cf246e19c55c P d603a922a30746c7b47f8d9e9e06d0bd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 04:41:27.312889 31095 raft_consensus.cc:729] T b237ddd14d3a49be8455cf246e19c55c P d603a922a30746c7b47f8d9e9e06d0bd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d603a922a30746c7b47f8d9e9e06d0bd, State: Initialized, Role: FOLLOWER
I20260810 04:41:27.312969 31092 ts_tablet_manager.cc:1428] T b237ddd14d3a49be8455cf246e19c55c P bd662df754cb4e8e951e49a0f31e2296: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260810 04:41:27.312994 31095 consensus_queue.cc:260] T b237ddd14d3a49be8455cf246e19c55c P d603a922a30746c7b47f8d9e9e06d0bd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d603a922a30746c7b47f8d9e9e06d0bd" member_type: VOTER last_known_addr { host: "127.22.167.66" port: 41685 } } peers { permanent_uuid: "bd662df754cb4e8e951e49a0f31e2296" member_type: VOTER last_known_addr { host: "127.22.167.67" port: 37225 } } peers { permanent_uuid: "331778d4b11b4b7f8609aefb32a1b264" member_type: VOTER last_known_addr { host: "127.22.167.65" port: 43541 } }
I20260810 04:41:27.313094 31069 heartbeater.cc:503] Master 127.22.167.126:45899 was elected leader, sending a full tablet report...
I20260810 04:41:27.313246 31095 ts_tablet_manager.cc:1428] T b237ddd14d3a49be8455cf246e19c55c P d603a922a30746c7b47f8d9e9e06d0bd: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260810 04:41:27.313256 30824 heartbeater.cc:503] Master 127.22.167.126:45899 was elected leader, sending a full tablet report...
I20260810 04:41:27.314265 31094 raft_consensus.cc:348] T b237ddd14d3a49be8455cf246e19c55c P 331778d4b11b4b7f8609aefb32a1b264 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d603a922a30746c7b47f8d9e9e06d0bd" member_type: VOTER last_known_addr { host: "127.22.167.66" port: 41685 } } peers { permanent_uuid: "bd662df754cb4e8e951e49a0f31e2296" member_type: VOTER last_known_addr { host: "127.22.167.67" port: 37225 } } peers { permanent_uuid: "331778d4b11b4b7f8609aefb32a1b264" member_type: VOTER last_known_addr { host: "127.22.167.65" port: 43541 } }
I20260810 04:41:27.314437 31094 raft_consensus.cc:374] T b237ddd14d3a49be8455cf246e19c55c P 331778d4b11b4b7f8609aefb32a1b264 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 04:41:27.314483 31094 raft_consensus.cc:729] T b237ddd14d3a49be8455cf246e19c55c P 331778d4b11b4b7f8609aefb32a1b264 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 331778d4b11b4b7f8609aefb32a1b264, State: Initialized, Role: FOLLOWER
I20260810 04:41:27.314599 31094 consensus_queue.cc:260] T b237ddd14d3a49be8455cf246e19c55c P 331778d4b11b4b7f8609aefb32a1b264 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d603a922a30746c7b47f8d9e9e06d0bd" member_type: VOTER last_known_addr { host: "127.22.167.66" port: 41685 } } peers { permanent_uuid: "bd662df754cb4e8e951e49a0f31e2296" member_type: VOTER last_known_addr { host: "127.22.167.67" port: 37225 } } peers { permanent_uuid: "331778d4b11b4b7f8609aefb32a1b264" member_type: VOTER last_known_addr { host: "127.22.167.65" port: 43541 } }
I20260810 04:41:27.314853 31094 ts_tablet_manager.cc:1428] T b237ddd14d3a49be8455cf246e19c55c P 331778d4b11b4b7f8609aefb32a1b264: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260810 04:41:27.314872 30575 heartbeater.cc:503] Master 127.22.167.126:45899 was elected leader, sending a full tablet report...
W20260810 04:41:27.384968 30827 tablet.cc:2369] T b237ddd14d3a49be8455cf246e19c55c P d603a922a30746c7b47f8d9e9e06d0bd: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 04:41:27.425829 31101 raft_consensus.cc:482] T b237ddd14d3a49be8455cf246e19c55c P bd662df754cb4e8e951e49a0f31e2296 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 04:41:27.425959 31101 raft_consensus.cc:504] T b237ddd14d3a49be8455cf246e19c55c P bd662df754cb4e8e951e49a0f31e2296 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d603a922a30746c7b47f8d9e9e06d0bd" member_type: VOTER last_known_addr { host: "127.22.167.66" port: 41685 } } peers { permanent_uuid: "bd662df754cb4e8e951e49a0f31e2296" member_type: VOTER last_known_addr { host: "127.22.167.67" port: 37225 } } peers { permanent_uuid: "331778d4b11b4b7f8609aefb32a1b264" member_type: VOTER last_known_addr { host: "127.22.167.65" port: 43541 } }
I20260810 04:41:27.426352 31101 leader_election.cc:288] T b237ddd14d3a49be8455cf246e19c55c P bd662df754cb4e8e951e49a0f31e2296 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d603a922a30746c7b47f8d9e9e06d0bd (127.22.167.66:41685), 331778d4b11b4b7f8609aefb32a1b264 (127.22.167.65:43541)
I20260810 04:41:27.429031 31087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:41:27.426625 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.167.67:42495 (local address 127.22.167.66:41685)
0810 04:41:27.426665 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0810 04:41:27.426667 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 04:41:27.426810 (+ 143us) server_negotiation.cc:407] Connection header received
0810 04:41:27.426842 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:41:27.426843 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:41:27.426852 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:41:27.426900 (+ 48us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 04:41:27.427033 (+ 133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:41:27.427476 (+ 443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:41:27.428375 (+ 899us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:41:27.428739 (+ 364us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:41:27.428798 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:41:27.428807 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0810 04:41:27.428935 (+ 128us) server_negotiation.cc:299] Negotiation successful
0810 04:41:27.428970 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260810 04:41:27.429075 31073 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:41:27.426535 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.65:43541 (local address 127.22.167.67:33365)
0810 04:41:27.426631 (+ 96us) negotiation.cc:107] Waiting for socket to connect
0810 04:41:27.426642 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0810 04:41:27.426685 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 04:41:27.426837 (+ 152us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 04:41:27.426838 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 04:41:27.426840 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 04:41:27.426910 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:41:27.426911 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:41:27.427525 (+ 614us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:41:27.427526 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:41:27.428288 (+ 762us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:41:27.428290 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:41:27.428821 (+ 531us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:41:27.428822 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:41:27.428894 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:41:27.428897 (+ 3us) client_negotiation.cc:769] Sending connection context
0810 04:41:27.428942 (+ 45us) client_negotiation.cc:240] Negotiation successful
0810 04:41:27.428982 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":76,"mutex_wait_us":3}
I20260810 04:41:27.429067 31086 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:41:27.426639 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.167.67:33365 (local address 127.22.167.65:43541)
0810 04:41:27.426688 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0810 04:41:27.426690 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 04:41:27.426700 (+ 10us) server_negotiation.cc:407] Connection header received
0810 04:41:27.426736 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:41:27.426737 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:41:27.426753 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:41:27.426820 (+ 67us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 04:41:27.426974 (+ 154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:41:27.427450 (+ 476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:41:27.428380 (+ 930us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:41:27.428748 (+ 368us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:41:27.428809 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:41:27.428817 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0810 04:41:27.428959 (+ 142us) server_negotiation.cc:299] Negotiation successful
0810 04:41:27.428992 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260810 04:41:27.429101 31104 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:41:27.426540 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.66:41685 (local address 127.22.167.67:42495)
0810 04:41:27.426733 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0810 04:41:27.426742 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0810 04:41:27.426798 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 04:41:27.426910 (+ 112us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 04:41:27.426910 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 04:41:27.426912 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 04:41:27.426974 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:41:27.426976 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:41:27.427546 (+ 570us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:41:27.427547 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:41:27.428290 (+ 743us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:41:27.428291 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:41:27.428811 (+ 520us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:41:27.428811 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:41:27.428874 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:41:27.428878 (+ 4us) client_negotiation.cc:769] Sending connection context
0810 04:41:27.428921 (+ 43us) client_negotiation.cc:240] Negotiation successful
0810 04:41:27.428958 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":178,"mutex_wait_us":16,"thread_start_us":104,"threads_started":1}
I20260810 04:41:27.429407 30725 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b237ddd14d3a49be8455cf246e19c55c" candidate_uuid: "bd662df754cb4e8e951e49a0f31e2296" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d603a922a30746c7b47f8d9e9e06d0bd" is_pre_election: true
I20260810 04:41:27.429410 30495 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b237ddd14d3a49be8455cf246e19c55c" candidate_uuid: "bd662df754cb4e8e951e49a0f31e2296" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "331778d4b11b4b7f8609aefb32a1b264" is_pre_election: true
I20260810 04:41:27.429556 30725 raft_consensus.cc:2454] T b237ddd14d3a49be8455cf246e19c55c P d603a922a30746c7b47f8d9e9e06d0bd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bd662df754cb4e8e951e49a0f31e2296 in term 0.
I20260810 04:41:27.429548 30495 raft_consensus.cc:2454] T b237ddd14d3a49be8455cf246e19c55c P 331778d4b11b4b7f8609aefb32a1b264 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bd662df754cb4e8e951e49a0f31e2296 in term 0.
I20260810 04:41:27.429788 30860 leader_election.cc:302] T b237ddd14d3a49be8455cf246e19c55c P bd662df754cb4e8e951e49a0f31e2296 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bd662df754cb4e8e951e49a0f31e2296, d603a922a30746c7b47f8d9e9e06d0bd; no voters:
I20260810 04:41:27.429949 31101 raft_consensus.cc:2781] T b237ddd14d3a49be8455cf246e19c55c P bd662df754cb4e8e951e49a0f31e2296 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 04:41:27.429998 31101 raft_consensus.cc:482] T b237ddd14d3a49be8455cf246e19c55c P bd662df754cb4e8e951e49a0f31e2296 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 04:41:27.430013 31101 raft_consensus.cc:3037] T b237ddd14d3a49be8455cf246e19c55c P bd662df754cb4e8e951e49a0f31e2296 [term 0 FOLLOWER]: Advancing to term 1
I20260810 04:41:27.430706 31101 raft_consensus.cc:504] T b237ddd14d3a49be8455cf246e19c55c P bd662df754cb4e8e951e49a0f31e2296 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d603a922a30746c7b47f8d9e9e06d0bd" member_type: VOTER last_known_addr { host: "127.22.167.66" port: 41685 } } peers { permanent_uuid: "bd662df754cb4e8e951e49a0f31e2296" member_type: VOTER last_known_addr { host: "127.22.167.67" port: 37225 } } peers { permanent_uuid: "331778d4b11b4b7f8609aefb32a1b264" member_type: VOTER last_known_addr { host: "127.22.167.65" port: 43541 } }
I20260810 04:41:27.430817 31101 leader_election.cc:288] T b237ddd14d3a49be8455cf246e19c55c P bd662df754cb4e8e951e49a0f31e2296 [CANDIDATE]: Term 1 election: Requested vote from peers d603a922a30746c7b47f8d9e9e06d0bd (127.22.167.66:41685), 331778d4b11b4b7f8609aefb32a1b264 (127.22.167.65:43541)
I20260810 04:41:27.430992 30725 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b237ddd14d3a49be8455cf246e19c55c" candidate_uuid: "bd662df754cb4e8e951e49a0f31e2296" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d603a922a30746c7b47f8d9e9e06d0bd"
I20260810 04:41:27.431005 30495 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b237ddd14d3a49be8455cf246e19c55c" candidate_uuid: "bd662df754cb4e8e951e49a0f31e2296" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "331778d4b11b4b7f8609aefb32a1b264"
I20260810 04:41:27.431064 30495 raft_consensus.cc:3037] T b237ddd14d3a49be8455cf246e19c55c P 331778d4b11b4b7f8609aefb32a1b264 [term 0 FOLLOWER]: Advancing to term 1
I20260810 04:41:27.431064 30725 raft_consensus.cc:3037] T b237ddd14d3a49be8455cf246e19c55c P d603a922a30746c7b47f8d9e9e06d0bd [term 0 FOLLOWER]: Advancing to term 1
I20260810 04:41:27.431746 30495 raft_consensus.cc:2454] T b237ddd14d3a49be8455cf246e19c55c P 331778d4b11b4b7f8609aefb32a1b264 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd662df754cb4e8e951e49a0f31e2296 in term 1.
I20260810 04:41:27.431754 30725 raft_consensus.cc:2454] T b237ddd14d3a49be8455cf246e19c55c P d603a922a30746c7b47f8d9e9e06d0bd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd662df754cb4e8e951e49a0f31e2296 in term 1.
I20260810 04:41:27.431890 30860 leader_election.cc:302] T b237ddd14d3a49be8455cf246e19c55c P bd662df754cb4e8e951e49a0f31e2296 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bd662df754cb4e8e951e49a0f31e2296, d603a922a30746c7b47f8d9e9e06d0bd; no voters:
I20260810 04:41:27.432001 31101 raft_consensus.cc:2781] T b237ddd14d3a49be8455cf246e19c55c P bd662df754cb4e8e951e49a0f31e2296 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 04:41:27.432157 31101 raft_consensus.cc:686] T b237ddd14d3a49be8455cf246e19c55c P bd662df754cb4e8e951e49a0f31e2296 [term 1 LEADER]: Becoming Leader. State: Replica: bd662df754cb4e8e951e49a0f31e2296, State: Running, Role: LEADER
I20260810 04:41:27.432238 31101 consensus_queue.cc:237] T b237ddd14d3a49be8455cf246e19c55c P bd662df754cb4e8e951e49a0f31e2296 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d603a922a30746c7b47f8d9e9e06d0bd" member_type: VOTER last_known_addr { host: "127.22.167.66" port: 41685 } } peers { permanent_uuid: "bd662df754cb4e8e951e49a0f31e2296" member_type: VOTER last_known_addr { host: "127.22.167.67" port: 37225 } } peers { permanent_uuid: "331778d4b11b4b7f8609aefb32a1b264" member_type: VOTER last_known_addr { host: "127.22.167.65" port: 43541 } }
I20260810 04:41:27.432987 30212 catalog_manager.cc:5495] T b237ddd14d3a49be8455cf246e19c55c P bd662df754cb4e8e951e49a0f31e2296 reported cstate change: term changed from 0 to 1, leader changed from <none> to bd662df754cb4e8e951e49a0f31e2296 (127.22.167.67). New cstate: current_term: 1 leader_uuid: "bd662df754cb4e8e951e49a0f31e2296" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d603a922a30746c7b47f8d9e9e06d0bd" member_type: VOTER last_known_addr { host: "127.22.167.66" port: 41685 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bd662df754cb4e8e951e49a0f31e2296" member_type: VOTER last_known_addr { host: "127.22.167.67" port: 37225 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "331778d4b11b4b7f8609aefb32a1b264" member_type: VOTER last_known_addr { host: "127.22.167.65" port: 43541 } health_report { overall_health: UNKNOWN } } }
I20260810 04:41:27.449965 31091 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:41:27.447770 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55132 (local address 127.22.167.67:37225)
0810 04:41:27.447818 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0810 04:41:27.447820 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 04:41:27.447827 (+ 7us) server_negotiation.cc:407] Connection header received
0810 04:41:27.447886 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:41:27.447887 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:41:27.447904 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:41:27.447962 (+ 58us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 04:41:27.448100 (+ 138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:41:27.448552 (+ 452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:41:27.449261 (+ 709us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:41:27.449456 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:41:27.449516 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:41:27.449664 (+ 148us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 04:41:27.449761 (+ 97us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 04:41:27.449830 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0810 04:41:27.449865 (+ 35us) server_negotiation.cc:299] Negotiation successful
0810 04:41:27.449883 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260810 04:41:27.451536 30725 raft_consensus.cc:1261] T b237ddd14d3a49be8455cf246e19c55c P d603a922a30746c7b47f8d9e9e06d0bd [term 1 FOLLOWER]: Refusing update from remote peer bd662df754cb4e8e951e49a0f31e2296: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 04:41:27.451536 30495 raft_consensus.cc:1261] T b237ddd14d3a49be8455cf246e19c55c P 331778d4b11b4b7f8609aefb32a1b264 [term 1 FOLLOWER]: Refusing update from remote peer bd662df754cb4e8e951e49a0f31e2296: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 04:41:27.451805 31101 consensus_queue.cc:1035] T b237ddd14d3a49be8455cf246e19c55c P bd662df754cb4e8e951e49a0f31e2296 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d603a922a30746c7b47f8d9e9e06d0bd" member_type: VOTER last_known_addr { host: "127.22.167.66" port: 41685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 04:41:27.451941 31105 consensus_queue.cc:1035] T b237ddd14d3a49be8455cf246e19c55c P bd662df754cb4e8e951e49a0f31e2296 [LEADER]: Connected to new peer: Peer: permanent_uuid: "331778d4b11b4b7f8609aefb32a1b264" member_type: VOTER last_known_addr { host: "127.22.167.65" port: 43541 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 04:41:27.454111 31110 mvcc.cc:205] Tried to move back new op lower bound from 7316835890999402496 to 7316835890923061248. Current Snapshot: MvccSnapshot[applied={T|T < 7316835890999402496}]
I20260810 04:41:27.454540 31114 mvcc.cc:205] Tried to move back new op lower bound from 7316835890999402496 to 7316835890923061248. Current Snapshot: MvccSnapshot[applied={T|T < 7316835890999402496}]
I20260810 04:41:27.454679 31111 mvcc.cc:205] Tried to move back new op lower bound from 7316835890999402496 to 7316835890923061248. Current Snapshot: MvccSnapshot[applied={T|T < 7316835890999402496}]
I20260810 04:41:27.456493 30214 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38344:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 04:41:27.456578 30214 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38344:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 04:41:27.457219 30214 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 001d663cc0da4b09af7cb8f1e4f0eda5: Sending DeleteTablet for 3 replicas of tablet b237ddd14d3a49be8455cf246e19c55c
I20260810 04:41:27.457547 30955 tablet_service.cc:1514] Processing DeleteTablet for tablet b237ddd14d3a49be8455cf246e19c55c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 04:41:27 UTC) from {username='kudu', principal='kudu/127.22.167.126@KRBTEST.COM'} at 127.0.0.1:55118
I20260810 04:41:27.457552 30468 tablet_service.cc:1514] Processing DeleteTablet for tablet b237ddd14d3a49be8455cf246e19c55c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 04:41:27 UTC) from {username='kudu', principal='kudu/127.22.167.126@KRBTEST.COM'} at 127.0.0.1:51630
I20260810 04:41:27.457751 30690 tablet_service.cc:1514] Processing DeleteTablet for tablet b237ddd14d3a49be8455cf246e19c55c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 04:41:27 UTC) from {username='kudu', principal='kudu/127.22.167.126@KRBTEST.COM'} at 127.0.0.1:55492
I20260810 04:41:27.457938 31129 tablet_replica.cc:330] T b237ddd14d3a49be8455cf246e19c55c P d603a922a30746c7b47f8d9e9e06d0bd: stopping tablet replica
I20260810 04:41:27.458040 31129 raft_consensus.cc:2229] T b237ddd14d3a49be8455cf246e19c55c P d603a922a30746c7b47f8d9e9e06d0bd [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 04:41:27.458169 31129 raft_consensus.cc:2258] T b237ddd14d3a49be8455cf246e19c55c P d603a922a30746c7b47f8d9e9e06d0bd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 04:41:27.458171 23197 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskemv_AM/build/release/bin/kudu with pid 30339
I20260810 04:41:27.469743 31128 tablet_replica.cc:330] T b237ddd14d3a49be8455cf246e19c55c P bd662df754cb4e8e951e49a0f31e2296: stopping tablet replica
I20260810 04:41:27.469883 31128 raft_consensus.cc:2229] T b237ddd14d3a49be8455cf246e19c55c P bd662df754cb4e8e951e49a0f31e2296 [term 1 LEADER]: Raft consensus shutting down.
I20260810 04:41:27.470023 31128 raft_consensus.cc:2258] T b237ddd14d3a49be8455cf246e19c55c P bd662df754cb4e8e951e49a0f31e2296 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 04:41:27.471418 31128 ts_tablet_manager.cc:1905] T b237ddd14d3a49be8455cf246e19c55c P bd662df754cb4e8e951e49a0f31e2296: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 04:41:27.473135 31128 ts_tablet_manager.cc:1918] T b237ddd14d3a49be8455cf246e19c55c P bd662df754cb4e8e951e49a0f31e2296: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 04:41:27.473213 31128 log.cc:1196] T b237ddd14d3a49be8455cf246e19c55c P bd662df754cb4e8e951e49a0f31e2296: Deleting WAL directory at /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786336863388938-23197-0/minicluster-data/ts-2/wal/wals/b237ddd14d3a49be8455cf246e19c55c
I20260810 04:41:27.473472 31128 ts_tablet_manager.cc:1939] T b237ddd14d3a49be8455cf246e19c55c P bd662df754cb4e8e951e49a0f31e2296: Deleting consensus metadata
I20260810 04:41:27.473919 30193 catalog_manager.cc:4841] TS bd662df754cb4e8e951e49a0f31e2296 (127.22.167.67:37225): tablet b237ddd14d3a49be8455cf246e19c55c (table test-table [id=a1af2e3b58e64131bbfe4885174066d8]) successfully deleted
I20260810 04:41:27.474315 31129 ts_tablet_manager.cc:1905] T b237ddd14d3a49be8455cf246e19c55c P d603a922a30746c7b47f8d9e9e06d0bd: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260810 04:41:27.475247 30188 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.22.167.65:43541 (error 108)
I20260810 04:41:27.475521 23197 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskemv_AM/build/release/bin/kudu with pid 30582
I20260810 04:41:27.475852 31129 ts_tablet_manager.cc:1918] T b237ddd14d3a49be8455cf246e19c55c P d603a922a30746c7b47f8d9e9e06d0bd: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 04:41:27.475867 31090 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:41:27.475759 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.65:43541 (local address 127.0.0.1:51632)
0810 04:41:27.475805 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0810 04:41:27.475835 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.167.65:43541: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
W20260810 04:41:27.476122 30188 catalog_manager.cc:4584] TS 331778d4b11b4b7f8609aefb32a1b264 (127.22.167.65:43541): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet b237ddd14d3a49be8455cf246e19c55c: Network error: Client connection negotiation failed: client connection to 127.22.167.65:43541: connect: Connection refused (error 111)
I20260810 04:41:27.485015 23197 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskemv_AM/build/release/bin/kudu with pid 30831
I20260810 04:41:27.490625 31090 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:41:27.485059 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.66:41685 (local address 127.0.0.1:55498)
0810 04:41:27.490556 (+ 5497us) negotiation.cc:107] Waiting for socket to connect
0810 04:41:27.490591 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.167.66:41685: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":5471}
I20260810 04:41:27.497937 23197 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskemv_AM/build/release/bin/kudu with pid 30161
2026-08-10T04:41:27Z chronyd exiting
[ OK ] SecurityITest.TestDisableWebUI (2114 ms)
[ RUN ] SecurityITest.TestDisableAuthenticationEncryption
2026-08-10T04:41:27Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T04:41:27Z Disabled control of system clock
I20260810 04:41:27.531461 23197 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskemv_AM/build/release/bin/kudu
/tmp/dist-test-taskemv_AM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786336863388938-23197-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786336863388938-23197-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786336863388938-23197-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786336863388938-23197-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.167.126:43457
--webserver_interface=127.22.167.126
--webserver_port=0
--builtin_ntp_servers=127.22.167.84:43571
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.167.126:43457
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260810 04:41:27.612958 31141 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 04:41:27.613174 31141 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 04:41:27.613197 31141 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 04:41:27.614333 31141 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 04:41:27.614377 31141 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 04:41:27.614391 31141 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 04:41:27.614404 31141 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 04:41:27.614418 31141 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 04:41:27.615772 31141 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.167.84:43571
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786336863388938-23197-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786336863388938-23197-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.22.167.126:43457
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.167.126:43457
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786336863388938-23197-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.167.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31141
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786336863388938-23197-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 03:54:01 UTC on ccycloud.quasar-icijmt.root.comops.site
build id 4577009
I20260810 04:41:27.616008 31141 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 04:41:27.616284 31141 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 04:41:27.620911 31152 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 04:41:27.620915 31150 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 04:41:27.620927 31148 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 04:41:27.620944 31141 server_base.cc:1034] running on GCE node
I20260810 04:41:27.621423 31141 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 04:41:27.621661 31141 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 04:41:27.622834 31141 hybrid_clock.cc:648] HybridClock initialized: now 1786336887622819 us; error 29 us; skew 500 ppm
I20260810 04:41:27.624038 31141 webserver.cc:460] Webserver started at http://127.22.167.126:44337/ using document root <none> and password file <none>
I20260810 04:41:27.624248 31141 fs_manager.cc:359] Metadata directory not provided
I20260810 04:41:27.624300 31141 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 04:41:27.624437 31141 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 04:41:27.625495 31141 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786336863388938-23197-0/minicluster-data/master-0/data/instance:
uuid: "8ee700781ae24713979cb0e90821d859"
format_stamp: "Formatted at 2026-08-10 04:41:27 on dist-test-slave-bqcl"
server_key: "ec697b5b379f1ba5711cea7158305803"
server_key_iv: "c8440c81a91dde5052f4c6c5eaeff3b7"
server_key_version: "encryptionkey@0"
I20260810 04:41:27.625839 31141 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786336863388938-23197-0/minicluster-data/master-0/wal/instance:
uuid: "8ee700781ae24713979cb0e90821d859"
format_stamp: "Formatted at 2026-08-10 04:41:27 on dist-test-slave-bqcl"
server_key: "ec697b5b379f1ba5711cea7158305803"
server_key_iv: "c8440c81a91dde5052f4c6c5eaeff3b7"
server_key_version: "encryptionkey@0"
I20260810 04:41:27.627154 31141 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260810 04:41:27.627888 31164 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 04:41:27.628069 31141 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260810 04:41:27.628158 31141 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786336863388938-23197-0/minicluster-data/master-0/data,/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786336863388938-23197-0/minicluster-data/master-0/wal
uuid: "8ee700781ae24713979cb0e90821d859"
format_stamp: "Formatted at 2026-08-10 04:41:27 on dist-test-slave-bqcl"
server_key: "ec697b5b379f1ba5711cea7158305803"
server_key_iv: "c8440c81a91dde5052f4c6c5eaeff3b7"
server_key_version: "encryptionkey@0"
I20260810 04:41:27.628229 31141 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786336863388938-23197-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786336863388938-23197-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786336863388938-23197-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 04:41:27.631522 31141 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 04:41:27.631767 31141 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 04:41:27.631898 31141 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 04:41:27.636011 31141 rpc_server.cc:307] RPC server started. Bound to: 127.22.167.126:43457
I20260810 04:41:27.636041 31271 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.167.126:43457 every 8 connection(s)
I20260810 04:41:27.636335 31141 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786336863388938-23197-0/minicluster-data/master-0/data/info.pb
I20260810 04:41:27.636946 31272 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 04:41:27.639590 31272 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8ee700781ae24713979cb0e90821d859: Bootstrap starting.
I20260810 04:41:27.640115 31272 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8ee700781ae24713979cb0e90821d859: Neither blocks nor log segments found. Creating new log.
I20260810 04:41:27.640389 31272 log.cc:824] T 00000000000000000000000000000000 P 8ee700781ae24713979cb0e90821d859: Log is configured to *not* fsync() on all Append() calls
I20260810 04:41:27.641002 31272 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8ee700781ae24713979cb0e90821d859: No bootstrap required, opened a new log
I20260810 04:41:27.642570 31272 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8ee700781ae24713979cb0e90821d859 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8ee700781ae24713979cb0e90821d859" member_type: VOTER last_known_addr { host: "127.22.167.126" port: 43457 } }
I20260810 04:41:27.642696 31272 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8ee700781ae24713979cb0e90821d859 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 04:41:27.642717 31272 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8ee700781ae24713979cb0e90821d859 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8ee700781ae24713979cb0e90821d859, State: Initialized, Role: FOLLOWER
I20260810 04:41:27.642843 31272 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8ee700781ae24713979cb0e90821d859 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8ee700781ae24713979cb0e90821d859" member_type: VOTER last_known_addr { host: "127.22.167.126" port: 43457 } }
I20260810 04:41:27.642902 31272 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8ee700781ae24713979cb0e90821d859 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 04:41:27.642925 31272 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8ee700781ae24713979cb0e90821d859 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 04:41:27.642968 31272 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8ee700781ae24713979cb0e90821d859 [term 0 FOLLOWER]: Advancing to term 1
I20260810 04:41:27.643468 31272 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8ee700781ae24713979cb0e90821d859 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8ee700781ae24713979cb0e90821d859" member_type: VOTER last_known_addr { host: "127.22.167.126" port: 43457 } }
I20260810 04:41:27.643559 31272 leader_election.cc:302] T 00000000000000000000000000000000 P 8ee700781ae24713979cb0e90821d859 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8ee700781ae24713979cb0e90821d859; no voters:
I20260810 04:41:27.643714 31272 leader_election.cc:288] T 00000000000000000000000000000000 P 8ee700781ae24713979cb0e90821d859 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 04:41:27.643781 31275 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8ee700781ae24713979cb0e90821d859 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 04:41:27.643935 31272 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8ee700781ae24713979cb0e90821d859 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 04:41:27.643942 31275 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8ee700781ae24713979cb0e90821d859 [term 1 LEADER]: Becoming Leader. State: Replica: 8ee700781ae24713979cb0e90821d859, State: Running, Role: LEADER
I20260810 04:41:27.644057 31275 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8ee700781ae24713979cb0e90821d859 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8ee700781ae24713979cb0e90821d859" member_type: VOTER last_known_addr { host: "127.22.167.126" port: 43457 } }
I20260810 04:41:27.644462 31276 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8ee700781ae24713979cb0e90821d859 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8ee700781ae24713979cb0e90821d859" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8ee700781ae24713979cb0e90821d859" member_type: VOTER last_known_addr { host: "127.22.167.126" port: 43457 } } }
I20260810 04:41:27.644564 31276 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8ee700781ae24713979cb0e90821d859 [sys.catalog]: This master's current role is: LEADER
I20260810 04:41:27.644656 31277 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8ee700781ae24713979cb0e90821d859 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8ee700781ae24713979cb0e90821d859. Latest consensus state: current_term: 1 leader_uuid: "8ee700781ae24713979cb0e90821d859" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8ee700781ae24713979cb0e90821d859" member_type: VOTER last_known_addr { host: "127.22.167.126" port: 43457 } } }
I20260810 04:41:27.644727 31277 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8ee700781ae24713979cb0e90821d859 [sys.catalog]: This master's current role is: LEADER
I20260810 04:41:27.644941 31291 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 04:41:27.645455 31291 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 04:41:27.645772 23197 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskemv_AM/build/release/bin/kudu as pid 31141
I20260810 04:41:27.645871 23197 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786336863388938-23197-0/minicluster-data/master-0/wal/instance
I20260810 04:41:27.646019 23197 external_mini_cluster.cc:1428] Setting key c64351711db5318f5b36c05b721a7229
I20260810 04:41:27.648083 31316 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:41:27.646433 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43072 (local address 127.22.167.126:43457)
0810 04:41:27.646618 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0810 04:41:27.646627 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 04:41:27.646756 (+ 129us) server_negotiation.cc:407] Connection header received
0810 04:41:27.646860 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:41:27.646865 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:41:27.646898 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:41:27.647021 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0810 04:41:27.647667 (+ 646us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 04:41:27.647671 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 04:41:27.647671 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 04:41:27.647674 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0810 04:41:27.647717 (+ 43us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 04:41:27.647727 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 04:41:27.647787 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0810 04:41:27.647861 (+ 74us) server_negotiation.cc:299] Negotiation successful
0810 04:41:27.647904 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":118,"thread_start_us":96,"threads_started":1}
I20260810 04:41:27.650395 31291 catalog_manager.cc:1347] Generated new cluster ID: 5e5274bdcb094730acfa263de3b4015e
I20260810 04:41:27.650442 31291 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 04:41:27.657120 31291 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 04:41:27.657269 31291 catalog_manager.cc:1504] Loading token signing keys...
I20260810 04:41:27.661437 31291 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8ee700781ae24713979cb0e90821d859: Generated new TSK 0
I20260810 04:41:27.661595 31291 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 04:41:27.665658 23197 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskemv_AM/build/release/bin/kudu
/tmp/dist-test-taskemv_AM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786336863388938-23197-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786336863388938-23197-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786336863388938-23197-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786336863388938-23197-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.167.65:0
--local_ip_for_outbound_sockets=127.22.167.65
--webserver_interface=127.22.167.65
--webserver_port=0
--tserver_master_addrs=127.22.167.126:43457
--builtin_ntp_servers=127.22.167.84:43571
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260810 04:41:27.750979 31318 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 04:41:27.751169 31318 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 04:41:27.751190 31318 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 04:41:27.752285 31318 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 04:41:27.752321 31318 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 04:41:27.752345 31318 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.167.65
I20260810 04:41:27.753610 31318 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.167.84:43571
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786336863388938-23197-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786336863388938-23197-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.167.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786336863388938-23197-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.22.167.65
--webserver_port=0
--tserver_master_addrs=127.22.167.126:43457
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31318
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.167.65
--log_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786336863388938-23197-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 03:54:01 UTC on ccycloud.quasar-icijmt.root.comops.site
build id 4577009
I20260810 04:41:27.753805 31318 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 04:41:27.754076 31318 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 04:41:27.758692 31324 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 04:41:27.758859 31318 server_base.cc:1034] running on GCE node
W20260810 04:41:27.758905 31328 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 04:41:27.758714 31325 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 04:41:27.759137 31318 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 04:41:27.759353 31318 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 04:41:27.760493 31318 hybrid_clock.cc:648] HybridClock initialized: now 1786336887760482 us; error 30 us; skew 500 ppm
I20260810 04:41:27.761708 31318 webserver.cc:460] Webserver started at http://127.22.167.65:41993/ using document root <none> and password file <none>
I20260810 04:41:27.761906 31318 fs_manager.cc:359] Metadata directory not provided
I20260810 04:41:27.761952 31318 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 04:41:27.762089 31318 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 04:41:27.763077 31318 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786336863388938-23197-0/minicluster-data/ts-0/data/instance:
uuid: "2e8cb4b0ce654df8b3f1e6a4127fc566"
format_stamp: "Formatted at 2026-08-10 04:41:27 on dist-test-slave-bqcl"
server_key: "2b94bdd01eeeb37a170a4f2963915941"
server_key_iv: "1c6770020580a9268377f4a7557076d2"
server_key_version: "encryptionkey@0"
I20260810 04:41:27.763396 31318 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786336863388938-23197-0/minicluster-data/ts-0/wal/instance:
uuid: "2e8cb4b0ce654df8b3f1e6a4127fc566"
format_stamp: "Formatted at 2026-08-10 04:41:27 on dist-test-slave-bqcl"
server_key: "2b94bdd01eeeb37a170a4f2963915941"
server_key_iv: "1c6770020580a9268377f4a7557076d2"
server_key_version: "encryptionkey@0"
I20260810 04:41:27.764570 31318 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260810 04:41:27.765264 31339 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 04:41:27.765439 31318 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260810 04:41:27.765548 31318 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786336863388938-23197-0/minicluster-data/ts-0/data,/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786336863388938-23197-0/minicluster-data/ts-0/wal
uuid: "2e8cb4b0ce654df8b3f1e6a4127fc566"
format_stamp: "Formatted at 2026-08-10 04:41:27 on dist-test-slave-bqcl"
server_key: "2b94bdd01eeeb37a170a4f2963915941"
server_key_iv: "1c6770020580a9268377f4a7557076d2"
server_key_version: "encryptionkey@0"
I20260810 04:41:27.765628 31318 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786336863388938-23197-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786336863388938-23197-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786336863388938-23197-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 04:41:27.769930 31318 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 04:41:27.770195 31318 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 04:41:27.770309 31318 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 04:41:27.770486 31318 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 04:41:27.770792 31318 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 04:41:27.770829 31318 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 04:41:27.770854 31318 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 04:41:27.770876 31318 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 04:41:27.777073 31318 rpc_server.cc:307] RPC server started. Bound to: 127.22.167.65:45891
I20260810 04:41:27.777143 31571 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.167.65:45891 every 8 connection(s)
I20260810 04:41:27.777407 31318 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786336863388938-23197-0/minicluster-data/ts-0/data/info.pb
I20260810 04:41:27.780042 31316 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:41:27.778027 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.167.65:56749 (local address 127.22.167.126:43457)
0810 04:41:27.778075 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0810 04:41:27.778077 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 04:41:27.778307 (+ 230us) server_negotiation.cc:407] Connection header received
0810 04:41:27.778756 (+ 449us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:41:27.778757 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:41:27.778765 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:41:27.778816 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0810 04:41:27.779818 (+ 1002us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 04:41:27.779820 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 04:41:27.779820 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 04:41:27.779825 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0810 04:41:27.779841 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 04:41:27.779845 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 04:41:27.779895 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0810 04:41:27.779963 (+ 68us) server_negotiation.cc:299] Negotiation successful
0810 04:41:27.779977 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260810 04:41:27.780228 31578 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:41:27.778012 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.126:43457 (local address 127.22.167.65:56749)
0810 04:41:27.778201 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0810 04:41:27.778232 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0810 04:41:27.778666 (+ 434us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 04:41:27.778879 (+ 213us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 04:41:27.778883 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 04:41:27.778892 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0810 04:41:27.779685 (+ 793us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 04:41:27.779685 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0810 04:41:27.779711 (+ 26us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 04:41:27.779736 (+ 25us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 04:41:27.779899 (+ 163us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 04:41:27.779904 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 04:41:27.779907 (+ 3us) client_negotiation.cc:769] Sending connection context
0810 04:41:27.779975 (+ 68us) client_negotiation.cc:240] Negotiation successful
0810 04:41:27.780033 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":80,"threads_started":1}
I20260810 04:41:27.780678 31573 heartbeater.cc:347] Connected to a master server at 127.22.167.126:43457
I20260810 04:41:27.780758 31573 heartbeater.cc:464] Registering TS with master...
I20260810 04:41:27.780893 31573 heartbeater.cc:511] Master 127.22.167.126:43457 requested a full tablet report, sending...
I20260810 04:41:27.781320 23197 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskemv_AM/build/release/bin/kudu as pid 31318
I20260810 04:41:27.781352 31203 ts_manager.cc:194] Registered new tserver with Master: 2e8cb4b0ce654df8b3f1e6a4127fc566 (127.22.167.65:45891)
I20260810 04:41:27.781401 23197 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786336863388938-23197-0/minicluster-data/ts-0/wal/instance
I20260810 04:41:27.781595 23197 external_mini_cluster.cc:1428] Setting key 01be97fa34c499503d20650349bb736b
I20260810 04:41:27.782382 23197 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskemv_AM/build/release/bin/kudu
/tmp/dist-test-taskemv_AM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786336863388938-23197-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786336863388938-23197-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786336863388938-23197-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786336863388938-23197-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.167.66:0
--local_ip_for_outbound_sockets=127.22.167.66
--webserver_interface=127.22.167.66
--webserver_port=0
--tserver_master_addrs=127.22.167.126:43457
--builtin_ntp_servers=127.22.167.84:43571
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260810 04:41:27.861508 31579 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 04:41:27.861702 31579 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 04:41:27.861733 31579 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 04:41:27.862880 31579 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 04:41:27.862918 31579 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 04:41:27.862951 31579 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.167.66
I20260810 04:41:27.864217 31579 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.167.84:43571
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786336863388938-23197-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786336863388938-23197-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.167.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786336863388938-23197-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.22.167.66
--webserver_port=0
--tserver_master_addrs=127.22.167.126:43457
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31579
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.167.66
--log_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786336863388938-23197-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 03:54:01 UTC on ccycloud.quasar-icijmt.root.comops.site
build id 4577009
I20260810 04:41:27.864432 31579 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 04:41:27.864692 31579 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260810 04:41:27.869433 31579 server_base.cc:1034] running on GCE node
W20260810 04:41:27.869395 31591 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 04:41:27.869421 31586 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 04:41:27.869405 31588 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 04:41:27.869817 31579 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 04:41:27.870085 31579 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 04:41:27.871241 31579 hybrid_clock.cc:648] HybridClock initialized: now 1786336887871230 us; error 31 us; skew 500 ppm
I20260810 04:41:27.872336 31579 webserver.cc:460] Webserver started at http://127.22.167.66:38185/ using document root <none> and password file <none>
I20260810 04:41:27.872553 31579 fs_manager.cc:359] Metadata directory not provided
I20260810 04:41:27.872602 31579 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 04:41:27.872738 31579 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 04:41:27.873677 31579 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786336863388938-23197-0/minicluster-data/ts-1/data/instance:
uuid: "12e2b7dfd7d94659951102f4927afcf3"
format_stamp: "Formatted at 2026-08-10 04:41:27 on dist-test-slave-bqcl"
server_key: "8462f28d1ef104121acfe9ad797e0931"
server_key_iv: "39fd0f15e6b336aa25fdd6d9c49af30b"
server_key_version: "encryptionkey@0"
I20260810 04:41:27.874007 31579 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786336863388938-23197-0/minicluster-data/ts-1/wal/instance:
uuid: "12e2b7dfd7d94659951102f4927afcf3"
format_stamp: "Formatted at 2026-08-10 04:41:27 on dist-test-slave-bqcl"
server_key: "8462f28d1ef104121acfe9ad797e0931"
server_key_iv: "39fd0f15e6b336aa25fdd6d9c49af30b"
server_key_version: "encryptionkey@0"
I20260810 04:41:27.875188 31579 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260810 04:41:27.875908 31599 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 04:41:27.876055 31579 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260810 04:41:27.876130 31579 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786336863388938-23197-0/minicluster-data/ts-1/data,/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786336863388938-23197-0/minicluster-data/ts-1/wal
uuid: "12e2b7dfd7d94659951102f4927afcf3"
format_stamp: "Formatted at 2026-08-10 04:41:27 on dist-test-slave-bqcl"
server_key: "8462f28d1ef104121acfe9ad797e0931"
server_key_iv: "39fd0f15e6b336aa25fdd6d9c49af30b"
server_key_version: "encryptionkey@0"
I20260810 04:41:27.876199 31579 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786336863388938-23197-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786336863388938-23197-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786336863388938-23197-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 04:41:27.879340 31579 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 04:41:27.879565 31579 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 04:41:27.879688 31579 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 04:41:27.879871 31579 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 04:41:27.880136 31579 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 04:41:27.880167 31579 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 04:41:27.880193 31579 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 04:41:27.880210 31579 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 04:41:27.885464 31579 rpc_server.cc:307] RPC server started. Bound to: 127.22.167.66:37223
I20260810 04:41:27.885525 31796 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.167.66:37223 every 8 connection(s)
I20260810 04:41:27.885794 31579 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786336863388938-23197-0/minicluster-data/ts-1/data/info.pb
I20260810 04:41:27.887238 23197 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskemv_AM/build/release/bin/kudu as pid 31579
I20260810 04:41:27.887328 23197 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786336863388938-23197-0/minicluster-data/ts-1/wal/instance
I20260810 04:41:27.887495 23197 external_mini_cluster.cc:1428] Setting key ae48d8a734db2e3830e5c3875354231b
I20260810 04:41:27.888353 23197 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskemv_AM/build/release/bin/kudu
/tmp/dist-test-taskemv_AM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786336863388938-23197-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786336863388938-23197-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786336863388938-23197-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786336863388938-23197-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.167.67:0
--local_ip_for_outbound_sockets=127.22.167.67
--webserver_interface=127.22.167.67
--webserver_port=0
--tserver_master_addrs=127.22.167.126:43457
--builtin_ntp_servers=127.22.167.84:43571
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260810 04:41:27.888636 31316 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:41:27.886421 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.167.66:57175 (local address 127.22.167.126:43457)
0810 04:41:27.886463 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0810 04:41:27.886464 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0810 04:41:27.886667 (+ 203us) server_negotiation.cc:407] Connection header received
0810 04:41:27.887087 (+ 420us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:41:27.887089 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:41:27.887096 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:41:27.887147 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0810 04:41:27.888346 (+ 1199us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 04:41:27.888347 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 04:41:27.888348 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 04:41:27.888352 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0810 04:41:27.888371 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 04:41:27.888375 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 04:41:27.888427 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0810 04:41:27.888568 (+ 141us) server_negotiation.cc:299] Negotiation successful
0810 04:41:27.888581 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260810 04:41:27.888830 31799 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:41:27.886393 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.126:43457 (local address 127.22.167.66:57175)
0810 04:41:27.886568 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0810 04:41:27.886597 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0810 04:41:27.886997 (+ 400us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 04:41:27.887208 (+ 211us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 04:41:27.887212 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 04:41:27.887222 (+ 10us) client_negotiation.cc:189] Negotiated authn=SASL
0810 04:41:27.888216 (+ 994us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 04:41:27.888216 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0810 04:41:27.888243 (+ 27us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 04:41:27.888269 (+ 26us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 04:41:27.888451 (+ 182us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 04:41:27.888461 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 04:41:27.888464 (+ 3us) client_negotiation.cc:769] Sending connection context
0810 04:41:27.888577 (+ 113us) client_negotiation.cc:240] Negotiation successful
0810 04:41:27.888638 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":84,"threads_started":1}
I20260810 04:41:27.890157 31797 heartbeater.cc:347] Connected to a master server at 127.22.167.126:43457
I20260810 04:41:27.890240 31797 heartbeater.cc:464] Registering TS with master...
I20260810 04:41:27.890377 31797 heartbeater.cc:511] Master 127.22.167.126:43457 requested a full tablet report, sending...
I20260810 04:41:27.890705 31203 ts_manager.cc:194] Registered new tserver with Master: 12e2b7dfd7d94659951102f4927afcf3 (127.22.167.66:37223)
W20260810 04:41:27.967886 31801 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 04:41:27.968068 31801 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 04:41:27.968088 31801 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 04:41:27.969236 31801 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 04:41:27.969273 31801 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 04:41:27.969297 31801 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.167.67
I20260810 04:41:27.970649 31801 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.167.84:43571
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786336863388938-23197-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786336863388938-23197-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.167.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786336863388938-23197-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.22.167.67
--webserver_port=0
--tserver_master_addrs=127.22.167.126:43457
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31801
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.167.67
--log_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786336863388938-23197-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 03:54:01 UTC on ccycloud.quasar-icijmt.root.comops.site
build id 4577009
I20260810 04:41:27.970829 31801 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 04:41:27.971055 31801 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 04:41:27.975667 31817 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 04:41:27.975703 31801 server_base.cc:1034] running on GCE node
W20260810 04:41:27.975764 31807 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 04:41:27.975675 31811 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 04:41:27.976053 31801 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 04:41:27.976253 31801 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 04:41:27.977391 31801 hybrid_clock.cc:648] HybridClock initialized: now 1786336887977378 us; error 31 us; skew 500 ppm
I20260810 04:41:27.978601 31801 webserver.cc:460] Webserver started at http://127.22.167.67:40099/ using document root <none> and password file <none>
I20260810 04:41:27.978817 31801 fs_manager.cc:359] Metadata directory not provided
I20260810 04:41:27.978860 31801 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 04:41:27.978986 31801 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 04:41:27.980034 31801 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786336863388938-23197-0/minicluster-data/ts-2/data/instance:
uuid: "34a6702627fa4d1d92ce8f3fd1e1f1af"
format_stamp: "Formatted at 2026-08-10 04:41:27 on dist-test-slave-bqcl"
server_key: "e7b855e1f8761d7ab04d15bb67bb15ba"
server_key_iv: "da7b2aa548b584ba316ae31c82cea91a"
server_key_version: "encryptionkey@0"
I20260810 04:41:27.980347 31801 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786336863388938-23197-0/minicluster-data/ts-2/wal/instance:
uuid: "34a6702627fa4d1d92ce8f3fd1e1f1af"
format_stamp: "Formatted at 2026-08-10 04:41:27 on dist-test-slave-bqcl"
server_key: "e7b855e1f8761d7ab04d15bb67bb15ba"
server_key_iv: "da7b2aa548b584ba316ae31c82cea91a"
server_key_version: "encryptionkey@0"
I20260810 04:41:27.981498 31801 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260810 04:41:27.982113 31830 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 04:41:27.982285 31801 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260810 04:41:27.982364 31801 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786336863388938-23197-0/minicluster-data/ts-2/data,/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786336863388938-23197-0/minicluster-data/ts-2/wal
uuid: "34a6702627fa4d1d92ce8f3fd1e1f1af"
format_stamp: "Formatted at 2026-08-10 04:41:27 on dist-test-slave-bqcl"
server_key: "e7b855e1f8761d7ab04d15bb67bb15ba"
server_key_iv: "da7b2aa548b584ba316ae31c82cea91a"
server_key_version: "encryptionkey@0"
I20260810 04:41:27.982429 31801 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786336863388938-23197-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786336863388938-23197-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786336863388938-23197-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 04:41:27.985575 31801 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 04:41:27.985810 31801 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 04:41:27.985925 31801 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 04:41:27.986117 31801 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 04:41:27.986421 31801 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 04:41:27.986454 31801 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 04:41:27.986479 31801 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 04:41:27.986496 31801 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 04:41:27.991650 31801 rpc_server.cc:307] RPC server started. Bound to: 127.22.167.67:42489
I20260810 04:41:27.991706 32023 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.167.67:42489 every 8 connection(s)
I20260810 04:41:27.991957 31801 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786336863388938-23197-0/minicluster-data/ts-2/data/info.pb
I20260810 04:41:27.993335 23197 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskemv_AM/build/release/bin/kudu as pid 31801
I20260810 04:41:27.993453 23197 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786336863388938-23197-0/minicluster-data/ts-2/wal/instance
I20260810 04:41:27.993687 23197 external_mini_cluster.cc:1428] Setting key cd927fcbd25c37509a673f914d913f90
I20260810 04:41:27.994767 31316 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:41:27.992560 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.167.67:60937 (local address 127.22.167.126:43457)
0810 04:41:27.992619 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0810 04:41:27.992621 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 04:41:27.992822 (+ 201us) server_negotiation.cc:407] Connection header received
0810 04:41:27.993256 (+ 434us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:41:27.993257 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:41:27.993264 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:41:27.993316 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0810 04:41:27.994538 (+ 1222us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 04:41:27.994540 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 04:41:27.994540 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 04:41:27.994545 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0810 04:41:27.994565 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 04:41:27.994570 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 04:41:27.994625 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0810 04:41:27.994700 (+ 75us) server_negotiation.cc:299] Negotiation successful
0810 04:41:27.994714 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260810 04:41:27.994926 32033 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:41:27.992521 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.126:43457 (local address 127.22.167.67:60937)
0810 04:41:27.992719 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0810 04:41:27.992750 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0810 04:41:27.993165 (+ 415us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 04:41:27.993378 (+ 213us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 04:41:27.993382 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 04:41:27.993392 (+ 10us) client_negotiation.cc:189] Negotiated authn=SASL
0810 04:41:27.994416 (+ 1024us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 04:41:27.994416 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0810 04:41:27.994444 (+ 28us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 04:41:27.994470 (+ 26us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 04:41:27.994639 (+ 169us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 04:41:27.994643 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 04:41:27.994646 (+ 3us) client_negotiation.cc:769] Sending connection context
0810 04:41:27.994704 (+ 58us) client_negotiation.cc:240] Negotiation successful
0810 04:41:27.994750 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":83,"threads_started":1}
I20260810 04:41:27.995352 32025 heartbeater.cc:347] Connected to a master server at 127.22.167.126:43457
I20260810 04:41:27.995424 32025 heartbeater.cc:464] Registering TS with master...
I20260810 04:41:27.995553 32025 heartbeater.cc:511] Master 127.22.167.126:43457 requested a full tablet report, sending...
I20260810 04:41:27.995913 31203 ts_manager.cc:194] Registered new tserver with Master: 34a6702627fa4d1d92ce8f3fd1e1f1af (127.22.167.67:42489)
I20260810 04:41:28.005292 23197 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260810 04:41:28.007694 31316 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:41:28.006607 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43078 (local address 127.22.167.126:43457)
0810 04:41:28.006668 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0810 04:41:28.006670 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 04:41:28.006734 (+ 64us) server_negotiation.cc:407] Connection header received
0810 04:41:28.006809 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:41:28.006810 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:41:28.006816 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:41:28.006880 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0810 04:41:28.007454 (+ 574us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 04:41:28.007455 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 04:41:28.007455 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 04:41:28.007460 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0810 04:41:28.007478 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 04:41:28.007482 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 04:41:28.007532 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0810 04:41:28.007606 (+ 74us) server_negotiation.cc:299] Negotiation successful
0810 04:41:28.007624 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260810 04:41:28.009127 31203 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:43078:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 04:41:28.010756 31203 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 04:41:28.014031 32045 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:41:28.012777 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55222 (local address 127.22.167.66:37223)
0810 04:41:28.012930 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0810 04:41:28.012934 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 04:41:28.012944 (+ 10us) server_negotiation.cc:407] Connection header received
0810 04:41:28.012969 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:41:28.012970 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:41:28.012977 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:41:28.013048 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0810 04:41:28.013643 (+ 595us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 04:41:28.013654 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 04:41:28.013654 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 04:41:28.013660 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0810 04:41:28.013698 (+ 38us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 04:41:28.013731 (+ 33us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 04:41:28.013804 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0810 04:41:28.013928 (+ 124us) server_negotiation.cc:299] Negotiation successful
0810 04:41:28.013946 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":121,"thread_start_us":82,"threads_started":1}
I20260810 04:41:28.014246 32044 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:41:28.012682 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.66:37223 (local address 127.0.0.1:55222)
0810 04:41:28.012820 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0810 04:41:28.012859 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0810 04:41:28.012913 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 04:41:28.013061 (+ 148us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 04:41:28.013068 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 04:41:28.013074 (+ 6us) client_negotiation.cc:189] Negotiated authn=SASL
0810 04:41:28.013536 (+ 462us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 04:41:28.013536 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0810 04:41:28.013548 (+ 12us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 04:41:28.013566 (+ 18us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 04:41:28.013812 (+ 246us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 04:41:28.013818 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 04:41:28.013821 (+ 3us) client_negotiation.cc:769] Sending connection context
0810 04:41:28.014166 (+ 345us) client_negotiation.cc:240] Negotiation successful
0810 04:41:28.014180 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":65,"threads_started":1}
I20260810 04:41:28.014863 32046 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:41:28.012897 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.65:45891 (local address 127.0.0.1:50984)
0810 04:41:28.013034 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0810 04:41:28.013043 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0810 04:41:28.013080 (+ 37us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 04:41:28.014057 (+ 977us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 04:41:28.014058 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 04:41:28.014060 (+ 2us) client_negotiation.cc:189] Negotiated authn=SASL
0810 04:41:28.014464 (+ 404us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 04:41:28.014464 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0810 04:41:28.014471 (+ 7us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 04:41:28.014484 (+ 13us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 04:41:28.014738 (+ 254us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 04:41:28.014739 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 04:41:28.014740 (+ 1us) client_negotiation.cc:769] Sending connection context
0810 04:41:28.014786 (+ 46us) client_negotiation.cc:240] Negotiation successful
0810 04:41:28.014797 (+ 11us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":82,"threads_started":1}
I20260810 04:41:28.014896 32055 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:41:28.013536 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50984 (local address 127.22.167.65:45891)
0810 04:41:28.013889 (+ 353us) server_negotiation.cc:206] Beginning negotiation
0810 04:41:28.013893 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 04:41:28.013904 (+ 11us) server_negotiation.cc:407] Connection header received
0810 04:41:28.013930 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:41:28.013932 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:41:28.013940 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:41:28.014056 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0810 04:41:28.014575 (+ 519us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 04:41:28.014585 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 04:41:28.014585 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 04:41:28.014590 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0810 04:41:28.014630 (+ 40us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 04:41:28.014664 (+ 34us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 04:41:28.014745 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0810 04:41:28.014792 (+ 47us) server_negotiation.cc:299] Negotiation successful
0810 04:41:28.014809 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":314,"thread_start_us":94,"threads_started":1}
I20260810 04:41:28.015008 32050 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:41:28.013323 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48500 (local address 127.22.167.67:42489)
0810 04:41:28.013758 (+ 435us) server_negotiation.cc:206] Beginning negotiation
0810 04:41:28.013762 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 04:41:28.013772 (+ 10us) server_negotiation.cc:407] Connection header received
0810 04:41:28.013794 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:41:28.013796 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:41:28.013804 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:41:28.013888 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0810 04:41:28.014657 (+ 769us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 04:41:28.014663 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 04:41:28.014664 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 04:41:28.014668 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0810 04:41:28.014699 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 04:41:28.014720 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 04:41:28.014903 (+ 183us) server_negotiation.cc:1035] Waiting for connection context
0810 04:41:28.014924 (+ 21us) server_negotiation.cc:299] Negotiation successful
0810 04:41:28.014938 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":401,"thread_start_us":74,"threads_started":1}
I20260810 04:41:28.015007 31688 tablet_service.cc:1467] Processing CreateTablet for tablet 944fcd35b63a45f0913739aee732e22a (DEFAULT_TABLE table=test-table [id=7f96c3fe1a844543a663b90ab80bbfe8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 04:41:28.015300 32048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:41:28.013075 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.67:42489 (local address 127.0.0.1:48500)
0810 04:41:28.013345 (+ 270us) negotiation.cc:107] Waiting for socket to connect
0810 04:41:28.013354 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0810 04:41:28.013398 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 04:41:28.013888 (+ 490us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 04:41:28.013888 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 04:41:28.013890 (+ 2us) client_negotiation.cc:189] Negotiated authn=SASL
0810 04:41:28.014590 (+ 700us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 04:41:28.014591 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0810 04:41:28.014596 (+ 5us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 04:41:28.014605 (+ 9us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 04:41:28.014793 (+ 188us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 04:41:28.014794 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 04:41:28.014794 (+ 0us) client_negotiation.cc:769] Sending connection context
0810 04:41:28.014819 (+ 25us) client_negotiation.cc:240] Negotiation successful
0810 04:41:28.014829 (+ 10us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":253,"thread_start_us":59,"threads_started":1}
I20260810 04:41:28.015394 31688 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 944fcd35b63a45f0913739aee732e22a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 04:41:28.015512 31444 tablet_service.cc:1467] Processing CreateTablet for tablet 944fcd35b63a45f0913739aee732e22a (DEFAULT_TABLE table=test-table [id=7f96c3fe1a844543a663b90ab80bbfe8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 04:41:28.015859 31444 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 944fcd35b63a45f0913739aee732e22a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 04:41:28.018431 31906 tablet_service.cc:1467] Processing CreateTablet for tablet 944fcd35b63a45f0913739aee732e22a (DEFAULT_TABLE table=test-table [id=7f96c3fe1a844543a663b90ab80bbfe8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 04:41:28.018703 31906 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 944fcd35b63a45f0913739aee732e22a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 04:41:28.018851 32057 tablet_bootstrap.cc:492] T 944fcd35b63a45f0913739aee732e22a P 12e2b7dfd7d94659951102f4927afcf3: Bootstrap starting.
I20260810 04:41:28.019191 32059 tablet_bootstrap.cc:492] T 944fcd35b63a45f0913739aee732e22a P 2e8cb4b0ce654df8b3f1e6a4127fc566: Bootstrap starting.
I20260810 04:41:28.019520 32057 tablet_bootstrap.cc:654] T 944fcd35b63a45f0913739aee732e22a P 12e2b7dfd7d94659951102f4927afcf3: Neither blocks nor log segments found. Creating new log.
I20260810 04:41:28.019779 32059 tablet_bootstrap.cc:654] T 944fcd35b63a45f0913739aee732e22a P 2e8cb4b0ce654df8b3f1e6a4127fc566: Neither blocks nor log segments found. Creating new log.
I20260810 04:41:28.019846 32057 log.cc:824] T 944fcd35b63a45f0913739aee732e22a P 12e2b7dfd7d94659951102f4927afcf3: Log is configured to *not* fsync() on all Append() calls
I20260810 04:41:28.020098 32059 log.cc:824] T 944fcd35b63a45f0913739aee732e22a P 2e8cb4b0ce654df8b3f1e6a4127fc566: Log is configured to *not* fsync() on all Append() calls
I20260810 04:41:28.020673 32057 tablet_bootstrap.cc:492] T 944fcd35b63a45f0913739aee732e22a P 12e2b7dfd7d94659951102f4927afcf3: No bootstrap required, opened a new log
I20260810 04:41:28.020752 32057 ts_tablet_manager.cc:1397] T 944fcd35b63a45f0913739aee732e22a P 12e2b7dfd7d94659951102f4927afcf3: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260810 04:41:28.021030 32059 tablet_bootstrap.cc:492] T 944fcd35b63a45f0913739aee732e22a P 2e8cb4b0ce654df8b3f1e6a4127fc566: No bootstrap required, opened a new log
I20260810 04:41:28.021158 32059 ts_tablet_manager.cc:1397] T 944fcd35b63a45f0913739aee732e22a P 2e8cb4b0ce654df8b3f1e6a4127fc566: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260810 04:41:28.021590 32063 tablet_bootstrap.cc:492] T 944fcd35b63a45f0913739aee732e22a P 34a6702627fa4d1d92ce8f3fd1e1f1af: Bootstrap starting.
I20260810 04:41:28.022065 32063 tablet_bootstrap.cc:654] T 944fcd35b63a45f0913739aee732e22a P 34a6702627fa4d1d92ce8f3fd1e1f1af: Neither blocks nor log segments found. Creating new log.
I20260810 04:41:28.022348 32063 log.cc:824] T 944fcd35b63a45f0913739aee732e22a P 34a6702627fa4d1d92ce8f3fd1e1f1af: Log is configured to *not* fsync() on all Append() calls
I20260810 04:41:28.022917 32063 tablet_bootstrap.cc:492] T 944fcd35b63a45f0913739aee732e22a P 34a6702627fa4d1d92ce8f3fd1e1f1af: No bootstrap required, opened a new log
I20260810 04:41:28.022972 32063 ts_tablet_manager.cc:1397] T 944fcd35b63a45f0913739aee732e22a P 34a6702627fa4d1d92ce8f3fd1e1f1af: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260810 04:41:28.023363 32057 raft_consensus.cc:348] T 944fcd35b63a45f0913739aee732e22a P 12e2b7dfd7d94659951102f4927afcf3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12e2b7dfd7d94659951102f4927afcf3" member_type: VOTER last_known_addr { host: "127.22.167.66" port: 37223 } } peers { permanent_uuid: "2e8cb4b0ce654df8b3f1e6a4127fc566" member_type: VOTER last_known_addr { host: "127.22.167.65" port: 45891 } } peers { permanent_uuid: "34a6702627fa4d1d92ce8f3fd1e1f1af" member_type: VOTER last_known_addr { host: "127.22.167.67" port: 42489 } }
I20260810 04:41:28.023554 32057 raft_consensus.cc:374] T 944fcd35b63a45f0913739aee732e22a P 12e2b7dfd7d94659951102f4927afcf3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 04:41:28.023602 32057 raft_consensus.cc:729] T 944fcd35b63a45f0913739aee732e22a P 12e2b7dfd7d94659951102f4927afcf3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 12e2b7dfd7d94659951102f4927afcf3, State: Initialized, Role: FOLLOWER
I20260810 04:41:28.023728 32057 consensus_queue.cc:260] T 944fcd35b63a45f0913739aee732e22a P 12e2b7dfd7d94659951102f4927afcf3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12e2b7dfd7d94659951102f4927afcf3" member_type: VOTER last_known_addr { host: "127.22.167.66" port: 37223 } } peers { permanent_uuid: "2e8cb4b0ce654df8b3f1e6a4127fc566" member_type: VOTER last_known_addr { host: "127.22.167.65" port: 45891 } } peers { permanent_uuid: "34a6702627fa4d1d92ce8f3fd1e1f1af" member_type: VOTER last_known_addr { host: "127.22.167.67" port: 42489 } }
I20260810 04:41:28.023726 32059 raft_consensus.cc:348] T 944fcd35b63a45f0913739aee732e22a P 2e8cb4b0ce654df8b3f1e6a4127fc566 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12e2b7dfd7d94659951102f4927afcf3" member_type: VOTER last_known_addr { host: "127.22.167.66" port: 37223 } } peers { permanent_uuid: "2e8cb4b0ce654df8b3f1e6a4127fc566" member_type: VOTER last_known_addr { host: "127.22.167.65" port: 45891 } } peers { permanent_uuid: "34a6702627fa4d1d92ce8f3fd1e1f1af" member_type: VOTER last_known_addr { host: "127.22.167.67" port: 42489 } }
I20260810 04:41:28.023872 32059 raft_consensus.cc:374] T 944fcd35b63a45f0913739aee732e22a P 2e8cb4b0ce654df8b3f1e6a4127fc566 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 04:41:28.023912 32059 raft_consensus.cc:729] T 944fcd35b63a45f0913739aee732e22a P 2e8cb4b0ce654df8b3f1e6a4127fc566 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2e8cb4b0ce654df8b3f1e6a4127fc566, State: Initialized, Role: FOLLOWER
I20260810 04:41:28.024004 32057 ts_tablet_manager.cc:1428] T 944fcd35b63a45f0913739aee732e22a P 12e2b7dfd7d94659951102f4927afcf3: Time spent starting tablet: real 0.003s user 0.004s sys 0.000s
I20260810 04:41:28.024063 32059 consensus_queue.cc:260] T 944fcd35b63a45f0913739aee732e22a P 2e8cb4b0ce654df8b3f1e6a4127fc566 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12e2b7dfd7d94659951102f4927afcf3" member_type: VOTER last_known_addr { host: "127.22.167.66" port: 37223 } } peers { permanent_uuid: "2e8cb4b0ce654df8b3f1e6a4127fc566" member_type: VOTER last_known_addr { host: "127.22.167.65" port: 45891 } } peers { permanent_uuid: "34a6702627fa4d1d92ce8f3fd1e1f1af" member_type: VOTER last_known_addr { host: "127.22.167.67" port: 42489 } }
I20260810 04:41:28.024120 31797 heartbeater.cc:503] Master 127.22.167.126:43457 was elected leader, sending a full tablet report...
I20260810 04:41:28.024312 32059 ts_tablet_manager.cc:1428] T 944fcd35b63a45f0913739aee732e22a P 2e8cb4b0ce654df8b3f1e6a4127fc566: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260810 04:41:28.024425 31573 heartbeater.cc:503] Master 127.22.167.126:43457 was elected leader, sending a full tablet report...
I20260810 04:41:28.025198 32063 raft_consensus.cc:348] T 944fcd35b63a45f0913739aee732e22a P 34a6702627fa4d1d92ce8f3fd1e1f1af [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12e2b7dfd7d94659951102f4927afcf3" member_type: VOTER last_known_addr { host: "127.22.167.66" port: 37223 } } peers { permanent_uuid: "2e8cb4b0ce654df8b3f1e6a4127fc566" member_type: VOTER last_known_addr { host: "127.22.167.65" port: 45891 } } peers { permanent_uuid: "34a6702627fa4d1d92ce8f3fd1e1f1af" member_type: VOTER last_known_addr { host: "127.22.167.67" port: 42489 } }
I20260810 04:41:28.025359 32063 raft_consensus.cc:374] T 944fcd35b63a45f0913739aee732e22a P 34a6702627fa4d1d92ce8f3fd1e1f1af [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 04:41:28.025404 32063 raft_consensus.cc:729] T 944fcd35b63a45f0913739aee732e22a P 34a6702627fa4d1d92ce8f3fd1e1f1af [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 34a6702627fa4d1d92ce8f3fd1e1f1af, State: Initialized, Role: FOLLOWER
I20260810 04:41:28.025508 32063 consensus_queue.cc:260] T 944fcd35b63a45f0913739aee732e22a P 34a6702627fa4d1d92ce8f3fd1e1f1af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12e2b7dfd7d94659951102f4927afcf3" member_type: VOTER last_known_addr { host: "127.22.167.66" port: 37223 } } peers { permanent_uuid: "2e8cb4b0ce654df8b3f1e6a4127fc566" member_type: VOTER last_known_addr { host: "127.22.167.65" port: 45891 } } peers { permanent_uuid: "34a6702627fa4d1d92ce8f3fd1e1f1af" member_type: VOTER last_known_addr { host: "127.22.167.67" port: 42489 } }
I20260810 04:41:28.025714 32063 ts_tablet_manager.cc:1428] T 944fcd35b63a45f0913739aee732e22a P 34a6702627fa4d1d92ce8f3fd1e1f1af: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260810 04:41:28.025755 32025 heartbeater.cc:503] Master 127.22.167.126:43457 was elected leader, sending a full tablet report...
W20260810 04:41:28.027876 31575 tablet.cc:2369] T 944fcd35b63a45f0913739aee732e22a P 2e8cb4b0ce654df8b3f1e6a4127fc566: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 04:41:28.091802 32067 raft_consensus.cc:482] T 944fcd35b63a45f0913739aee732e22a P 12e2b7dfd7d94659951102f4927afcf3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 04:41:28.091922 32067 raft_consensus.cc:504] T 944fcd35b63a45f0913739aee732e22a P 12e2b7dfd7d94659951102f4927afcf3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12e2b7dfd7d94659951102f4927afcf3" member_type: VOTER last_known_addr { host: "127.22.167.66" port: 37223 } } peers { permanent_uuid: "2e8cb4b0ce654df8b3f1e6a4127fc566" member_type: VOTER last_known_addr { host: "127.22.167.65" port: 45891 } } peers { permanent_uuid: "34a6702627fa4d1d92ce8f3fd1e1f1af" member_type: VOTER last_known_addr { host: "127.22.167.67" port: 42489 } }
I20260810 04:41:28.092229 32067 leader_election.cc:288] T 944fcd35b63a45f0913739aee732e22a P 12e2b7dfd7d94659951102f4927afcf3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2e8cb4b0ce654df8b3f1e6a4127fc566 (127.22.167.65:45891), 34a6702627fa4d1d92ce8f3fd1e1f1af (127.22.167.67:42489)
I20260810 04:41:28.093412 32055 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:41:28.092504 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.167.66:53573 (local address 127.22.167.65:45891)
0810 04:41:28.092555 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0810 04:41:28.092557 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 04:41:28.092584 (+ 27us) server_negotiation.cc:407] Connection header received
0810 04:41:28.092611 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:41:28.092612 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:41:28.092618 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:41:28.092676 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0810 04:41:28.093185 (+ 509us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 04:41:28.093186 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 04:41:28.093186 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 04:41:28.093190 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0810 04:41:28.093206 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 04:41:28.093210 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 04:41:28.093265 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0810 04:41:28.093329 (+ 64us) server_negotiation.cc:299] Negotiation successful
0810 04:41:28.093340 (+ 11us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260810 04:41:28.093430 31799 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:41:28.092402 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.65:45891 (local address 127.22.167.66:53573)
0810 04:41:28.092458 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0810 04:41:28.092470 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0810 04:41:28.092521 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 04:41:28.092681 (+ 160us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 04:41:28.092682 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 04:41:28.092685 (+ 3us) client_negotiation.cc:189] Negotiated authn=SASL
0810 04:41:28.093078 (+ 393us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 04:41:28.093079 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0810 04:41:28.093085 (+ 6us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 04:41:28.093098 (+ 13us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 04:41:28.093271 (+ 173us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 04:41:28.093272 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 04:41:28.093273 (+ 1us) client_negotiation.cc:769] Sending connection context
0810 04:41:28.093323 (+ 50us) client_negotiation.cc:240] Negotiation successful
0810 04:41:28.093338 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":33}
I20260810 04:41:28.093449 32050 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:41:28.092594 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.167.66:57883 (local address 127.22.167.67:42489)
0810 04:41:28.092650 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0810 04:41:28.092652 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 04:41:28.092734 (+ 82us) server_negotiation.cc:407] Connection header received
0810 04:41:28.092776 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:41:28.092777 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:41:28.092783 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:41:28.092834 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0810 04:41:28.093249 (+ 415us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 04:41:28.093250 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 04:41:28.093250 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 04:41:28.093255 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0810 04:41:28.093271 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 04:41:28.093276 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 04:41:28.093323 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0810 04:41:28.093368 (+ 45us) server_negotiation.cc:299] Negotiation successful
0810 04:41:28.093380 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260810 04:41:28.093672 32073 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:41:28.092501 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.67:42489 (local address 127.22.167.66:57883)
0810 04:41:28.092671 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0810 04:41:28.092680 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0810 04:41:28.092729 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 04:41:28.092846 (+ 117us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 04:41:28.092847 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 04:41:28.092849 (+ 2us) client_negotiation.cc:189] Negotiated authn=SASL
0810 04:41:28.093177 (+ 328us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 04:41:28.093178 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0810 04:41:28.093183 (+ 5us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 04:41:28.093192 (+ 9us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 04:41:28.093318 (+ 126us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 04:41:28.093318 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 04:41:28.093319 (+ 1us) client_negotiation.cc:769] Sending connection context
0810 04:41:28.093367 (+ 48us) client_negotiation.cc:240] Negotiation successful
0810 04:41:28.093380 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":155,"thread_start_us":110,"threads_started":1}
I20260810 04:41:28.093739 31483 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "944fcd35b63a45f0913739aee732e22a" candidate_uuid: "12e2b7dfd7d94659951102f4927afcf3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2e8cb4b0ce654df8b3f1e6a4127fc566" is_pre_election: true
I20260810 04:41:28.093899 31483 raft_consensus.cc:2454] T 944fcd35b63a45f0913739aee732e22a P 2e8cb4b0ce654df8b3f1e6a4127fc566 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12e2b7dfd7d94659951102f4927afcf3 in term 0.
I20260810 04:41:28.094174 31607 leader_election.cc:302] T 944fcd35b63a45f0913739aee732e22a P 12e2b7dfd7d94659951102f4927afcf3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12e2b7dfd7d94659951102f4927afcf3, 2e8cb4b0ce654df8b3f1e6a4127fc566; no voters:
I20260810 04:41:28.094251 31939 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "944fcd35b63a45f0913739aee732e22a" candidate_uuid: "12e2b7dfd7d94659951102f4927afcf3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "34a6702627fa4d1d92ce8f3fd1e1f1af" is_pre_election: true
I20260810 04:41:28.094311 32067 raft_consensus.cc:2781] T 944fcd35b63a45f0913739aee732e22a P 12e2b7dfd7d94659951102f4927afcf3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 04:41:28.094377 32067 raft_consensus.cc:482] T 944fcd35b63a45f0913739aee732e22a P 12e2b7dfd7d94659951102f4927afcf3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 04:41:28.094405 32067 raft_consensus.cc:3037] T 944fcd35b63a45f0913739aee732e22a P 12e2b7dfd7d94659951102f4927afcf3 [term 0 FOLLOWER]: Advancing to term 1
I20260810 04:41:28.094412 31939 raft_consensus.cc:2454] T 944fcd35b63a45f0913739aee732e22a P 34a6702627fa4d1d92ce8f3fd1e1f1af [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12e2b7dfd7d94659951102f4927afcf3 in term 0.
I20260810 04:41:28.095155 32067 raft_consensus.cc:504] T 944fcd35b63a45f0913739aee732e22a P 12e2b7dfd7d94659951102f4927afcf3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12e2b7dfd7d94659951102f4927afcf3" member_type: VOTER last_known_addr { host: "127.22.167.66" port: 37223 } } peers { permanent_uuid: "2e8cb4b0ce654df8b3f1e6a4127fc566" member_type: VOTER last_known_addr { host: "127.22.167.65" port: 45891 } } peers { permanent_uuid: "34a6702627fa4d1d92ce8f3fd1e1f1af" member_type: VOTER last_known_addr { host: "127.22.167.67" port: 42489 } }
I20260810 04:41:28.095255 32067 leader_election.cc:288] T 944fcd35b63a45f0913739aee732e22a P 12e2b7dfd7d94659951102f4927afcf3 [CANDIDATE]: Term 1 election: Requested vote from peers 2e8cb4b0ce654df8b3f1e6a4127fc566 (127.22.167.65:45891), 34a6702627fa4d1d92ce8f3fd1e1f1af (127.22.167.67:42489)
I20260810 04:41:28.095404 31483 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "944fcd35b63a45f0913739aee732e22a" candidate_uuid: "12e2b7dfd7d94659951102f4927afcf3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2e8cb4b0ce654df8b3f1e6a4127fc566"
I20260810 04:41:28.095467 31939 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "944fcd35b63a45f0913739aee732e22a" candidate_uuid: "12e2b7dfd7d94659951102f4927afcf3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "34a6702627fa4d1d92ce8f3fd1e1f1af"
I20260810 04:41:28.095481 31483 raft_consensus.cc:3037] T 944fcd35b63a45f0913739aee732e22a P 2e8cb4b0ce654df8b3f1e6a4127fc566 [term 0 FOLLOWER]: Advancing to term 1
I20260810 04:41:28.095554 31939 raft_consensus.cc:3037] T 944fcd35b63a45f0913739aee732e22a P 34a6702627fa4d1d92ce8f3fd1e1f1af [term 0 FOLLOWER]: Advancing to term 1
I20260810 04:41:28.096197 31939 raft_consensus.cc:2454] T 944fcd35b63a45f0913739aee732e22a P 34a6702627fa4d1d92ce8f3fd1e1f1af [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12e2b7dfd7d94659951102f4927afcf3 in term 1.
I20260810 04:41:28.096199 31483 raft_consensus.cc:2454] T 944fcd35b63a45f0913739aee732e22a P 2e8cb4b0ce654df8b3f1e6a4127fc566 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12e2b7dfd7d94659951102f4927afcf3 in term 1.
I20260810 04:41:28.096336 31607 leader_election.cc:302] T 944fcd35b63a45f0913739aee732e22a P 12e2b7dfd7d94659951102f4927afcf3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12e2b7dfd7d94659951102f4927afcf3, 2e8cb4b0ce654df8b3f1e6a4127fc566; no voters:
I20260810 04:41:28.096422 32067 raft_consensus.cc:2781] T 944fcd35b63a45f0913739aee732e22a P 12e2b7dfd7d94659951102f4927afcf3 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 04:41:28.096578 32067 raft_consensus.cc:686] T 944fcd35b63a45f0913739aee732e22a P 12e2b7dfd7d94659951102f4927afcf3 [term 1 LEADER]: Becoming Leader. State: Replica: 12e2b7dfd7d94659951102f4927afcf3, State: Running, Role: LEADER
I20260810 04:41:28.096664 32067 consensus_queue.cc:237] T 944fcd35b63a45f0913739aee732e22a P 12e2b7dfd7d94659951102f4927afcf3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12e2b7dfd7d94659951102f4927afcf3" member_type: VOTER last_known_addr { host: "127.22.167.66" port: 37223 } } peers { permanent_uuid: "2e8cb4b0ce654df8b3f1e6a4127fc566" member_type: VOTER last_known_addr { host: "127.22.167.65" port: 45891 } } peers { permanent_uuid: "34a6702627fa4d1d92ce8f3fd1e1f1af" member_type: VOTER last_known_addr { host: "127.22.167.67" port: 42489 } }
I20260810 04:41:28.097426 31203 catalog_manager.cc:5495] T 944fcd35b63a45f0913739aee732e22a P 12e2b7dfd7d94659951102f4927afcf3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 12e2b7dfd7d94659951102f4927afcf3 (127.22.167.66). New cstate: current_term: 1 leader_uuid: "12e2b7dfd7d94659951102f4927afcf3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12e2b7dfd7d94659951102f4927afcf3" member_type: VOTER last_known_addr { host: "127.22.167.66" port: 37223 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2e8cb4b0ce654df8b3f1e6a4127fc566" member_type: VOTER last_known_addr { host: "127.22.167.65" port: 45891 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "34a6702627fa4d1d92ce8f3fd1e1f1af" member_type: VOTER last_known_addr { host: "127.22.167.67" port: 42489 } health_report { overall_health: UNKNOWN } } }
I20260810 04:41:28.105579 32045 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:41:28.104764 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55238 (local address 127.22.167.66:37223)
0810 04:41:28.104803 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0810 04:41:28.104805 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 04:41:28.104812 (+ 7us) server_negotiation.cc:407] Connection header received
0810 04:41:28.104863 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:41:28.104864 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:41:28.104874 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:41:28.104914 (+ 40us) server_negotiation.cc:226] Negotiated authn=SASL
0810 04:41:28.105404 (+ 490us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 04:41:28.105406 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 04:41:28.105406 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 04:41:28.105410 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0810 04:41:28.105421 (+ 11us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 04:41:28.105424 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 04:41:28.105459 (+ 35us) server_negotiation.cc:1035] Waiting for connection context
0810 04:41:28.105494 (+ 35us) server_negotiation.cc:299] Negotiation successful
0810 04:41:28.105504 (+ 10us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260810 04:41:28.107132 31483 raft_consensus.cc:1261] T 944fcd35b63a45f0913739aee732e22a P 2e8cb4b0ce654df8b3f1e6a4127fc566 [term 1 FOLLOWER]: Refusing update from remote peer 12e2b7dfd7d94659951102f4927afcf3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 04:41:28.107144 31939 raft_consensus.cc:1261] T 944fcd35b63a45f0913739aee732e22a P 34a6702627fa4d1d92ce8f3fd1e1f1af [term 1 FOLLOWER]: Refusing update from remote peer 12e2b7dfd7d94659951102f4927afcf3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 04:41:28.107421 32075 consensus_queue.cc:1035] T 944fcd35b63a45f0913739aee732e22a P 12e2b7dfd7d94659951102f4927afcf3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2e8cb4b0ce654df8b3f1e6a4127fc566" member_type: VOTER last_known_addr { host: "127.22.167.65" port: 45891 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 04:41:28.107542 32067 consensus_queue.cc:1035] T 944fcd35b63a45f0913739aee732e22a P 12e2b7dfd7d94659951102f4927afcf3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34a6702627fa4d1d92ce8f3fd1e1f1af" member_type: VOTER last_known_addr { host: "127.22.167.67" port: 42489 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 04:41:28.110666 32081 mvcc.cc:205] Tried to move back new op lower bound from 7316835893684703232 to 7316835893644566528. Current Snapshot: MvccSnapshot[applied={T|T < 7316835893684703232}]
I20260810 04:41:28.111971 31203 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:43078:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 04:41:28.112057 31203 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:43078:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 04:41:28.112718 31203 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 8ee700781ae24713979cb0e90821d859: Sending DeleteTablet for 3 replicas of tablet 944fcd35b63a45f0913739aee732e22a
I20260810 04:41:28.113054 31688 tablet_service.cc:1514] Processing DeleteTablet for tablet 944fcd35b63a45f0913739aee732e22a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 04:41:28 UTC) from {username='slave'} at 127.0.0.1:55222
I20260810 04:41:28.113117 31444 tablet_service.cc:1514] Processing DeleteTablet for tablet 944fcd35b63a45f0913739aee732e22a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 04:41:28 UTC) from {username='slave'} at 127.0.0.1:50984
I20260810 04:41:28.113159 31906 tablet_service.cc:1514] Processing DeleteTablet for tablet 944fcd35b63a45f0913739aee732e22a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 04:41:28 UTC) from {username='slave'} at 127.0.0.1:48500
I20260810 04:41:28.113296 32093 tablet_replica.cc:330] T 944fcd35b63a45f0913739aee732e22a P 2e8cb4b0ce654df8b3f1e6a4127fc566: stopping tablet replica
I20260810 04:41:28.113390 32093 raft_consensus.cc:2229] T 944fcd35b63a45f0913739aee732e22a P 2e8cb4b0ce654df8b3f1e6a4127fc566 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 04:41:28.113410 32092 tablet_replica.cc:330] T 944fcd35b63a45f0913739aee732e22a P 12e2b7dfd7d94659951102f4927afcf3: stopping tablet replica
I20260810 04:41:28.113494 32093 raft_consensus.cc:2258] T 944fcd35b63a45f0913739aee732e22a P 2e8cb4b0ce654df8b3f1e6a4127fc566 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 04:41:28.113580 23197 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskemv_AM/build/release/bin/kudu with pid 31318
I20260810 04:41:28.113619 32092 raft_consensus.cc:2229] T 944fcd35b63a45f0913739aee732e22a P 12e2b7dfd7d94659951102f4927afcf3 [term 1 LEADER]: Raft consensus shutting down.
I20260810 04:41:28.113781 32092 raft_consensus.cc:2258] T 944fcd35b63a45f0913739aee732e22a P 12e2b7dfd7d94659951102f4927afcf3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 04:41:28.114368 32092 ts_tablet_manager.cc:1905] T 944fcd35b63a45f0913739aee732e22a P 12e2b7dfd7d94659951102f4927afcf3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 04:41:28.115684 32092 ts_tablet_manager.cc:1918] T 944fcd35b63a45f0913739aee732e22a P 12e2b7dfd7d94659951102f4927afcf3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 04:41:28.115747 32092 log.cc:1196] T 944fcd35b63a45f0913739aee732e22a P 12e2b7dfd7d94659951102f4927afcf3: Deleting WAL directory at /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786336863388938-23197-0/minicluster-data/ts-1/wal/wals/944fcd35b63a45f0913739aee732e22a
I20260810 04:41:28.115974 32092 ts_tablet_manager.cc:1939] T 944fcd35b63a45f0913739aee732e22a P 12e2b7dfd7d94659951102f4927afcf3: Deleting consensus metadata
I20260810 04:41:28.116322 31184 catalog_manager.cc:4841] TS 12e2b7dfd7d94659951102f4927afcf3 (127.22.167.66:37223): tablet 944fcd35b63a45f0913739aee732e22a (table test-table [id=7f96c3fe1a844543a663b90ab80bbfe8]) successfully deleted
W20260810 04:41:28.127086 31184 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.22.167.65:45891 (error 108)
I20260810 04:41:28.127324 32094 tablet_replica.cc:330] T 944fcd35b63a45f0913739aee732e22a P 34a6702627fa4d1d92ce8f3fd1e1f1af: stopping tablet replica
I20260810 04:41:28.127506 23197 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskemv_AM/build/release/bin/kudu with pid 31579
I20260810 04:41:28.127538 32094 raft_consensus.cc:2229] T 944fcd35b63a45f0913739aee732e22a P 34a6702627fa4d1d92ce8f3fd1e1f1af [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 04:41:28.130224 32048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:41:28.129089 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.65:45891 (local address 127.0.0.1:50992)
0810 04:41:28.130142 (+ 1053us) negotiation.cc:107] Waiting for socket to connect
0810 04:41:28.130189 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.167.65:45891: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1011}
W20260810 04:41:28.130337 31184 catalog_manager.cc:4584] TS 2e8cb4b0ce654df8b3f1e6a4127fc566 (127.22.167.65:45891): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 944fcd35b63a45f0913739aee732e22a: Network error: Client connection negotiation failed: client connection to 127.22.167.65:45891: connect: Connection refused (error 111)
I20260810 04:41:28.135571 32094 raft_consensus.cc:2258] T 944fcd35b63a45f0913739aee732e22a P 34a6702627fa4d1d92ce8f3fd1e1f1af [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 04:41:28.136029 32094 ts_tablet_manager.cc:1905] T 944fcd35b63a45f0913739aee732e22a P 34a6702627fa4d1d92ce8f3fd1e1f1af: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 04:41:28.137445 32094 ts_tablet_manager.cc:1918] T 944fcd35b63a45f0913739aee732e22a P 34a6702627fa4d1d92ce8f3fd1e1f1af: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 04:41:28.137558 32094 log.cc:1196] T 944fcd35b63a45f0913739aee732e22a P 34a6702627fa4d1d92ce8f3fd1e1f1af: Deleting WAL directory at /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786336863388938-23197-0/minicluster-data/ts-2/wal/wals/944fcd35b63a45f0913739aee732e22a
I20260810 04:41:28.137809 32094 ts_tablet_manager.cc:1939] T 944fcd35b63a45f0913739aee732e22a P 34a6702627fa4d1d92ce8f3fd1e1f1af: Deleting consensus metadata
I20260810 04:41:28.138235 31184 catalog_manager.cc:4841] TS 34a6702627fa4d1d92ce8f3fd1e1f1af (127.22.167.67:42489): tablet 944fcd35b63a45f0913739aee732e22a (table test-table [id=7f96c3fe1a844543a663b90ab80bbfe8]) successfully deleted
I20260810 04:41:28.139707 23197 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskemv_AM/build/release/bin/kudu with pid 31801
I20260810 04:41:28.150561 23197 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskemv_AM/build/release/bin/kudu with pid 31141
2026-08-10T04:41:28Z chronyd exiting
[ OK ] SecurityITest.TestDisableAuthenticationEncryption (647 ms)
[ RUN ] SecurityITest.TestJwtMiniCluster
I20260810 04:41:28.171624 23197 mini_oidc.cc:152] Starting JWKS server
I20260810 04:41:28.172676 23197 webserver.cc:460] Webserver started at https://127.0.0.1:33925/ using document root <none> and password file <none>
I20260810 04:41:28.172750 23197 mini_oidc.cc:189] Starting OIDC Discovery server
I20260810 04:41:28.172876 23197 webserver.cc:460] Webserver started at http://127.0.0.1:42131/ using document root <none> and password file <none>
2026-08-10T04:41:28Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T04:41:28Z Disabled control of system clock
I20260810 04:41:28.180711 23197 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskemv_AM/build/release/bin/kudu
/tmp/dist-test-taskemv_AM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786336863388938-23197-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786336863388938-23197-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786336863388938-23197-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786336863388938-23197-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.167.126:44827
--webserver_interface=127.22.167.126
--webserver_port=0
--builtin_ntp_servers=127.22.167.84:40205
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.167.126:44827
--enable_jwt_token_auth=true
--jwks_url=https://localhost:33925/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786336863388938-23197-0/testchainca.cert with env {}
W20260810 04:41:28.258978 32105 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 04:41:28.259176 32105 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 04:41:28.259197 32105 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 04:41:28.260272 32105 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 04:41:28.260306 32105 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 04:41:28.260318 32105 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 04:41:28.260329 32105 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 04:41:28.260339 32105 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260810 04:41:28.260349 32105 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260810 04:41:28.260360 32105 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:33925/jwks.json
I20260810 04:41:28.261816 32105 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.167.84:40205
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786336863388938-23197-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786336863388938-23197-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.22.167.126:44827
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.167.126:44827
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:33925/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786336863388938-23197-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.167.126
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786336863388938-23197-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32105
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786336863388938-23197-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 03:54:01 UTC on ccycloud.quasar-icijmt.root.comops.site
build id 4577009
I20260810 04:41:28.261998 32105 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 04:41:28.262197 32105 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 04:41:28.266628 32113 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 04:41:28.266639 32123 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 04:41:28.266755 32114 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 04:41:28.266782 32105 server_base.cc:1034] running on GCE node
I20260810 04:41:28.267151 32105 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 04:41:28.267405 32105 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 04:41:28.268541 32105 hybrid_clock.cc:648] HybridClock initialized: now 1786336888268525 us; error 32 us; skew 500 ppm
I20260810 04:41:28.269744 32105 webserver.cc:460] Webserver started at http://127.22.167.126:34133/ using document root <none> and password file <none>
I20260810 04:41:28.269940 32105 fs_manager.cc:359] Metadata directory not provided
I20260810 04:41:28.269987 32105 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 04:41:28.270109 32105 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 04:41:28.271114 32105 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786336863388938-23197-0/minicluster-data/master-0/data/instance:
uuid: "52517878bff64958ae67ea7e999a7596"
format_stamp: "Formatted at 2026-08-10 04:41:28 on dist-test-slave-bqcl"
server_key: "965e4b700191b75ee31c62cb411ee4fa"
server_key_iv: "1a0a9fb3c4d7a9eeee2c598ede7fc23e"
server_key_version: "encryptionkey@0"
I20260810 04:41:28.271435 32105 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786336863388938-23197-0/minicluster-data/master-0/wal/instance:
uuid: "52517878bff64958ae67ea7e999a7596"
format_stamp: "Formatted at 2026-08-10 04:41:28 on dist-test-slave-bqcl"
server_key: "965e4b700191b75ee31c62cb411ee4fa"
server_key_iv: "1a0a9fb3c4d7a9eeee2c598ede7fc23e"
server_key_version: "encryptionkey@0"
I20260810 04:41:28.272576 32105 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260810 04:41:28.273186 32136 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 04:41:28.273357 32105 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260810 04:41:28.273442 32105 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786336863388938-23197-0/minicluster-data/master-0/data,/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786336863388938-23197-0/minicluster-data/master-0/wal
uuid: "52517878bff64958ae67ea7e999a7596"
format_stamp: "Formatted at 2026-08-10 04:41:28 on dist-test-slave-bqcl"
server_key: "965e4b700191b75ee31c62cb411ee4fa"
server_key_iv: "1a0a9fb3c4d7a9eeee2c598ede7fc23e"
server_key_version: "encryptionkey@0"
I20260810 04:41:28.273500 32105 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786336863388938-23197-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786336863388938-23197-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786336863388938-23197-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 04:41:28.302176 32105 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 04:41:28.302445 32105 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 04:41:28.302577 32105 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 04:41:28.306305 32105 rpc_server.cc:307] RPC server started. Bound to: 127.22.167.126:44827
I20260810 04:41:28.306345 32224 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.167.126:44827 every 8 connection(s)
I20260810 04:41:28.306640 32105 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786336863388938-23197-0/minicluster-data/master-0/data/info.pb
I20260810 04:41:28.307559 32226 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 04:41:28.309741 32226 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 52517878bff64958ae67ea7e999a7596: Bootstrap starting.
I20260810 04:41:28.310309 32226 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 52517878bff64958ae67ea7e999a7596: Neither blocks nor log segments found. Creating new log.
I20260810 04:41:28.310552 32226 log.cc:824] T 00000000000000000000000000000000 P 52517878bff64958ae67ea7e999a7596: Log is configured to *not* fsync() on all Append() calls
I20260810 04:41:28.311157 32226 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 52517878bff64958ae67ea7e999a7596: No bootstrap required, opened a new log
I20260810 04:41:28.312698 32226 raft_consensus.cc:348] T 00000000000000000000000000000000 P 52517878bff64958ae67ea7e999a7596 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "52517878bff64958ae67ea7e999a7596" member_type: VOTER last_known_addr { host: "127.22.167.126" port: 44827 } }
I20260810 04:41:28.312834 32226 raft_consensus.cc:374] T 00000000000000000000000000000000 P 52517878bff64958ae67ea7e999a7596 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 04:41:28.312855 32226 raft_consensus.cc:729] T 00000000000000000000000000000000 P 52517878bff64958ae67ea7e999a7596 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 52517878bff64958ae67ea7e999a7596, State: Initialized, Role: FOLLOWER
I20260810 04:41:28.312973 32226 consensus_queue.cc:260] T 00000000000000000000000000000000 P 52517878bff64958ae67ea7e999a7596 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "52517878bff64958ae67ea7e999a7596" member_type: VOTER last_known_addr { host: "127.22.167.126" port: 44827 } }
I20260810 04:41:28.313046 32226 raft_consensus.cc:388] T 00000000000000000000000000000000 P 52517878bff64958ae67ea7e999a7596 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 04:41:28.313073 32226 raft_consensus.cc:482] T 00000000000000000000000000000000 P 52517878bff64958ae67ea7e999a7596 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 04:41:28.313105 32226 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 52517878bff64958ae67ea7e999a7596 [term 0 FOLLOWER]: Advancing to term 1
I20260810 04:41:28.313560 32226 raft_consensus.cc:504] T 00000000000000000000000000000000 P 52517878bff64958ae67ea7e999a7596 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "52517878bff64958ae67ea7e999a7596" member_type: VOTER last_known_addr { host: "127.22.167.126" port: 44827 } }
I20260810 04:41:28.313642 32226 leader_election.cc:302] T 00000000000000000000000000000000 P 52517878bff64958ae67ea7e999a7596 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 52517878bff64958ae67ea7e999a7596; no voters:
I20260810 04:41:28.313764 32226 leader_election.cc:288] T 00000000000000000000000000000000 P 52517878bff64958ae67ea7e999a7596 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 04:41:28.313830 32229 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 52517878bff64958ae67ea7e999a7596 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 04:41:28.313980 32229 raft_consensus.cc:686] T 00000000000000000000000000000000 P 52517878bff64958ae67ea7e999a7596 [term 1 LEADER]: Becoming Leader. State: Replica: 52517878bff64958ae67ea7e999a7596, State: Running, Role: LEADER
I20260810 04:41:28.313995 32226 sys_catalog.cc:564] T 00000000000000000000000000000000 P 52517878bff64958ae67ea7e999a7596 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 04:41:28.314086 32229 consensus_queue.cc:237] T 00000000000000000000000000000000 P 52517878bff64958ae67ea7e999a7596 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "52517878bff64958ae67ea7e999a7596" member_type: VOTER last_known_addr { host: "127.22.167.126" port: 44827 } }
I20260810 04:41:28.314543 32233 sys_catalog.cc:455] T 00000000000000000000000000000000 P 52517878bff64958ae67ea7e999a7596 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 52517878bff64958ae67ea7e999a7596. Latest consensus state: current_term: 1 leader_uuid: "52517878bff64958ae67ea7e999a7596" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "52517878bff64958ae67ea7e999a7596" member_type: VOTER last_known_addr { host: "127.22.167.126" port: 44827 } } }
I20260810 04:41:28.314728 32233 sys_catalog.cc:458] T 00000000000000000000000000000000 P 52517878bff64958ae67ea7e999a7596 [sys.catalog]: This master's current role is: LEADER
I20260810 04:41:28.314747 23197 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskemv_AM/build/release/bin/kudu as pid 32105
I20260810 04:41:28.314851 23197 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786336863388938-23197-0/minicluster-data/master-0/wal/instance
I20260810 04:41:28.314669 32230 sys_catalog.cc:455] T 00000000000000000000000000000000 P 52517878bff64958ae67ea7e999a7596 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "52517878bff64958ae67ea7e999a7596" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "52517878bff64958ae67ea7e999a7596" member_type: VOTER last_known_addr { host: "127.22.167.126" port: 44827 } } }
I20260810 04:41:28.314882 32230 sys_catalog.cc:458] T 00000000000000000000000000000000 P 52517878bff64958ae67ea7e999a7596 [sys.catalog]: This master's current role is: LEADER
I20260810 04:41:28.315045 23197 external_mini_cluster.cc:1428] Setting key bc74615a2bbb9d74c93648e16b34ced0
I20260810 04:41:28.315142 32248 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 04:41:28.315904 32248 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 04:41:28.317265 32248 catalog_manager.cc:1347] Generated new cluster ID: e337b0af482f467bb88461b504f42ee5
I20260810 04:41:28.317312 32248 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 04:41:28.321178 32258 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:41:28.315469 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47730 (local address 127.22.167.126:44827)
0810 04:41:28.315715 (+ 246us) server_negotiation.cc:206] Beginning negotiation
0810 04:41:28.315725 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 04:41:28.315777 (+ 52us) server_negotiation.cc:407] Connection header received
0810 04:41:28.315897 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:41:28.315904 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:41:28.315974 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:41:28.316412 (+ 438us) server_negotiation.cc:226] Negotiated authn=SASL
0810 04:41:28.316514 (+ 102us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:41:28.317844 (+ 1330us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:41:28.318974 (+ 1130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:41:28.319620 (+ 646us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:41:28.319747 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:41:28.320374 (+ 627us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 04:41:28.320378 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 04:41:28.320379 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 04:41:28.320584 (+ 205us) server_negotiation.cc:922] calling sasl_server_start()
0810 04:41:28.320628 (+ 44us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 04:41:28.320643 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 04:41:28.320713 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0810 04:41:28.320772 (+ 59us) server_negotiation.cc:299] Negotiation successful
0810 04:41:28.320950 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":116,"thread_start_us":75,"threads_started":1}
I20260810 04:41:28.322041 23197 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 04:41:28.331327 32248 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 04:41:28.331809 32248 catalog_manager.cc:1504] Loading token signing keys...
I20260810 04:41:28.337958 32248 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 52517878bff64958ae67ea7e999a7596: Generated new TSK 0
I20260810 04:41:28.338112 32248 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260810 04:41:28.359220 32276 webserver.cc:533] Webserver: sslize failed: error:14094415:SSL routines:ssl3_read_bytes:sslv3 alert certificate expired
E20260810 04:41:28.359359 32276 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
W20260810 04:41:28.359817 32275 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 04:41:28.344831 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.126:44827 (local address 127.0.0.1:47738)
0810 04:41:28.345002 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0810 04:41:28.345012 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0810 04:41:28.345076 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 04:41:28.345253 (+ 177us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 04:41:28.345254 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 04:41:28.345256 (+ 2us) client_negotiation.cc:189] Negotiated authn=JWT
0810 04:41:28.345329 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:41:28.345332 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:41:28.345950 (+ 618us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:41:28.345951 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:41:28.346579 (+ 628us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:41:28.346582 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:41:28.346896 (+ 314us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:41:28.346897 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:41:28.346965 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:41:28.346987 (+ 22us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0810 04:41:28.359432 (+ 12445us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33925/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0810 04:41:28.359518 (+ 86us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.22.167.126:44827: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33925/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":80,"threads_started":1}
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:597: Failure
Failed
Bad status: Runtime error: Could not connect to the cluster: Client connection negotiation failed: client connection to 127.22.167.126:44827: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33925/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Google Test trace:
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:593: Valid JWT
I20260810 04:41:28.360095 32258 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:41:28.344896 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47738 (local address 127.22.167.126:44827)
0810 04:41:28.344943 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0810 04:41:28.344944 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0810 04:41:28.345073 (+ 129us) server_negotiation.cc:407] Connection header received
0810 04:41:28.345163 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:41:28.345164 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:41:28.345176 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:41:28.345231 (+ 55us) server_negotiation.cc:226] Negotiated authn=JWT
0810 04:41:28.345385 (+ 154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:41:28.345879 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:41:28.346650 (+ 771us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:41:28.346831 (+ 181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:41:28.346884 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:41:28.347089 (+ 205us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0810 04:41:28.359287 (+ 12198us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33925/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0810 04:41:28.360026 (+ 739us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:47738: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33925/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"server-negotiator.queue_time_us":24}
I20260810 04:41:28.360124 23197 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskemv_AM/build/release/bin/kudu with pid 32105
W20260810 04:41:28.360137 32258 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:47738: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33925/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
2026-08-10T04:41:28Z chronyd exiting
I20260810 04:41:28.427166 23197 test_util.cc:175] -----------------------------------------------
I20260810 04:41:28.427245 23197 test_util.cc:176] Had fatal failures, leaving test files at /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786336863388938-23197-0
[ FAILED ] SecurityITest.TestJwtMiniCluster (256 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithInvalidCert
I20260810 04:41:28.427950 23197 mini_oidc.cc:152] Starting JWKS server
I20260810 04:41:28.428586 23197 webserver.cc:460] Webserver started at https://127.0.0.1:33713/ using document root <none> and password file <none>
I20260810 04:41:28.428625 23197 mini_oidc.cc:189] Starting OIDC Discovery server
I20260810 04:41:28.428731 23197 webserver.cc:460] Webserver started at http://127.0.0.1:34797/ using document root <none> and password file <none>
2026-08-10T04:41:28Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T04:41:28Z Disabled control of system clock
I20260810 04:41:28.436960 23197 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskemv_AM/build/release/bin/kudu
/tmp/dist-test-taskemv_AM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786336863388938-23197-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786336863388938-23197-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786336863388938-23197-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786336863388938-23197-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.167.126:43647
--webserver_interface=127.22.167.126
--webserver_port=0
--builtin_ntp_servers=127.22.167.84:38501
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.167.126:43647
--enable_jwt_token_auth=true
--jwks_url=https://localhost:33713/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786336863388938-23197-0/testchainca.cert with env {}
W20260810 04:41:28.515888 32285 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 04:41:28.516085 32285 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 04:41:28.516114 32285 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 04:41:28.517213 32285 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 04:41:28.517252 32285 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 04:41:28.517268 32285 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 04:41:28.517283 32285 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 04:41:28.517298 32285 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260810 04:41:28.517315 32285 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260810 04:41:28.517338 32285 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:33713/jwks.json
I20260810 04:41:28.518828 32285 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.167.84:38501
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786336863388938-23197-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786336863388938-23197-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.22.167.126:43647
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.167.126:43647
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:33713/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786336863388938-23197-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.167.126
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786336863388938-23197-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32285
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786336863388938-23197-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 03:54:01 UTC on ccycloud.quasar-icijmt.root.comops.site
build id 4577009
I20260810 04:41:28.519059 32285 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 04:41:28.519244 32285 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 04:41:28.523617 32296 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 04:41:28.523622 32294 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 04:41:28.523639 32293 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 04:41:28.523892 32285 server_base.cc:1034] running on GCE node
I20260810 04:41:28.524123 32285 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 04:41:28.524385 32285 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 04:41:28.525522 32285 hybrid_clock.cc:648] HybridClock initialized: now 1786336888525503 us; error 32 us; skew 500 ppm
I20260810 04:41:28.526710 32285 webserver.cc:460] Webserver started at http://127.22.167.126:37597/ using document root <none> and password file <none>
I20260810 04:41:28.526913 32285 fs_manager.cc:359] Metadata directory not provided
I20260810 04:41:28.526961 32285 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 04:41:28.527084 32285 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 04:41:28.528039 32285 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786336863388938-23197-0/minicluster-data/master-0/data/instance:
uuid: "8f5625217b744ee1884de686df186eba"
format_stamp: "Formatted at 2026-08-10 04:41:28 on dist-test-slave-bqcl"
server_key: "b66691524bb4e6dc35fb7d8b13130277"
server_key_iv: "c7afb921def9bc5cbe8f75f0a6840b4b"
server_key_version: "encryptionkey@0"
I20260810 04:41:28.528386 32285 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786336863388938-23197-0/minicluster-data/master-0/wal/instance:
uuid: "8f5625217b744ee1884de686df186eba"
format_stamp: "Formatted at 2026-08-10 04:41:28 on dist-test-slave-bqcl"
server_key: "b66691524bb4e6dc35fb7d8b13130277"
server_key_iv: "c7afb921def9bc5cbe8f75f0a6840b4b"
server_key_version: "encryptionkey@0"
I20260810 04:41:28.529556 32285 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260810 04:41:28.530172 32309 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 04:41:28.530306 32285 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260810 04:41:28.530388 32285 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786336863388938-23197-0/minicluster-data/master-0/data,/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786336863388938-23197-0/minicluster-data/master-0/wal
uuid: "8f5625217b744ee1884de686df186eba"
format_stamp: "Formatted at 2026-08-10 04:41:28 on dist-test-slave-bqcl"
server_key: "b66691524bb4e6dc35fb7d8b13130277"
server_key_iv: "c7afb921def9bc5cbe8f75f0a6840b4b"
server_key_version: "encryptionkey@0"
I20260810 04:41:28.530454 32285 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786336863388938-23197-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786336863388938-23197-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786336863388938-23197-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 04:41:28.544349 32285 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 04:41:28.544610 32285 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 04:41:28.544731 32285 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 04:41:28.548492 32285 rpc_server.cc:307] RPC server started. Bound to: 127.22.167.126:43647
I20260810 04:41:28.548523 32396 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.167.126:43647 every 8 connection(s)
I20260810 04:41:28.548820 32285 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786336863388938-23197-0/minicluster-data/master-0/data/info.pb
I20260810 04:41:28.549467 32397 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 04:41:28.550876 23197 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskemv_AM/build/release/bin/kudu as pid 32285
I20260810 04:41:28.550972 23197 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786336863388938-23197-0/minicluster-data/master-0/wal/instance
I20260810 04:41:28.551151 23197 external_mini_cluster.cc:1428] Setting key 9c4cbb78619eccf61fd157a13939285d
I20260810 04:41:28.552434 32397 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8f5625217b744ee1884de686df186eba: Bootstrap starting.
I20260810 04:41:28.553057 32397 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8f5625217b744ee1884de686df186eba: Neither blocks nor log segments found. Creating new log.
I20260810 04:41:28.553318 32397 log.cc:824] T 00000000000000000000000000000000 P 8f5625217b744ee1884de686df186eba: Log is configured to *not* fsync() on all Append() calls
I20260810 04:41:28.554044 32397 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8f5625217b744ee1884de686df186eba: No bootstrap required, opened a new log
I20260810 04:41:28.556322 32397 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8f5625217b744ee1884de686df186eba [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f5625217b744ee1884de686df186eba" member_type: VOTER last_known_addr { host: "127.22.167.126" port: 43647 } }
I20260810 04:41:28.556470 32401 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:41:28.551623 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45628 (local address 127.22.167.126:43647)
0810 04:41:28.551799 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0810 04:41:28.551806 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 04:41:28.551829 (+ 23us) server_negotiation.cc:407] Connection header received
0810 04:41:28.552269 (+ 440us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:41:28.552276 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:41:28.552334 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:41:28.552445 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0810 04:41:28.552678 (+ 233us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:41:28.553611 (+ 933us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:41:28.554442 (+ 831us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:41:28.555039 (+ 597us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:41:28.555165 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:41:28.555728 (+ 563us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 04:41:28.555733 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 04:41:28.555733 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 04:41:28.555945 (+ 212us) server_negotiation.cc:922] calling sasl_server_start()
0810 04:41:28.555993 (+ 48us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 04:41:28.556008 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 04:41:28.556085 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0810 04:41:28.556142 (+ 57us) server_negotiation.cc:299] Negotiation successful
0810 04:41:28.556223 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":110,"thread_start_us":65,"threads_started":1}
I20260810 04:41:28.556511 32397 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8f5625217b744ee1884de686df186eba [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 04:41:28.556557 32397 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8f5625217b744ee1884de686df186eba [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8f5625217b744ee1884de686df186eba, State: Initialized, Role: FOLLOWER
I20260810 04:41:28.556694 32397 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8f5625217b744ee1884de686df186eba [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f5625217b744ee1884de686df186eba" member_type: VOTER last_known_addr { host: "127.22.167.126" port: 43647 } }
I20260810 04:41:28.556775 32397 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8f5625217b744ee1884de686df186eba [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 04:41:28.556806 32397 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8f5625217b744ee1884de686df186eba [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 04:41:28.556854 32397 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8f5625217b744ee1884de686df186eba [term 0 FOLLOWER]: Advancing to term 1
I20260810 04:41:28.557322 23197 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 04:41:28.557482 32397 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8f5625217b744ee1884de686df186eba [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f5625217b744ee1884de686df186eba" member_type: VOTER last_known_addr { host: "127.22.167.126" port: 43647 } }
I20260810 04:41:28.557587 32397 leader_election.cc:302] T 00000000000000000000000000000000 P 8f5625217b744ee1884de686df186eba [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8f5625217b744ee1884de686df186eba; no voters:
I20260810 04:41:28.557744 32397 leader_election.cc:288] T 00000000000000000000000000000000 P 8f5625217b744ee1884de686df186eba [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 04:41:28.557791 32409 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8f5625217b744ee1884de686df186eba [term 1 FOLLOWER]: Leader election won for term 1
I20260810 04:41:28.557926 32409 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8f5625217b744ee1884de686df186eba [term 1 LEADER]: Becoming Leader. State: Replica: 8f5625217b744ee1884de686df186eba, State: Running, Role: LEADER
I20260810 04:41:28.557971 32397 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8f5625217b744ee1884de686df186eba [sys.catalog]: configured and running, proceeding with master startup.
I20260810 04:41:28.558001 32409 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8f5625217b744ee1884de686df186eba [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f5625217b744ee1884de686df186eba" member_type: VOTER last_known_addr { host: "127.22.167.126" port: 43647 } }
I20260810 04:41:28.558388 32410 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8f5625217b744ee1884de686df186eba [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8f5625217b744ee1884de686df186eba" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f5625217b744ee1884de686df186eba" member_type: VOTER last_known_addr { host: "127.22.167.126" port: 43647 } } }
I20260810 04:41:28.558477 32410 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8f5625217b744ee1884de686df186eba [sys.catalog]: This master's current role is: LEADER
I20260810 04:41:28.558393 32412 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8f5625217b744ee1884de686df186eba [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8f5625217b744ee1884de686df186eba. Latest consensus state: current_term: 1 leader_uuid: "8f5625217b744ee1884de686df186eba" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f5625217b744ee1884de686df186eba" member_type: VOTER last_known_addr { host: "127.22.167.126" port: 43647 } } }
I20260810 04:41:28.558676 32412 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8f5625217b744ee1884de686df186eba [sys.catalog]: This master's current role is: LEADER
I20260810 04:41:28.558779 32418 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 04:41:28.559199 32418 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 04:41:28.560549 32418 catalog_manager.cc:1347] Generated new cluster ID: 8194bcb283b749afaab536728a8e405a
I20260810 04:41:28.560595 32418 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 04:41:28.581185 32418 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 04:41:28.581743 32418 catalog_manager.cc:1504] Loading token signing keys...
I20260810 04:41:28.584674 32418 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8f5625217b744ee1884de686df186eba: Generated new TSK 0
I20260810 04:41:28.584834 32418 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260810 04:41:28.610950 32445 webserver.cc:533] Webserver: sslize failed: error:14094418:SSL routines:ssl3_read_bytes:tlsv1 alert unknown ca
E20260810 04:41:28.611023 32445 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260810 04:41:28.611266 32401 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:41:28.589887 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45644 (local address 127.22.167.126:43647)
0810 04:41:28.589923 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0810 04:41:28.589926 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 04:41:28.590054 (+ 128us) server_negotiation.cc:407] Connection header received
0810 04:41:28.590117 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:41:28.590117 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:41:28.590157 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:41:28.590218 (+ 61us) server_negotiation.cc:226] Negotiated authn=JWT
0810 04:41:28.590348 (+ 130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:41:28.590840 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:41:28.591566 (+ 726us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:41:28.591783 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:41:28.591837 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:41:28.592008 (+ 171us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0810 04:41:28.611082 (+ 19074us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33713/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0810 04:41:28.611213 (+ 131us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:45644: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33713/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"server-negotiator.queue_time_us":16}
W20260810 04:41:28.611311 32401 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:45644: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33713/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
W20260810 04:41:28.611400 32444 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 04:41:28.589840 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.126:43647 (local address 127.0.0.1:45644)
0810 04:41:28.589989 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0810 04:41:28.589999 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0810 04:41:28.590058 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 04:41:28.590226 (+ 168us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 04:41:28.590227 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 04:41:28.590229 (+ 2us) client_negotiation.cc:189] Negotiated authn=JWT
0810 04:41:28.590289 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:41:28.590292 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:41:28.590921 (+ 629us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:41:28.590922 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:41:28.591493 (+ 571us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:41:28.591495 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:41:28.591863 (+ 368us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:41:28.591864 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:41:28.591920 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:41:28.591926 (+ 6us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0810 04:41:28.611200 (+ 19274us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33713/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0810 04:41:28.611305 (+ 105us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.22.167.126:43647: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33713/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":89,"threads_started":1}
I20260810 04:41:28.611670 23197 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskemv_AM/build/release/bin/kudu with pid 32285
2026-08-10T04:41:28Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithInvalidCert (235 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithUntrustedCert
I20260810 04:41:28.663499 23197 mini_oidc.cc:152] Starting JWKS server
I20260810 04:41:28.664176 23197 webserver.cc:460] Webserver started at https://127.0.0.1:37719/ using document root <none> and password file <none>
I20260810 04:41:28.664222 23197 mini_oidc.cc:189] Starting OIDC Discovery server
I20260810 04:41:28.664356 23197 webserver.cc:460] Webserver started at http://127.0.0.1:43617/ using document root <none> and password file <none>
2026-08-10T04:41:28Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T04:41:28Z Disabled control of system clock
I20260810 04:41:28.673019 23197 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskemv_AM/build/release/bin/kudu
/tmp/dist-test-taskemv_AM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786336863388938-23197-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786336863388938-23197-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786336863388938-23197-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786336863388938-23197-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.167.126:34505
--webserver_interface=127.22.167.126
--webserver_port=0
--builtin_ntp_servers=127.22.167.84:40495
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.167.126:34505
--enable_jwt_token_auth=true
--jwks_url=https://localhost:37719/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260810 04:41:28.753973 32455 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 04:41:28.754184 32455 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 04:41:28.754215 32455 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 04:41:28.755317 32455 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 04:41:28.755355 32455 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 04:41:28.755371 32455 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 04:41:28.755386 32455 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 04:41:28.755401 32455 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260810 04:41:28.755427 32455 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260810 04:41:28.755450 32455 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:37719/jwks.json
I20260810 04:41:28.756754 32455 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.167.84:40495
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786336863388938-23197-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786336863388938-23197-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.22.167.126:34505
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.167.126:34505
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:37719/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786336863388938-23197-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.167.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32455
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786336863388938-23197-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 03:54:01 UTC on ccycloud.quasar-icijmt.root.comops.site
build id 4577009
I20260810 04:41:28.756984 32455 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 04:41:28.757232 32455 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 04:41:28.761620 32467 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 04:41:28.761667 32462 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 04:41:28.761632 32461 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 04:41:28.761866 32455 server_base.cc:1034] running on GCE node
I20260810 04:41:28.762084 32455 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 04:41:28.762357 32455 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 04:41:28.763497 32455 hybrid_clock.cc:648] HybridClock initialized: now 1786336888763485 us; error 33 us; skew 500 ppm
I20260810 04:41:28.764664 32455 webserver.cc:460] Webserver started at http://127.22.167.126:46631/ using document root <none> and password file <none>
I20260810 04:41:28.764894 32455 fs_manager.cc:359] Metadata directory not provided
I20260810 04:41:28.764954 32455 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 04:41:28.765080 32455 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 04:41:28.766067 32455 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786336863388938-23197-0/minicluster-data/master-0/data/instance:
uuid: "8c8958c2d494493e98bdfa916d1bca1b"
format_stamp: "Formatted at 2026-08-10 04:41:28 on dist-test-slave-bqcl"
server_key: "b39e2f44baf9a577dd6d31f0060adbbd"
server_key_iv: "acf8e6472c14125f2599c4aa431be41f"
server_key_version: "encryptionkey@0"
I20260810 04:41:28.766434 32455 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786336863388938-23197-0/minicluster-data/master-0/wal/instance:
uuid: "8c8958c2d494493e98bdfa916d1bca1b"
format_stamp: "Formatted at 2026-08-10 04:41:28 on dist-test-slave-bqcl"
server_key: "b39e2f44baf9a577dd6d31f0060adbbd"
server_key_iv: "acf8e6472c14125f2599c4aa431be41f"
server_key_version: "encryptionkey@0"
I20260810 04:41:28.767640 32455 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260810 04:41:28.768273 32474 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 04:41:28.768424 32455 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260810 04:41:28.768505 32455 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786336863388938-23197-0/minicluster-data/master-0/data,/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786336863388938-23197-0/minicluster-data/master-0/wal
uuid: "8c8958c2d494493e98bdfa916d1bca1b"
format_stamp: "Formatted at 2026-08-10 04:41:28 on dist-test-slave-bqcl"
server_key: "b39e2f44baf9a577dd6d31f0060adbbd"
server_key_iv: "acf8e6472c14125f2599c4aa431be41f"
server_key_version: "encryptionkey@0"
I20260810 04:41:28.768582 32455 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786336863388938-23197-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786336863388938-23197-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786336863388938-23197-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 04:41:28.799206 32455 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 04:41:28.799489 32455 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 04:41:28.799628 32455 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 04:41:28.803403 32455 rpc_server.cc:307] RPC server started. Bound to: 127.22.167.126:34505
I20260810 04:41:28.803442 32567 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.167.126:34505 every 8 connection(s)
I20260810 04:41:28.803745 32455 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786336863388938-23197-0/minicluster-data/master-0/data/info.pb
I20260810 04:41:28.804365 32568 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 04:41:28.807096 32568 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8c8958c2d494493e98bdfa916d1bca1b: Bootstrap starting.
I20260810 04:41:28.807195 23197 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskemv_AM/build/release/bin/kudu as pid 32455
I20260810 04:41:28.807304 23197 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786336863388938-23197-0/minicluster-data/master-0/wal/instance
I20260810 04:41:28.807492 23197 external_mini_cluster.cc:1428] Setting key 99b4056e90d38f5df7471bda2c20f197
I20260810 04:41:28.807775 32568 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8c8958c2d494493e98bdfa916d1bca1b: Neither blocks nor log segments found. Creating new log.
I20260810 04:41:28.808089 32568 log.cc:824] T 00000000000000000000000000000000 P 8c8958c2d494493e98bdfa916d1bca1b: Log is configured to *not* fsync() on all Append() calls
I20260810 04:41:28.808837 32568 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8c8958c2d494493e98bdfa916d1bca1b: No bootstrap required, opened a new log
I20260810 04:41:28.811074 32568 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8c8958c2d494493e98bdfa916d1bca1b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8c8958c2d494493e98bdfa916d1bca1b" member_type: VOTER last_known_addr { host: "127.22.167.126" port: 34505 } }
I20260810 04:41:28.811211 32568 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8c8958c2d494493e98bdfa916d1bca1b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 04:41:28.811233 32568 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8c8958c2d494493e98bdfa916d1bca1b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8c8958c2d494493e98bdfa916d1bca1b, State: Initialized, Role: FOLLOWER
I20260810 04:41:28.811336 32568 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8c8958c2d494493e98bdfa916d1bca1b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8c8958c2d494493e98bdfa916d1bca1b" member_type: VOTER last_known_addr { host: "127.22.167.126" port: 34505 } }
I20260810 04:41:28.811403 32568 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8c8958c2d494493e98bdfa916d1bca1b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 04:41:28.811424 32568 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8c8958c2d494493e98bdfa916d1bca1b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 04:41:28.811467 32568 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8c8958c2d494493e98bdfa916d1bca1b [term 0 FOLLOWER]: Advancing to term 1
I20260810 04:41:28.811985 32568 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8c8958c2d494493e98bdfa916d1bca1b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8c8958c2d494493e98bdfa916d1bca1b" member_type: VOTER last_known_addr { host: "127.22.167.126" port: 34505 } }
I20260810 04:41:28.812080 32568 leader_election.cc:302] T 00000000000000000000000000000000 P 8c8958c2d494493e98bdfa916d1bca1b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8c8958c2d494493e98bdfa916d1bca1b; no voters:
I20260810 04:41:28.812228 32568 leader_election.cc:288] T 00000000000000000000000000000000 P 8c8958c2d494493e98bdfa916d1bca1b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 04:41:28.812284 32578 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8c8958c2d494493e98bdfa916d1bca1b [term 1 FOLLOWER]: Leader election won for term 1
I20260810 04:41:28.812405 32578 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8c8958c2d494493e98bdfa916d1bca1b [term 1 LEADER]: Becoming Leader. State: Replica: 8c8958c2d494493e98bdfa916d1bca1b, State: Running, Role: LEADER
I20260810 04:41:28.812487 32568 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8c8958c2d494493e98bdfa916d1bca1b [sys.catalog]: configured and running, proceeding with master startup.
I20260810 04:41:28.812577 32578 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8c8958c2d494493e98bdfa916d1bca1b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8c8958c2d494493e98bdfa916d1bca1b" member_type: VOTER last_known_addr { host: "127.22.167.126" port: 34505 } }
I20260810 04:41:28.812945 32580 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8c8958c2d494493e98bdfa916d1bca1b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8c8958c2d494493e98bdfa916d1bca1b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8c8958c2d494493e98bdfa916d1bca1b" member_type: VOTER last_known_addr { host: "127.22.167.126" port: 34505 } } }
I20260810 04:41:28.813030 32580 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8c8958c2d494493e98bdfa916d1bca1b [sys.catalog]: This master's current role is: LEADER
I20260810 04:41:28.813186 32581 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8c8958c2d494493e98bdfa916d1bca1b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8c8958c2d494493e98bdfa916d1bca1b. Latest consensus state: current_term: 1 leader_uuid: "8c8958c2d494493e98bdfa916d1bca1b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8c8958c2d494493e98bdfa916d1bca1b" member_type: VOTER last_known_addr { host: "127.22.167.126" port: 34505 } } }
I20260810 04:41:28.813244 32581 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8c8958c2d494493e98bdfa916d1bca1b [sys.catalog]: This master's current role is: LEADER
I20260810 04:41:28.813418 32572 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:41:28.807931 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52072 (local address 127.22.167.126:34505)
0810 04:41:28.808167 (+ 236us) server_negotiation.cc:206] Beginning negotiation
0810 04:41:28.808176 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 04:41:28.808206 (+ 30us) server_negotiation.cc:407] Connection header received
0810 04:41:28.808734 (+ 528us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:41:28.808743 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:41:28.808816 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:41:28.808964 (+ 148us) server_negotiation.cc:226] Negotiated authn=SASL
0810 04:41:28.809173 (+ 209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:41:28.810578 (+ 1405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:41:28.811313 (+ 735us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:41:28.811781 (+ 468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:41:28.811890 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:41:28.812496 (+ 606us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 04:41:28.812500 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 04:41:28.812501 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 04:41:28.812707 (+ 206us) server_negotiation.cc:922] calling sasl_server_start()
0810 04:41:28.812749 (+ 42us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 04:41:28.812763 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 04:41:28.812821 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0810 04:41:28.813141 (+ 320us) server_negotiation.cc:299] Negotiation successful
0810 04:41:28.813224 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":165,"thread_start_us":73,"threads_started":1}
I20260810 04:41:28.814240 23197 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
W20260810 04:41:28.814385 32606 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 8c8958c2d494493e98bdfa916d1bca1b: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260810 04:41:28.814435 32606 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260810 04:41:28.814481 32592 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 04:41:28.814647 32592 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 04:41:28.815964 32592 catalog_manager.cc:1347] Generated new cluster ID: ad704a1a7bd24ae7b77bf84ed3a56584
I20260810 04:41:28.816018 32592 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 04:41:28.832901 32592 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 04:41:28.833547 32592 catalog_manager.cc:1504] Loading token signing keys...
I20260810 04:41:28.836040 32592 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8c8958c2d494493e98bdfa916d1bca1b: Generated new TSK 0
I20260810 04:41:28.836192 32592 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 04:41:28.846978 32572 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:41:28.843962 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52082 (local address 127.22.167.126:34505)
0810 04:41:28.844006 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0810 04:41:28.844008 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 04:41:28.844098 (+ 90us) server_negotiation.cc:407] Connection header received
0810 04:41:28.844169 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:41:28.844170 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:41:28.844180 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:41:28.844236 (+ 56us) server_negotiation.cc:226] Negotiated authn=JWT
0810 04:41:28.844353 (+ 117us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:41:28.844826 (+ 473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:41:28.845554 (+ 728us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:41:28.845746 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:41:28.845800 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:41:28.845963 (+ 163us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0810 04:41:28.846783 (+ 820us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:37719/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0810 04:41:28.846906 (+ 123us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:52082: Failed to load JWKS: Error downloading JWKS from 'https://localhost:37719/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"server-negotiator.queue_time_us":23}
W20260810 04:41:28.847030 32572 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:52082: Failed to load JWKS: Error downloading JWKS from 'https://localhost:37719/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
W20260810 04:41:28.847090 32623 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 04:41:28.843905 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.126:34505 (local address 127.0.0.1:52082)
0810 04:41:28.844037 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0810 04:41:28.844047 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0810 04:41:28.844107 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 04:41:28.844241 (+ 134us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 04:41:28.844242 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 04:41:28.844243 (+ 1us) client_negotiation.cc:189] Negotiated authn=JWT
0810 04:41:28.844292 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:41:28.844294 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:41:28.844899 (+ 605us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:41:28.844900 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:41:28.845483 (+ 583us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:41:28.845485 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:41:28.845812 (+ 327us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:41:28.845814 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:41:28.845871 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:41:28.845877 (+ 6us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0810 04:41:28.846883 (+ 1006us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:37719/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0810 04:41:28.846982 (+ 99us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.22.167.126:34505: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:37719/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":86,"threads_started":1}
I20260810 04:41:28.847302 23197 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskemv_AM/build/release/bin/kudu with pid 32455
E20260810 04:41:28.868938 32627 webserver.cc:533] Webserver: sslize failed:
E20260810 04:41:28.869004 32627 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
2026-08-10T04:41:28Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithUntrustedCert (233 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS
2026-08-10T04:41:28Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T04:41:28Z Disabled control of system clock
I20260810 04:41:28.904798 23197 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskemv_AM/build/release/bin/kudu
/tmp/dist-test-taskemv_AM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786336863388938-23197-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786336863388938-23197-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786336863388938-23197-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786336863388938-23197-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.167.126:45969
--webserver_interface=127.22.167.126
--webserver_port=0
--builtin_ntp_servers=127.22.167.84:38781
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.167.126:45969
--enable_jwt_token_auth=true
--jwks_url=default.url
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786336863388938-23197-0/testchainca.cert with env {}
W20260810 04:41:28.983192 32633 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 04:41:28.983383 32633 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 04:41:28.983403 32633 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 04:41:28.984539 32633 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 04:41:28.984579 32633 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 04:41:28.984591 32633 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 04:41:28.984609 32633 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 04:41:28.984642 32633 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260810 04:41:28.984664 32633 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260810 04:41:28.984685 32633 flags.cc:425] Enabled experimental flag: --jwks_url=default.url
I20260810 04:41:28.986189 32633 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.167.84:38781
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786336863388938-23197-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786336863388938-23197-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.22.167.126:45969
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.167.126:45969
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=default.url
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786336863388938-23197-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.167.126
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786336863388938-23197-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32633
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786336863388938-23197-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 03:54:01 UTC on ccycloud.quasar-icijmt.root.comops.site
build id 4577009
I20260810 04:41:28.986404 32633 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 04:41:28.986591 32633 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 04:41:28.991367 32642 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 04:41:28.991413 32633 server_base.cc:1034] running on GCE node
W20260810 04:41:28.991353 32646 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 04:41:28.991353 32643 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 04:41:28.991837 32633 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 04:41:28.992097 32633 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 04:41:28.993230 32633 hybrid_clock.cc:648] HybridClock initialized: now 1786336888993216 us; error 31 us; skew 500 ppm
I20260810 04:41:28.994412 32633 webserver.cc:460] Webserver started at http://127.22.167.126:38469/ using document root <none> and password file <none>
I20260810 04:41:28.994591 32633 fs_manager.cc:359] Metadata directory not provided
I20260810 04:41:28.994629 32633 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 04:41:28.994729 32633 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 04:41:28.995682 32633 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786336863388938-23197-0/minicluster-data/master-0/data/instance:
uuid: "386a61ba7e394d448c086cb38e113bf5"
format_stamp: "Formatted at 2026-08-10 04:41:28 on dist-test-slave-bqcl"
server_key: "c12c5b65515597777affc9af94c30967"
server_key_iv: "c897f5fb35aae39640de0d670f67aece"
server_key_version: "encryptionkey@0"
I20260810 04:41:28.995972 32633 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786336863388938-23197-0/minicluster-data/master-0/wal/instance:
uuid: "386a61ba7e394d448c086cb38e113bf5"
format_stamp: "Formatted at 2026-08-10 04:41:28 on dist-test-slave-bqcl"
server_key: "c12c5b65515597777affc9af94c30967"
server_key_iv: "c897f5fb35aae39640de0d670f67aece"
server_key_version: "encryptionkey@0"
I20260810 04:41:28.997129 32633 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260810 04:41:28.997761 32654 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 04:41:28.997912 32633 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260810 04:41:28.997991 32633 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786336863388938-23197-0/minicluster-data/master-0/data,/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786336863388938-23197-0/minicluster-data/master-0/wal
uuid: "386a61ba7e394d448c086cb38e113bf5"
format_stamp: "Formatted at 2026-08-10 04:41:28 on dist-test-slave-bqcl"
server_key: "c12c5b65515597777affc9af94c30967"
server_key_iv: "c897f5fb35aae39640de0d670f67aece"
server_key_version: "encryptionkey@0"
I20260810 04:41:28.998041 32633 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786336863388938-23197-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786336863388938-23197-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786336863388938-23197-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 04:41:29.014465 32633 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 04:41:29.014681 32633 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 04:41:29.014777 32633 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 04:41:29.018319 32633 rpc_server.cc:307] RPC server started. Bound to: 127.22.167.126:45969
I20260810 04:41:29.018391 32743 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.167.126:45969 every 8 connection(s)
I20260810 04:41:29.018651 32633 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786336863388938-23197-0/minicluster-data/master-0/data/info.pb
I20260810 04:41:29.019569 32744 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 04:41:29.021786 32744 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 386a61ba7e394d448c086cb38e113bf5: Bootstrap starting.
I20260810 04:41:29.022389 32744 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 386a61ba7e394d448c086cb38e113bf5: Neither blocks nor log segments found. Creating new log.
I20260810 04:41:29.022675 32744 log.cc:824] T 00000000000000000000000000000000 P 386a61ba7e394d448c086cb38e113bf5: Log is configured to *not* fsync() on all Append() calls
I20260810 04:41:29.023290 32744 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 386a61ba7e394d448c086cb38e113bf5: No bootstrap required, opened a new log
I20260810 04:41:29.024911 32744 raft_consensus.cc:348] T 00000000000000000000000000000000 P 386a61ba7e394d448c086cb38e113bf5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "386a61ba7e394d448c086cb38e113bf5" member_type: VOTER last_known_addr { host: "127.22.167.126" port: 45969 } }
I20260810 04:41:29.025061 32744 raft_consensus.cc:374] T 00000000000000000000000000000000 P 386a61ba7e394d448c086cb38e113bf5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 04:41:29.025100 32744 raft_consensus.cc:729] T 00000000000000000000000000000000 P 386a61ba7e394d448c086cb38e113bf5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 386a61ba7e394d448c086cb38e113bf5, State: Initialized, Role: FOLLOWER
I20260810 04:41:29.025257 32744 consensus_queue.cc:260] T 00000000000000000000000000000000 P 386a61ba7e394d448c086cb38e113bf5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "386a61ba7e394d448c086cb38e113bf5" member_type: VOTER last_known_addr { host: "127.22.167.126" port: 45969 } }
I20260810 04:41:29.025348 32744 raft_consensus.cc:388] T 00000000000000000000000000000000 P 386a61ba7e394d448c086cb38e113bf5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 04:41:29.025384 32744 raft_consensus.cc:482] T 00000000000000000000000000000000 P 386a61ba7e394d448c086cb38e113bf5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 04:41:29.025437 32744 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 386a61ba7e394d448c086cb38e113bf5 [term 0 FOLLOWER]: Advancing to term 1
I20260810 04:41:29.025956 32744 raft_consensus.cc:504] T 00000000000000000000000000000000 P 386a61ba7e394d448c086cb38e113bf5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "386a61ba7e394d448c086cb38e113bf5" member_type: VOTER last_known_addr { host: "127.22.167.126" port: 45969 } }
I20260810 04:41:29.026065 32744 leader_election.cc:302] T 00000000000000000000000000000000 P 386a61ba7e394d448c086cb38e113bf5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 386a61ba7e394d448c086cb38e113bf5; no voters:
I20260810 04:41:29.026268 32744 leader_election.cc:288] T 00000000000000000000000000000000 P 386a61ba7e394d448c086cb38e113bf5 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 04:41:29.026311 32747 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 386a61ba7e394d448c086cb38e113bf5 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 04:41:29.026448 32747 raft_consensus.cc:686] T 00000000000000000000000000000000 P 386a61ba7e394d448c086cb38e113bf5 [term 1 LEADER]: Becoming Leader. State: Replica: 386a61ba7e394d448c086cb38e113bf5, State: Running, Role: LEADER
I20260810 04:41:29.026517 32744 sys_catalog.cc:564] T 00000000000000000000000000000000 P 386a61ba7e394d448c086cb38e113bf5 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 04:41:29.026546 32747 consensus_queue.cc:237] T 00000000000000000000000000000000 P 386a61ba7e394d448c086cb38e113bf5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "386a61ba7e394d448c086cb38e113bf5" member_type: VOTER last_known_addr { host: "127.22.167.126" port: 45969 } }
I20260810 04:41:29.026911 32750 sys_catalog.cc:455] T 00000000000000000000000000000000 P 386a61ba7e394d448c086cb38e113bf5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 386a61ba7e394d448c086cb38e113bf5. Latest consensus state: current_term: 1 leader_uuid: "386a61ba7e394d448c086cb38e113bf5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "386a61ba7e394d448c086cb38e113bf5" member_type: VOTER last_known_addr { host: "127.22.167.126" port: 45969 } } }
I20260810 04:41:29.026993 32750 sys_catalog.cc:458] T 00000000000000000000000000000000 P 386a61ba7e394d448c086cb38e113bf5 [sys.catalog]: This master's current role is: LEADER
I20260810 04:41:29.027078 32748 sys_catalog.cc:455] T 00000000000000000000000000000000 P 386a61ba7e394d448c086cb38e113bf5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "386a61ba7e394d448c086cb38e113bf5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "386a61ba7e394d448c086cb38e113bf5" member_type: VOTER last_known_addr { host: "127.22.167.126" port: 45969 } } }
I20260810 04:41:29.027220 32748 sys_catalog.cc:458] T 00000000000000000000000000000000 P 386a61ba7e394d448c086cb38e113bf5 [sys.catalog]: This master's current role is: LEADER
I20260810 04:41:29.027391 32762 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 04:41:29.027868 32762 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 04:41:29.028635 23197 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskemv_AM/build/release/bin/kudu as pid 32633
I20260810 04:41:29.028738 23197 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786336863388938-23197-0/minicluster-data/master-0/wal/instance
I20260810 04:41:29.028924 23197 external_mini_cluster.cc:1428] Setting key eb06714f7b7fbd5d50d5e385bee9234d
I20260810 04:41:29.029613 32762 catalog_manager.cc:1347] Generated new cluster ID: 2af7e74181594437a161e9f8a3367abc
I20260810 04:41:29.029666 32762 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 04:41:29.034485 311 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:41:29.029478 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59258 (local address 127.22.167.126:45969)
0810 04:41:29.030307 (+ 829us) server_negotiation.cc:206] Beginning negotiation
0810 04:41:29.030314 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 04:41:29.030338 (+ 24us) server_negotiation.cc:407] Connection header received
0810 04:41:29.030421 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:41:29.030426 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:41:29.030475 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:41:29.030604 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0810 04:41:29.030838 (+ 234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:41:29.031761 (+ 923us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:41:29.032527 (+ 766us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:41:29.032964 (+ 437us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:41:29.033088 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:41:29.033740 (+ 652us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 04:41:29.033744 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 04:41:29.033744 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 04:41:29.033923 (+ 179us) server_negotiation.cc:922] calling sasl_server_start()
0810 04:41:29.033957 (+ 34us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 04:41:29.033968 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 04:41:29.034037 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0810 04:41:29.034147 (+ 110us) server_negotiation.cc:299] Negotiation successful
0810 04:41:29.034316 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":775,"thread_start_us":98,"threads_started":1}
I20260810 04:41:29.035295 23197 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 04:41:29.046141 32762 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 04:41:29.046625 32762 catalog_manager.cc:1504] Loading token signing keys...
I20260810 04:41:29.050899 32762 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 386a61ba7e394d448c086cb38e113bf5: Generated new TSK 0
I20260810 04:41:29.051049 32762 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 04:41:29.471047 311 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:41:29.056345 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59266 (local address 127.22.167.126:45969)
0810 04:41:29.056387 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0810 04:41:29.056389 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 04:41:29.056467 (+ 78us) server_negotiation.cc:407] Connection header received
0810 04:41:29.056536 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:41:29.056537 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:41:29.056549 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:41:29.056607 (+ 58us) server_negotiation.cc:226] Negotiated authn=JWT
0810 04:41:29.056734 (+ 127us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:41:29.057226 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:41:29.057968 (+ 742us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:41:29.058205 (+ 237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:41:29.058268 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:41:29.058432 (+ 164us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0810 04:41:29.470765 (+412333us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0810 04:41:29.470966 (+ 201us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:59266: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"server-negotiator.queue_time_us":22}
W20260810 04:41:29.471136 322 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 04:41:29.056243 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.126:45969 (local address 127.0.0.1:59266)
0810 04:41:29.056394 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0810 04:41:29.056404 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0810 04:41:29.056470 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 04:41:29.056615 (+ 145us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 04:41:29.056615 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 04:41:29.056617 (+ 2us) client_negotiation.cc:189] Negotiated authn=JWT
0810 04:41:29.056669 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:41:29.056672 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:41:29.057301 (+ 629us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:41:29.057302 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:41:29.057889 (+ 587us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:41:29.057890 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:41:29.058282 (+ 392us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:41:29.058283 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:41:29.058344 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:41:29.058350 (+ 6us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0810 04:41:29.470916 (+412566us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0810 04:41:29.471037 (+ 121us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.22.167.126:45969: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":84,"threads_started":1}
W20260810 04:41:29.471148 311 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:59266: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
I20260810 04:41:29.471482 23197 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskemv_AM/build/release/bin/kudu with pid 32633
2026-08-10T04:41:29Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS (596 ms)
[ RUN ] SecurityITest.TestWorldReadableKeytab
[ OK ] SecurityITest.TestWorldReadableKeytab (88 ms)
[ RUN ] SecurityITest.TestWorldReadablePrivateKey
[ OK ] SecurityITest.TestWorldReadablePrivateKey (87 ms)
[ RUN ] SecurityITest.TestCorruptKerberosCC
Loading random data
Initializing database '/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786336863388938-23197-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 04:41:29 dist-test-slave-bqcl krb5kdc[341](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 04:41:29 dist-test-slave-bqcl krb5kdc[341](info): set up 2 sockets
Aug 10 04:41:29 dist-test-slave-bqcl krb5kdc[341](info): commencing operation
krb5kdc: starting...
W20260810 04:41:30.794486 23197 mini_kdc.cc:121] Time spent starting KDC: real 1.125s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786336863388938-23197-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786336863388938-23197-0/krb5kdc/test-user.keytab.
Aug 10 04:41:30 dist-test-slave-bqcl krb5kdc[341](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786336890, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T04:41:30Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T04:41:30Z Disabled control of system clock
WARNING: no policy specified for kudu/127.22.167.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.167.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.167.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786336863388938-23197-0/krb5kdc/kudu_127.22.167.126.keytab.
Entry for principal kudu/127.22.167.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786336863388938-23197-0/krb5kdc/kudu_127.22.167.126.keytab.
I20260810 04:41:30.905154 23197 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskemv_AM/build/release/bin/kudu
/tmp/dist-test-taskemv_AM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786336863388938-23197-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786336863388938-23197-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786336863388938-23197-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786336863388938-23197-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.167.126:46065
--webserver_interface=127.22.167.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786336863388938-23197-0/krb5kdc/kudu_127.22.167.126.keytab
--principal=kudu/127.22.167.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.167.84:36665
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.167.126:46065
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786336863388938-23197-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786336863388938-23197-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786336863388938-23197-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 04:41:30.983891 360 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 04:41:30.984082 360 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 04:41:30.984104 360 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 04:41:30.985227 360 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 04:41:30.985285 360 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 04:41:30.985298 360 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 04:41:30.985322 360 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 04:41:30.985345 360 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 04:41:30.986799 360 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.167.84:36665
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786336863388938-23197-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786336863388938-23197-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.22.167.126:46065
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786336863388938-23197-0/krb5kdc/kudu_127.22.167.126.keytab
--principal=kudu/127.22.167.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.167.126:46065
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786336863388938-23197-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.167.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.360
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786336863388938-23197-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 03:54:01 UTC on ccycloud.quasar-icijmt.root.comops.site
build id 4577009
I20260810 04:41:30.987017 360 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 04:41:30.987214 360 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 04:41:30.991724 381 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 04:41:30.991755 379 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 04:41:30.991737 377 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 04:41:30.991875 360 server_base.cc:1034] running on GCE node
I20260810 04:41:30.992201 360 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 04:41:30.992504 360 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 04:41:30.993667 360 hybrid_clock.cc:648] HybridClock initialized: now 1786336890993646 us; error 33 us; skew 500 ppm
Aug 10 04:41:30 dist-test-slave-bqcl krb5kdc[341](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786336890, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.167.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 04:41:30.998569 360 init.cc:377] Logged in from keytab as kudu/127.22.167.126@KRBTEST.COM (short username kudu)
I20260810 04:41:30.998997 360 webserver.cc:460] Webserver started at http://127.22.167.126:40429/ using document root <none> and password file <none>
I20260810 04:41:30.999209 360 fs_manager.cc:359] Metadata directory not provided
I20260810 04:41:30.999257 360 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 04:41:30.999383 360 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 04:41:31.000366 360 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786336863388938-23197-0/minicluster-data/master-0/data/instance:
uuid: "d3a786daf63d4b7795fc271d712bdba5"
format_stamp: "Formatted at 2026-08-10 04:41:30 on dist-test-slave-bqcl"
server_key: "16c60d6bd94323a90e70b4192f3ddf1f"
server_key_iv: "27f75efc96dd5b360cab1192e2119fc3"
server_key_version: "encryptionkey@0"
I20260810 04:41:31.000692 360 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786336863388938-23197-0/minicluster-data/master-0/wal/instance:
uuid: "d3a786daf63d4b7795fc271d712bdba5"
format_stamp: "Formatted at 2026-08-10 04:41:30 on dist-test-slave-bqcl"
server_key: "16c60d6bd94323a90e70b4192f3ddf1f"
server_key_iv: "27f75efc96dd5b360cab1192e2119fc3"
server_key_version: "encryptionkey@0"
I20260810 04:41:31.001888 360 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260810 04:41:31.002576 394 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 04:41:31.002735 360 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260810 04:41:31.002807 360 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786336863388938-23197-0/minicluster-data/master-0/data,/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786336863388938-23197-0/minicluster-data/master-0/wal
uuid: "d3a786daf63d4b7795fc271d712bdba5"
format_stamp: "Formatted at 2026-08-10 04:41:30 on dist-test-slave-bqcl"
server_key: "16c60d6bd94323a90e70b4192f3ddf1f"
server_key_iv: "27f75efc96dd5b360cab1192e2119fc3"
server_key_version: "encryptionkey@0"
I20260810 04:41:31.002872 360 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786336863388938-23197-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786336863388938-23197-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786336863388938-23197-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 04:41:31.053462 360 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 04:41:31.055280 360 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 04:41:31.055423 360 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 04:41:31.059257 360 rpc_server.cc:307] RPC server started. Bound to: 127.22.167.126:46065
I20260810 04:41:31.059307 484 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.167.126:46065 every 8 connection(s)
I20260810 04:41:31.059646 360 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786336863388938-23197-0/minicluster-data/master-0/data/info.pb
I20260810 04:41:31.060549 485 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 04:41:31.062888 485 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d3a786daf63d4b7795fc271d712bdba5: Bootstrap starting.
I20260810 04:41:31.063427 485 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d3a786daf63d4b7795fc271d712bdba5: Neither blocks nor log segments found. Creating new log.
I20260810 04:41:31.063680 485 log.cc:824] T 00000000000000000000000000000000 P d3a786daf63d4b7795fc271d712bdba5: Log is configured to *not* fsync() on all Append() calls
I20260810 04:41:31.064316 485 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d3a786daf63d4b7795fc271d712bdba5: No bootstrap required, opened a new log
I20260810 04:41:31.065894 485 raft_consensus.cc:348] T 00000000000000000000000000000000 P d3a786daf63d4b7795fc271d712bdba5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d3a786daf63d4b7795fc271d712bdba5" member_type: VOTER last_known_addr { host: "127.22.167.126" port: 46065 } }
I20260810 04:41:31.066030 485 raft_consensus.cc:374] T 00000000000000000000000000000000 P d3a786daf63d4b7795fc271d712bdba5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 04:41:31.066066 485 raft_consensus.cc:729] T 00000000000000000000000000000000 P d3a786daf63d4b7795fc271d712bdba5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d3a786daf63d4b7795fc271d712bdba5, State: Initialized, Role: FOLLOWER
I20260810 04:41:31.066208 485 consensus_queue.cc:260] T 00000000000000000000000000000000 P d3a786daf63d4b7795fc271d712bdba5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d3a786daf63d4b7795fc271d712bdba5" member_type: VOTER last_known_addr { host: "127.22.167.126" port: 46065 } }
I20260810 04:41:31.066287 485 raft_consensus.cc:388] T 00000000000000000000000000000000 P d3a786daf63d4b7795fc271d712bdba5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 04:41:31.066315 485 raft_consensus.cc:482] T 00000000000000000000000000000000 P d3a786daf63d4b7795fc271d712bdba5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 04:41:31.066349 485 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d3a786daf63d4b7795fc271d712bdba5 [term 0 FOLLOWER]: Advancing to term 1
I20260810 04:41:31.066800 485 raft_consensus.cc:504] T 00000000000000000000000000000000 P d3a786daf63d4b7795fc271d712bdba5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d3a786daf63d4b7795fc271d712bdba5" member_type: VOTER last_known_addr { host: "127.22.167.126" port: 46065 } }
I20260810 04:41:31.066883 485 leader_election.cc:302] T 00000000000000000000000000000000 P d3a786daf63d4b7795fc271d712bdba5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d3a786daf63d4b7795fc271d712bdba5; no voters:
I20260810 04:41:31.067013 485 leader_election.cc:288] T 00000000000000000000000000000000 P d3a786daf63d4b7795fc271d712bdba5 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 04:41:31.067062 493 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d3a786daf63d4b7795fc271d712bdba5 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 04:41:31.067246 485 sys_catalog.cc:564] T 00000000000000000000000000000000 P d3a786daf63d4b7795fc271d712bdba5 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 04:41:31.067399 493 raft_consensus.cc:686] T 00000000000000000000000000000000 P d3a786daf63d4b7795fc271d712bdba5 [term 1 LEADER]: Becoming Leader. State: Replica: d3a786daf63d4b7795fc271d712bdba5, State: Running, Role: LEADER
I20260810 04:41:31.067493 493 consensus_queue.cc:237] T 00000000000000000000000000000000 P d3a786daf63d4b7795fc271d712bdba5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d3a786daf63d4b7795fc271d712bdba5" member_type: VOTER last_known_addr { host: "127.22.167.126" port: 46065 } }
I20260810 04:41:31.067867 496 sys_catalog.cc:455] T 00000000000000000000000000000000 P d3a786daf63d4b7795fc271d712bdba5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d3a786daf63d4b7795fc271d712bdba5. Latest consensus state: current_term: 1 leader_uuid: "d3a786daf63d4b7795fc271d712bdba5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d3a786daf63d4b7795fc271d712bdba5" member_type: VOTER last_known_addr { host: "127.22.167.126" port: 46065 } } }
I20260810 04:41:31.067875 495 sys_catalog.cc:455] T 00000000000000000000000000000000 P d3a786daf63d4b7795fc271d712bdba5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d3a786daf63d4b7795fc271d712bdba5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d3a786daf63d4b7795fc271d712bdba5" member_type: VOTER last_known_addr { host: "127.22.167.126" port: 46065 } } }
I20260810 04:41:31.067984 496 sys_catalog.cc:458] T 00000000000000000000000000000000 P d3a786daf63d4b7795fc271d712bdba5 [sys.catalog]: This master's current role is: LEADER
I20260810 04:41:31.067991 495 sys_catalog.cc:458] T 00000000000000000000000000000000 P d3a786daf63d4b7795fc271d712bdba5 [sys.catalog]: This master's current role is: LEADER
I20260810 04:41:31.068194 519 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 04:41:31.068750 519 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 04:41:31.069653 23197 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskemv_AM/build/release/bin/kudu as pid 360
I20260810 04:41:31.069757 23197 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786336863388938-23197-0/minicluster-data/master-0/wal/instance
I20260810 04:41:31.069947 23197 external_mini_cluster.cc:1428] Setting key 3cec2741f3690983245a9e330517f535
I20260810 04:41:31.070593 519 catalog_manager.cc:1347] Generated new cluster ID: a5398398545f4a88958b4d630f1e348b
I20260810 04:41:31.070645 519 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 10 04:41:31 dist-test-slave-bqcl krb5kdc[341](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786336890, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.167.126@KRBTEST.COM
I20260810 04:41:31.081064 535 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:41:31.070395 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53532 (local address 127.22.167.126:46065)
0810 04:41:31.070535 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0810 04:41:31.070545 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 04:41:31.070746 (+ 201us) server_negotiation.cc:407] Connection header received
0810 04:41:31.070885 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:41:31.070891 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:41:31.070942 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:41:31.071078 (+ 136us) server_negotiation.cc:226] Negotiated authn=SASL
0810 04:41:31.071822 (+ 744us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:41:31.073240 (+ 1418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:41:31.074318 (+ 1078us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:41:31.074904 (+ 586us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:41:31.074996 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:41:31.077662 (+ 2666us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 04:41:31.077666 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 04:41:31.077666 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 04:41:31.077678 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0810 04:41:31.079559 (+ 1881us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 04:41:31.079910 (+ 351us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 04:41:31.079913 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 04:41:31.079913 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 04:41:31.080017 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 04:41:31.080187 (+ 170us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 04:41:31.080187 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 04:41:31.080188 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 04:41:31.080480 (+ 292us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 04:41:31.080617 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0810 04:41:31.080687 (+ 70us) server_negotiation.cc:299] Negotiation successful
0810 04:41:31.080830 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":94,"thread_start_us":68,"threads_started":1}
I20260810 04:41:31.084802 519 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 04:41:31.085299 519 catalog_manager.cc:1504] Loading token signing keys...
I20260810 04:41:31.090752 519 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d3a786daf63d4b7795fc271d712bdba5: Generated new TSK 0
I20260810 04:41:31.090888 519 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.22.167.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.167.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.167.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786336863388938-23197-0/krb5kdc/kudu_127.22.167.65.keytab.
Entry for principal kudu/127.22.167.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786336863388938-23197-0/krb5kdc/kudu_127.22.167.65.keytab.
I20260810 04:41:31.124408 23197 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskemv_AM/build/release/bin/kudu
/tmp/dist-test-taskemv_AM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786336863388938-23197-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786336863388938-23197-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786336863388938-23197-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786336863388938-23197-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.167.65:0
--local_ip_for_outbound_sockets=127.22.167.65
--webserver_interface=127.22.167.65
--webserver_port=0
--tserver_master_addrs=127.22.167.126:46065
--keytab_file=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786336863388938-23197-0/krb5kdc/kudu_127.22.167.65.keytab
--principal=kudu/127.22.167.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.167.84:36665
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786336863388938-23197-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786336863388938-23197-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786336863388938-23197-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 04:41:31.230716 540 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 04:41:31.230906 540 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 04:41:31.230927 540 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 04:41:31.232033 540 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 04:41:31.232074 540 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 04:41:31.232102 540 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.167.65
I20260810 04:41:31.233564 540 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.167.84:36665
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786336863388938-23197-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786336863388938-23197-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786336863388938-23197-0/krb5kdc/kudu_127.22.167.65.keytab
--principal=kudu/127.22.167.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.167.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786336863388938-23197-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.22.167.65
--webserver_port=0
--tserver_master_addrs=127.22.167.126:46065
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.540
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.167.65
--log_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786336863388938-23197-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 03:54:01 UTC on ccycloud.quasar-icijmt.root.comops.site
build id 4577009
I20260810 04:41:31.233780 540 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 04:41:31.233978 540 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 04:41:31.238675 550 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 04:41:31.238643 548 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 04:41:31.238686 540 server_base.cc:1034] running on GCE node
W20260810 04:41:31.238652 547 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 04:41:31.239065 540 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 04:41:31.239301 540 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 04:41:31.240432 540 hybrid_clock.cc:648] HybridClock initialized: now 1786336891240416 us; error 35 us; skew 500 ppm
Aug 10 04:41:31 dist-test-slave-bqcl krb5kdc[341](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786336891, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.167.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 04:41:31.242806 540 init.cc:377] Logged in from keytab as kudu/127.22.167.65@KRBTEST.COM (short username kudu)
I20260810 04:41:31.243189 540 webserver.cc:460] Webserver started at http://127.22.167.65:40443/ using document root <none> and password file <none>
I20260810 04:41:31.243404 540 fs_manager.cc:359] Metadata directory not provided
I20260810 04:41:31.243450 540 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 04:41:31.243573 540 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 04:41:31.244525 540 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786336863388938-23197-0/minicluster-data/ts-0/data/instance:
uuid: "fb758df9ee9b40fb96885b19c2516157"
format_stamp: "Formatted at 2026-08-10 04:41:31 on dist-test-slave-bqcl"
server_key: "1e6e178f7c41a4d6a486ee2601d91be8"
server_key_iv: "41068547dc2ca1b874ef3e90a56a16b0"
server_key_version: "encryptionkey@0"
I20260810 04:41:31.244848 540 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786336863388938-23197-0/minicluster-data/ts-0/wal/instance:
uuid: "fb758df9ee9b40fb96885b19c2516157"
format_stamp: "Formatted at 2026-08-10 04:41:31 on dist-test-slave-bqcl"
server_key: "1e6e178f7c41a4d6a486ee2601d91be8"
server_key_iv: "41068547dc2ca1b874ef3e90a56a16b0"
server_key_version: "encryptionkey@0"
I20260810 04:41:31.246069 540 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260810 04:41:31.246797 562 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 04:41:31.246968 540 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260810 04:41:31.247040 540 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786336863388938-23197-0/minicluster-data/ts-0/data,/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786336863388938-23197-0/minicluster-data/ts-0/wal
uuid: "fb758df9ee9b40fb96885b19c2516157"
format_stamp: "Formatted at 2026-08-10 04:41:31 on dist-test-slave-bqcl"
server_key: "1e6e178f7c41a4d6a486ee2601d91be8"
server_key_iv: "41068547dc2ca1b874ef3e90a56a16b0"
server_key_version: "encryptionkey@0"
I20260810 04:41:31.247102 540 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786336863388938-23197-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786336863388938-23197-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786336863388938-23197-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 04:41:31.267390 540 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 04:41:31.269439 540 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 04:41:31.269575 540 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 04:41:31.269795 540 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 04:41:31.270094 540 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 04:41:31.270152 540 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 04:41:31.270192 540 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 04:41:31.270224 540 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 04:41:31.275631 540 rpc_server.cc:307] RPC server started. Bound to: 127.22.167.65:35345
I20260810 04:41:31.275693 745 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.167.65:35345 every 8 connection(s)
I20260810 04:41:31.275997 540 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786336863388938-23197-0/minicluster-data/ts-0/data/info.pb
I20260810 04:41:31.279718 23197 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskemv_AM/build/release/bin/kudu as pid 540
I20260810 04:41:31.279826 23197 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786336863388938-23197-0/minicluster-data/ts-0/wal/instance
I20260810 04:41:31.279983 23197 external_mini_cluster.cc:1428] Setting key 34443da5566b8efc8eacc40c2bf331c2
Aug 10 04:41:31 dist-test-slave-bqcl krb5kdc[341](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786336891, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.167.65@KRBTEST.COM for kudu/127.22.167.126@KRBTEST.COM
I20260810 04:41:31.285106 535 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:41:31.276688 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.167.65:49703 (local address 127.22.167.126:46065)
0810 04:41:31.276748 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0810 04:41:31.276750 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 04:41:31.276927 (+ 177us) server_negotiation.cc:407] Connection header received
0810 04:41:31.277847 (+ 920us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:41:31.277848 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:41:31.277866 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:41:31.277916 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0810 04:41:31.278544 (+ 628us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:41:31.279043 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:41:31.280042 (+ 999us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:41:31.280335 (+ 293us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:41:31.280392 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:41:31.282568 (+ 2176us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 04:41:31.282569 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 04:41:31.282570 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 04:41:31.282580 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0810 04:41:31.283876 (+ 1296us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 04:41:31.284238 (+ 362us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 04:41:31.284239 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 04:41:31.284239 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 04:41:31.284291 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 04:41:31.284503 (+ 212us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 04:41:31.284504 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 04:41:31.284504 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 04:41:31.284616 (+ 112us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 04:41:31.284671 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0810 04:41:31.284934 (+ 263us) server_negotiation.cc:299] Negotiation successful
0810 04:41:31.285021 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260810 04:41:31.285369 749 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:41:31.276669 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.126:46065 (local address 127.22.167.65:49703)
0810 04:41:31.276811 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0810 04:41:31.276846 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0810 04:41:31.277739 (+ 893us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 04:41:31.277986 (+ 247us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 04:41:31.277989 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 04:41:31.278265 (+ 276us) client_negotiation.cc:189] Negotiated authn=SASL
0810 04:41:31.278448 (+ 183us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:41:31.278452 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:41:31.279127 (+ 675us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:41:31.279128 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:41:31.279961 (+ 833us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:41:31.279964 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:41:31.280412 (+ 448us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:41:31.280413 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:41:31.280529 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:41:31.281000 (+ 471us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 04:41:31.281007 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0810 04:41:31.282474 (+ 1467us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 04:41:31.283949 (+ 1475us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 04:41:31.283956 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 04:41:31.283957 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 04:41:31.284160 (+ 203us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 04:41:31.284345 (+ 185us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 04:41:31.284346 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 04:41:31.284346 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 04:41:31.284447 (+ 101us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 04:41:31.284672 (+ 225us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 04:41:31.284679 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 04:41:31.284816 (+ 137us) client_negotiation.cc:769] Sending connection context
0810 04:41:31.284903 (+ 87us) client_negotiation.cc:240] Negotiation successful
0810 04:41:31.285049 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":106,"thread_start_us":75,"threads_started":1}
I20260810 04:41:31.285789 746 heartbeater.cc:347] Connected to a master server at 127.22.167.126:46065
I20260810 04:41:31.285871 746 heartbeater.cc:464] Registering TS with master...
I20260810 04:41:31.286034 746 heartbeater.cc:511] Master 127.22.167.126:46065 requested a full tablet report, sending...
I20260810 04:41:31.286607 423 ts_manager.cc:194] Registered new tserver with Master: fb758df9ee9b40fb96885b19c2516157 (127.22.167.65:35345)
I20260810 04:41:31.287500 423 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.167.65@KRBTEST.COM'} at 127.22.167.65:49703
WARNING: no policy specified for kudu/127.22.167.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.167.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.167.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786336863388938-23197-0/krb5kdc/kudu_127.22.167.66.keytab.
Entry for principal kudu/127.22.167.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786336863388938-23197-0/krb5kdc/kudu_127.22.167.66.keytab.
I20260810 04:41:31.306331 23197 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskemv_AM/build/release/bin/kudu
/tmp/dist-test-taskemv_AM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786336863388938-23197-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786336863388938-23197-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786336863388938-23197-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786336863388938-23197-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.167.66:0
--local_ip_for_outbound_sockets=127.22.167.66
--webserver_interface=127.22.167.66
--webserver_port=0
--tserver_master_addrs=127.22.167.126:46065
--keytab_file=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786336863388938-23197-0/krb5kdc/kudu_127.22.167.66.keytab
--principal=kudu/127.22.167.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.167.84:36665
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786336863388938-23197-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786336863388938-23197-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786336863388938-23197-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 04:41:31.385545 752 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 04:41:31.385735 752 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 04:41:31.385756 752 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 04:41:31.386837 752 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 04:41:31.386869 752 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 04:41:31.386893 752 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.167.66
I20260810 04:41:31.388249 752 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.167.84:36665
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786336863388938-23197-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786336863388938-23197-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786336863388938-23197-0/krb5kdc/kudu_127.22.167.66.keytab
--principal=kudu/127.22.167.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.167.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786336863388938-23197-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.22.167.66
--webserver_port=0
--tserver_master_addrs=127.22.167.126:46065
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.752
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.167.66
--log_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786336863388938-23197-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 03:54:01 UTC on ccycloud.quasar-icijmt.root.comops.site
build id 4577009
I20260810 04:41:31.388417 752 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 04:41:31.388629 752 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 04:41:31.393244 759 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 04:41:31.393314 752 server_base.cc:1034] running on GCE node
W20260810 04:41:31.393268 760 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 04:41:31.393266 762 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 04:41:31.393704 752 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 04:41:31.393926 752 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 04:41:31.395061 752 hybrid_clock.cc:648] HybridClock initialized: now 1786336891395048 us; error 29 us; skew 500 ppm
Aug 10 04:41:31 dist-test-slave-bqcl krb5kdc[341](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786336891, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.167.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 04:41:31.397307 752 init.cc:377] Logged in from keytab as kudu/127.22.167.66@KRBTEST.COM (short username kudu)
I20260810 04:41:31.397675 752 webserver.cc:460] Webserver started at http://127.22.167.66:43285/ using document root <none> and password file <none>
I20260810 04:41:31.397837 752 fs_manager.cc:359] Metadata directory not provided
I20260810 04:41:31.397888 752 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 04:41:31.397979 752 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 04:41:31.398926 752 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786336863388938-23197-0/minicluster-data/ts-1/data/instance:
uuid: "7e062f62e9234d67a997c66a6ecce612"
format_stamp: "Formatted at 2026-08-10 04:41:31 on dist-test-slave-bqcl"
server_key: "c4a7e7011e3b0402e975f9d1a329a232"
server_key_iv: "33137600bcf6b096992e4a8d49609292"
server_key_version: "encryptionkey@0"
I20260810 04:41:31.399216 752 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786336863388938-23197-0/minicluster-data/ts-1/wal/instance:
uuid: "7e062f62e9234d67a997c66a6ecce612"
format_stamp: "Formatted at 2026-08-10 04:41:31 on dist-test-slave-bqcl"
server_key: "c4a7e7011e3b0402e975f9d1a329a232"
server_key_iv: "33137600bcf6b096992e4a8d49609292"
server_key_version: "encryptionkey@0"
I20260810 04:41:31.400368 752 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260810 04:41:31.400998 776 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 04:41:31.401136 752 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260810 04:41:31.401216 752 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786336863388938-23197-0/minicluster-data/ts-1/data,/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786336863388938-23197-0/minicluster-data/ts-1/wal
uuid: "7e062f62e9234d67a997c66a6ecce612"
format_stamp: "Formatted at 2026-08-10 04:41:31 on dist-test-slave-bqcl"
server_key: "c4a7e7011e3b0402e975f9d1a329a232"
server_key_iv: "33137600bcf6b096992e4a8d49609292"
server_key_version: "encryptionkey@0"
I20260810 04:41:31.401268 752 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786336863388938-23197-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786336863388938-23197-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786336863388938-23197-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 04:41:31.409027 752 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 04:41:31.410645 752 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 04:41:31.410754 752 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 04:41:31.410907 752 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 04:41:31.411129 752 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 04:41:31.411158 752 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 04:41:31.411176 752 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 04:41:31.411193 752 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 04:41:31.416355 752 rpc_server.cc:307] RPC server started. Bound to: 127.22.167.66:33909
I20260810 04:41:31.416416 956 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.167.66:33909 every 8 connection(s)
I20260810 04:41:31.416652 752 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786336863388938-23197-0/minicluster-data/ts-1/data/info.pb
I20260810 04:41:31.420876 23197 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskemv_AM/build/release/bin/kudu as pid 752
I20260810 04:41:31.420960 23197 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786336863388938-23197-0/minicluster-data/ts-1/wal/instance
I20260810 04:41:31.421134 23197 external_mini_cluster.cc:1428] Setting key ee8dcd2b34112e28c35fd3fb89038818
Aug 10 04:41:31 dist-test-slave-bqcl krb5kdc[341](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786336891, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.167.66@KRBTEST.COM for kudu/127.22.167.126@KRBTEST.COM
I20260810 04:41:31.425482 535 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:41:31.417270 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.167.66:38391 (local address 127.22.167.126:46065)
0810 04:41:31.417331 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0810 04:41:31.417333 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 04:41:31.417500 (+ 167us) server_negotiation.cc:407] Connection header received
0810 04:41:31.418417 (+ 917us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:41:31.418418 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:41:31.418431 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:41:31.418482 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0810 04:41:31.419060 (+ 578us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:41:31.419564 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:41:31.420503 (+ 939us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:41:31.420718 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:41:31.420782 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:41:31.422947 (+ 2165us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 04:41:31.422948 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 04:41:31.422948 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 04:41:31.422955 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 04:41:31.424247 (+ 1292us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 04:41:31.424599 (+ 352us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 04:41:31.424600 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 04:41:31.424600 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 04:41:31.424662 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 04:41:31.424911 (+ 249us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 04:41:31.424912 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 04:41:31.424912 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 04:41:31.425011 (+ 99us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 04:41:31.425055 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0810 04:41:31.425355 (+ 300us) server_negotiation.cc:299] Negotiation successful
0810 04:41:31.425421 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260810 04:41:31.425681 960 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:41:31.417246 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.126:46065 (local address 127.22.167.66:38391)
0810 04:41:31.417398 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0810 04:41:31.417430 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0810 04:41:31.418314 (+ 884us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 04:41:31.418554 (+ 240us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 04:41:31.418558 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 04:41:31.418793 (+ 235us) client_negotiation.cc:189] Negotiated authn=SASL
0810 04:41:31.418959 (+ 166us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:41:31.418963 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:41:31.419646 (+ 683us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:41:31.419647 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:41:31.420414 (+ 767us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:41:31.420416 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:41:31.420812 (+ 396us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:41:31.420814 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:41:31.420923 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:41:31.421408 (+ 485us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 04:41:31.421418 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0810 04:41:31.422860 (+ 1442us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 04:41:31.424321 (+ 1461us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 04:41:31.424327 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 04:41:31.424329 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 04:41:31.424533 (+ 204us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 04:41:31.424723 (+ 190us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 04:41:31.424724 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 04:41:31.424724 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 04:41:31.424850 (+ 126us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 04:41:31.425065 (+ 215us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 04:41:31.425071 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 04:41:31.425216 (+ 145us) client_negotiation.cc:769] Sending connection context
0810 04:41:31.425334 (+ 118us) client_negotiation.cc:240] Negotiation successful
0810 04:41:31.425480 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":77,"threads_started":1}
I20260810 04:41:31.426199 957 heartbeater.cc:347] Connected to a master server at 127.22.167.126:46065
I20260810 04:41:31.426298 957 heartbeater.cc:464] Registering TS with master...
I20260810 04:41:31.426474 957 heartbeater.cc:511] Master 127.22.167.126:46065 requested a full tablet report, sending...
I20260810 04:41:31.426846 423 ts_manager.cc:194] Registered new tserver with Master: 7e062f62e9234d67a997c66a6ecce612 (127.22.167.66:33909)
I20260810 04:41:31.427341 423 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.167.66@KRBTEST.COM'} at 127.22.167.66:38391
WARNING: no policy specified for kudu/127.22.167.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.167.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.167.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786336863388938-23197-0/krb5kdc/kudu_127.22.167.67.keytab.
Entry for principal kudu/127.22.167.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786336863388938-23197-0/krb5kdc/kudu_127.22.167.67.keytab.
I20260810 04:41:31.445554 23197 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskemv_AM/build/release/bin/kudu
/tmp/dist-test-taskemv_AM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786336863388938-23197-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786336863388938-23197-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786336863388938-23197-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786336863388938-23197-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.167.67:0
--local_ip_for_outbound_sockets=127.22.167.67
--webserver_interface=127.22.167.67
--webserver_port=0
--tserver_master_addrs=127.22.167.126:46065
--keytab_file=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786336863388938-23197-0/krb5kdc/kudu_127.22.167.67.keytab
--principal=kudu/127.22.167.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.167.84:36665
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786336863388938-23197-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786336863388938-23197-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786336863388938-23197-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 04:41:31.524283 966 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 04:41:31.524482 966 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 04:41:31.524502 966 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 04:41:31.525597 966 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 04:41:31.525631 966 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 04:41:31.525655 966 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.167.67
I20260810 04:41:31.527079 966 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.167.84:36665
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786336863388938-23197-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786336863388938-23197-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786336863388938-23197-0/krb5kdc/kudu_127.22.167.67.keytab
--principal=kudu/127.22.167.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.167.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786336863388938-23197-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.22.167.67
--webserver_port=0
--tserver_master_addrs=127.22.167.126:46065
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.966
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.167.67
--log_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786336863388938-23197-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 03:54:01 UTC on ccycloud.quasar-icijmt.root.comops.site
build id 4577009
I20260810 04:41:31.527262 966 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 04:41:31.527431 966 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 04:41:31.531973 976 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 04:41:31.531980 974 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 04:41:31.532020 979 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 04:41:31.532213 966 server_base.cc:1034] running on GCE node
I20260810 04:41:31.532509 966 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 04:41:31.532712 966 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 04:41:31.533843 966 hybrid_clock.cc:648] HybridClock initialized: now 1786336891533831 us; error 27 us; skew 500 ppm
Aug 10 04:41:31 dist-test-slave-bqcl krb5kdc[341](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786336891, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.167.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 04:41:31.536211 966 init.cc:377] Logged in from keytab as kudu/127.22.167.67@KRBTEST.COM (short username kudu)
I20260810 04:41:31.536576 966 webserver.cc:460] Webserver started at http://127.22.167.67:41019/ using document root <none> and password file <none>
I20260810 04:41:31.536772 966 fs_manager.cc:359] Metadata directory not provided
I20260810 04:41:31.536820 966 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 04:41:31.536940 966 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 04:41:31.537874 966 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786336863388938-23197-0/minicluster-data/ts-2/data/instance:
uuid: "bfbd9fbe114a47b8aba80b8650e118a0"
format_stamp: "Formatted at 2026-08-10 04:41:31 on dist-test-slave-bqcl"
server_key: "93302dfc6e4174d245c43dbc638e7066"
server_key_iv: "9c4050b8f90fc30f2b2fb2fdda1a115b"
server_key_version: "encryptionkey@0"
I20260810 04:41:31.538249 966 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786336863388938-23197-0/minicluster-data/ts-2/wal/instance:
uuid: "bfbd9fbe114a47b8aba80b8650e118a0"
format_stamp: "Formatted at 2026-08-10 04:41:31 on dist-test-slave-bqcl"
server_key: "93302dfc6e4174d245c43dbc638e7066"
server_key_iv: "9c4050b8f90fc30f2b2fb2fdda1a115b"
server_key_version: "encryptionkey@0"
I20260810 04:41:31.539474 966 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260810 04:41:31.540105 989 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 04:41:31.540254 966 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260810 04:41:31.540334 966 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786336863388938-23197-0/minicluster-data/ts-2/data,/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786336863388938-23197-0/minicluster-data/ts-2/wal
uuid: "bfbd9fbe114a47b8aba80b8650e118a0"
format_stamp: "Formatted at 2026-08-10 04:41:31 on dist-test-slave-bqcl"
server_key: "93302dfc6e4174d245c43dbc638e7066"
server_key_iv: "9c4050b8f90fc30f2b2fb2fdda1a115b"
server_key_version: "encryptionkey@0"
I20260810 04:41:31.540397 966 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786336863388938-23197-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786336863388938-23197-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786336863388938-23197-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 04:41:31.552381 966 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 04:41:31.554055 966 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 04:41:31.554206 966 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 04:41:31.554394 966 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 04:41:31.554661 966 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 04:41:31.554693 966 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 04:41:31.554718 966 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 04:41:31.554734 966 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 04:41:31.559939 966 rpc_server.cc:307] RPC server started. Bound to: 127.22.167.67:39007
I20260810 04:41:31.559998 1172 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.167.67:39007 every 8 connection(s)
I20260810 04:41:31.560262 966 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786336863388938-23197-0/minicluster-data/ts-2/data/info.pb
Aug 10 04:41:31 dist-test-slave-bqcl krb5kdc[341](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786336891, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.167.67@KRBTEST.COM for kudu/127.22.167.126@KRBTEST.COM
I20260810 04:41:31.568838 535 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:41:31.560912 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.167.67:58115 (local address 127.22.167.126:46065)
0810 04:41:31.560957 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0810 04:41:31.560959 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 04:41:31.561123 (+ 164us) server_negotiation.cc:407] Connection header received
0810 04:41:31.562060 (+ 937us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:41:31.562062 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:41:31.562073 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:41:31.562169 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0810 04:41:31.562729 (+ 560us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:41:31.563227 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:41:31.564137 (+ 910us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:41:31.564352 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:41:31.564408 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:41:31.566285 (+ 1877us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 04:41:31.566286 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 04:41:31.566286 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 04:41:31.566293 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 04:41:31.567566 (+ 1273us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 04:41:31.567955 (+ 389us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 04:41:31.567956 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 04:41:31.567956 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 04:41:31.568005 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 04:41:31.568302 (+ 297us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 04:41:31.568304 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 04:41:31.568304 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 04:41:31.568425 (+ 121us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 04:41:31.568495 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0810 04:41:31.568715 (+ 220us) server_negotiation.cc:299] Negotiation successful
0810 04:41:31.568777 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260810 04:41:31.568997 1177 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:41:31.560877 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.126:46065 (local address 127.22.167.67:58115)
0810 04:41:31.561021 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0810 04:41:31.561061 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0810 04:41:31.561948 (+ 887us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 04:41:31.562226 (+ 278us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 04:41:31.562229 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 04:41:31.562461 (+ 232us) client_negotiation.cc:189] Negotiated authn=SASL
0810 04:41:31.562636 (+ 175us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:41:31.562640 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:41:31.563294 (+ 654us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:41:31.563294 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:41:31.564049 (+ 755us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:41:31.564050 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:41:31.564415 (+ 365us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:41:31.564415 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:41:31.564507 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:41:31.564878 (+ 371us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 04:41:31.564886 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0810 04:41:31.566205 (+ 1319us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 04:41:31.567636 (+ 1431us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 04:41:31.567642 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 04:41:31.567643 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 04:41:31.567881 (+ 238us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 04:41:31.568055 (+ 174us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 04:41:31.568055 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 04:41:31.568056 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 04:41:31.568178 (+ 122us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 04:41:31.568486 (+ 308us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 04:41:31.568490 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 04:41:31.568612 (+ 122us) client_negotiation.cc:769] Sending connection context
0810 04:41:31.568693 (+ 81us) client_negotiation.cc:240] Negotiation successful
0810 04:41:31.568809 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":63,"threads_started":1}
I20260810 04:41:31.569567 1175 heartbeater.cc:347] Connected to a master server at 127.22.167.126:46065
I20260810 04:41:31.569660 1175 heartbeater.cc:464] Registering TS with master...
I20260810 04:41:31.569720 23197 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskemv_AM/build/release/bin/kudu as pid 966
I20260810 04:41:31.569813 23197 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786336863388938-23197-0/minicluster-data/ts-2/wal/instance
I20260810 04:41:31.569844 1175 heartbeater.cc:511] Master 127.22.167.126:46065 requested a full tablet report, sending...
I20260810 04:41:31.569996 23197 external_mini_cluster.cc:1428] Setting key b91a07d6446b5ef86fee179649a45a4c
I20260810 04:41:31.570232 423 ts_manager.cc:194] Registered new tserver with Master: bfbd9fbe114a47b8aba80b8650e118a0 (127.22.167.67:39007)
I20260810 04:41:31.570581 423 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.167.67@KRBTEST.COM'} at 127.22.167.67:58115
I20260810 04:41:31.570801 23197 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal test-admin with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786336863388938-23197-0/krb5kdc/test-admin.keytab.
Entry for principal test-admin with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786336863388938-23197-0/krb5kdc/test-admin.keytab.
Aug 10 04:41:31 dist-test-slave-bqcl krb5kdc[341](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786336891, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 04:41:31.586606 23197 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
W20260810 04:41:31.588071 1191 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 04:41:31.587409 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57486 (local address 127.22.167.65:35345)
0810 04:41:31.587556 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0810 04:41:31.587561 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 04:41:31.587569 (+ 8us) server_negotiation.cc:407] Connection header received
0810 04:41:31.587592 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:41:31.587593 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:41:31.587606 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:41:31.587669 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0810 04:41:31.587994 (+ 325us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:57486: BlockingRecv error: recv got EOF from 127.0.0.1:57486 (error 108)
Metrics: {"server-negotiator.queue_time_us":118,"thread_start_us":71,"threads_started":1}
W20260810 04:41:31.588145 23197 init.cc:244] Error while opening credential cache '/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786336863388938-23197-0/krb5kdc/krb5cc' of type FILE: Runtime error: End of credential cache reached (filename: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786336863388938-23197-0/krb5kdc/krb5cc)
Aug 10 04:41:31 dist-test-slave-bqcl krb5kdc[341](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786336891, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 04:41:31.589015 23197 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 10 04:41:31 dist-test-slave-bqcl krb5kdc[341](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786336891, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 04:41:31.589812 23197 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 10 04:41:31 dist-test-slave-bqcl krb5kdc[341](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786336891, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.167.65@KRBTEST.COM
I20260810 04:41:31.596722 1191 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:41:31.590304 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57488 (local address 127.22.167.65:35345)
0810 04:41:31.590376 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0810 04:41:31.590378 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 04:41:31.590390 (+ 12us) server_negotiation.cc:407] Connection header received
0810 04:41:31.590411 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:41:31.590411 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:41:31.590421 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:41:31.590473 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0810 04:41:31.590864 (+ 391us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:41:31.591566 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:41:31.592263 (+ 697us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:41:31.592635 (+ 372us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:41:31.592697 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:41:31.594605 (+ 1908us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 04:41:31.594613 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 04:41:31.594614 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 04:41:31.594620 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0810 04:41:31.595847 (+ 1227us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 04:41:31.596125 (+ 278us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 04:41:31.596127 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 04:41:31.596127 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 04:41:31.596178 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 04:41:31.596300 (+ 122us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 04:41:31.596301 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 04:41:31.596301 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 04:41:31.596407 (+ 106us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 04:41:31.596503 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0810 04:41:31.596558 (+ 55us) server_negotiation.cc:299] Negotiation successful
0810 04:41:31.596620 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
W20260810 04:41:31.598348 1191 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 04:41:31.597761 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57504 (local address 127.22.167.65:35345)
0810 04:41:31.597808 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0810 04:41:31.597809 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0810 04:41:31.597817 (+ 8us) server_negotiation.cc:407] Connection header received
0810 04:41:31.597866 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:41:31.597867 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:41:31.597876 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:41:31.597936 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0810 04:41:31.598309 (+ 373us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:57504: BlockingRecv error: recv got EOF from 127.0.0.1:57504 (error 108)
Metrics: {"server-negotiator.queue_time_us":29}
W20260810 04:41:31.598407 23197 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786336863388938-23197-0/krb5kdc/krb5cc' of type FILE
Aug 10 04:41:31 dist-test-slave-bqcl krb5kdc[341](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786336891, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 04:41:31.599190 23197 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 10 04:41:31 dist-test-slave-bqcl krb5kdc[341](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786336891, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 04:41:31.599900 23197 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 10 04:41:31 dist-test-slave-bqcl krb5kdc[341](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786336891, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.167.65@KRBTEST.COM
I20260810 04:41:31.606441 1191 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:41:31.600300 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57520 (local address 127.22.167.65:35345)
0810 04:41:31.600339 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0810 04:41:31.600341 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 04:41:31.600407 (+ 66us) server_negotiation.cc:407] Connection header received
0810 04:41:31.600464 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:41:31.600464 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:41:31.600473 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:41:31.600516 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0810 04:41:31.600824 (+ 308us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:41:31.601331 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:41:31.602013 (+ 682us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:41:31.602271 (+ 258us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:41:31.602329 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:41:31.604212 (+ 1883us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 04:41:31.604213 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 04:41:31.604214 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 04:41:31.604220 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0810 04:41:31.605416 (+ 1196us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 04:41:31.605723 (+ 307us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 04:41:31.605723 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 04:41:31.605724 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 04:41:31.605773 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 04:41:31.605946 (+ 173us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 04:41:31.605947 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 04:41:31.605947 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 04:41:31.606037 (+ 90us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 04:41:31.606116 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0810 04:41:31.606286 (+ 170us) server_negotiation.cc:299] Negotiation successful
0810 04:41:31.606358 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
W20260810 04:41:31.607702 1191 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 04:41:31.607272 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57534 (local address 127.22.167.65:35345)
0810 04:41:31.607303 (+ 31us) server_negotiation.cc:206] Beginning negotiation
0810 04:41:31.607305 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 04:41:31.607339 (+ 34us) server_negotiation.cc:407] Connection header received
0810 04:41:31.607393 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:41:31.607394 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:41:31.607400 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:41:31.607435 (+ 35us) server_negotiation.cc:226] Negotiated authn=SASL
0810 04:41:31.607663 (+ 228us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:57534: BlockingRecv error: recv got EOF from 127.0.0.1:57534 (error 108)
Metrics: {"server-negotiator.queue_time_us":15}
W20260810 04:41:31.607807 23197 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786336863388938-23197-0/krb5kdc/krb5cc' of type FILE
Aug 10 04:41:31 dist-test-slave-bqcl krb5kdc[341](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786336891, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 04:41:31.608649 23197 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 10 04:41:31 dist-test-slave-bqcl krb5kdc[341](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786336891, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 04:41:31.609370 23197 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 10 04:41:31 dist-test-slave-bqcl krb5kdc[341](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786336891, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.167.65@KRBTEST.COM
I20260810 04:41:31.615379 1191 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:41:31.609807 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57548 (local address 127.22.167.65:35345)
0810 04:41:31.609854 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0810 04:41:31.609855 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0810 04:41:31.609866 (+ 11us) server_negotiation.cc:407] Connection header received
0810 04:41:31.609938 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:41:31.609938 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:41:31.609946 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:41:31.609985 (+ 39us) server_negotiation.cc:226] Negotiated authn=SASL
0810 04:41:31.610356 (+ 371us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:41:31.610813 (+ 457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:41:31.611459 (+ 646us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:41:31.611644 (+ 185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:41:31.611694 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:41:31.613387 (+ 1693us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 04:41:31.613388 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 04:41:31.613388 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 04:41:31.613394 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0810 04:41:31.614463 (+ 1069us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 04:41:31.614747 (+ 284us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 04:41:31.614748 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 04:41:31.614749 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 04:41:31.614795 (+ 46us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 04:41:31.614953 (+ 158us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 04:41:31.614954 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 04:41:31.614954 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 04:41:31.615039 (+ 85us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 04:41:31.615114 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0810 04:41:31.615233 (+ 119us) server_negotiation.cc:299] Negotiation successful
0810 04:41:31.615300 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260810 04:41:31.615940 23197 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskemv_AM/build/release/bin/kudu with pid 540
I20260810 04:41:31.628587 23197 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskemv_AM/build/release/bin/kudu with pid 752
I20260810 04:41:31.646086 23197 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskemv_AM/build/release/bin/kudu with pid 966
I20260810 04:41:31.658329 23197 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskemv_AM/build/release/bin/kudu with pid 360
2026-08-10T04:41:31Z chronyd exiting
[ OK ] SecurityITest.TestCorruptKerberosCC (2013 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipalMultipleMaster
Loading random data
Initializing database '/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786336863388938-23197-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 04:41:31 dist-test-slave-bqcl krb5kdc[1217](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 04:41:31 dist-test-slave-bqcl krb5kdc[1217](info): set up 2 sockets
Aug 10 04:41:31 dist-test-slave-bqcl krb5kdc[1217](info): commencing operation
krb5kdc: starting...
W20260810 04:41:32.830363 23197 mini_kdc.cc:121] Time spent starting KDC: real 1.148s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786336863388938-23197-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786336863388938-23197-0/krb5kdc/test-user.keytab.
Aug 10 04:41:32 dist-test-slave-bqcl krb5kdc[1217](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786336892, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T04:41:32Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T04:41:32Z Disabled control of system clock
WARNING: no policy specified for oryx/127.22.167.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.22.167.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.22.167.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786336863388938-23197-0/krb5kdc/oryx_127.22.167.126.keytab.
Entry for principal oryx/127.22.167.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786336863388938-23197-0/krb5kdc/oryx_127.22.167.126.keytab.
I20260810 04:41:32.940739 23197 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskemv_AM/build/release/bin/kudu
/tmp/dist-test-taskemv_AM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786336863388938-23197-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786336863388938-23197-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786336863388938-23197-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786336863388938-23197-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.167.126:41961
--webserver_interface=127.22.167.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786336863388938-23197-0/krb5kdc/oryx_127.22.167.126.keytab
--principal=oryx/127.22.167.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.167.84:33603
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.167.126:41961
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786336863388938-23197-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786336863388938-23197-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786336863388938-23197-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 04:41:33.020033 1252 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 04:41:33.020228 1252 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 04:41:33.020256 1252 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 04:41:33.021334 1252 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 04:41:33.021373 1252 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 04:41:33.021392 1252 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 04:41:33.021406 1252 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 04:41:33.021423 1252 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 04:41:33.022873 1252 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.167.84:33603
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786336863388938-23197-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786336863388938-23197-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.22.167.126:41961
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786336863388938-23197-0/krb5kdc/oryx_127.22.167.126.keytab
--principal=oryx/127.22.167.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.167.126:41961
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786336863388938-23197-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.167.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1252
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786336863388938-23197-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 03:54:01 UTC on ccycloud.quasar-icijmt.root.comops.site
build id 4577009
I20260810 04:41:33.023102 1252 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 04:41:33.023296 1252 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 04:41:33.027841 1263 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 04:41:33.027809 1259 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 04:41:33.027858 1252 server_base.cc:1034] running on GCE node
W20260810 04:41:33.027798 1258 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 04:41:33.028290 1252 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 04:41:33.028565 1252 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 04:41:33.029706 1252 hybrid_clock.cc:648] HybridClock initialized: now 1786336893029681 us; error 31 us; skew 500 ppm
Aug 10 04:41:33 dist-test-slave-bqcl krb5kdc[1217](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786336893, etypes {rep=17 tkt=17 ses=17}, oryx/127.22.167.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 04:41:33.031986 1252 init.cc:377] Logged in from keytab as oryx/127.22.167.126@KRBTEST.COM (short username oryx)
I20260810 04:41:33.032367 1252 webserver.cc:460] Webserver started at http://127.22.167.126:33369/ using document root <none> and password file <none>
I20260810 04:41:33.032549 1252 fs_manager.cc:359] Metadata directory not provided
I20260810 04:41:33.032594 1252 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 04:41:33.032722 1252 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 04:41:33.033685 1252 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786336863388938-23197-0/minicluster-data/master-0/data/instance:
uuid: "b56636c33cce464eba3d4b547a7c704e"
format_stamp: "Formatted at 2026-08-10 04:41:33 on dist-test-slave-bqcl"
server_key: "fb59d9bb04f85faafe6a81d83ac5e760"
server_key_iv: "18a2ad985d7b6258c833987c17653123"
server_key_version: "encryptionkey@0"
I20260810 04:41:33.034013 1252 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786336863388938-23197-0/minicluster-data/master-0/wal/instance:
uuid: "b56636c33cce464eba3d4b547a7c704e"
format_stamp: "Formatted at 2026-08-10 04:41:33 on dist-test-slave-bqcl"
server_key: "fb59d9bb04f85faafe6a81d83ac5e760"
server_key_iv: "18a2ad985d7b6258c833987c17653123"
server_key_version: "encryptionkey@0"
I20260810 04:41:33.035259 1252 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260810 04:41:33.035878 1276 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 04:41:33.036024 1252 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260810 04:41:33.036105 1252 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786336863388938-23197-0/minicluster-data/master-0/data,/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786336863388938-23197-0/minicluster-data/master-0/wal
uuid: "b56636c33cce464eba3d4b547a7c704e"
format_stamp: "Formatted at 2026-08-10 04:41:33 on dist-test-slave-bqcl"
server_key: "fb59d9bb04f85faafe6a81d83ac5e760"
server_key_iv: "18a2ad985d7b6258c833987c17653123"
server_key_version: "encryptionkey@0"
I20260810 04:41:33.036162 1252 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786336863388938-23197-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786336863388938-23197-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786336863388938-23197-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 04:41:33.047634 1252 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 04:41:33.049708 1252 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 04:41:33.049832 1252 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 04:41:33.053532 1252 rpc_server.cc:307] RPC server started. Bound to: 127.22.167.126:41961
I20260810 04:41:33.053573 1364 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.167.126:41961 every 8 connection(s)
I20260810 04:41:33.053869 1252 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786336863388938-23197-0/minicluster-data/master-0/data/info.pb
I20260810 04:41:33.054529 23197 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskemv_AM/build/release/bin/kudu as pid 1252
I20260810 04:41:33.054628 23197 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786336863388938-23197-0/minicluster-data/master-0/wal/instance
I20260810 04:41:33.054806 23197 external_mini_cluster.cc:1428] Setting key d173f3912ed27580d440abf210efcd4a
I20260810 04:41:33.054823 1365 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 04:41:33.057983 1365 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b56636c33cce464eba3d4b547a7c704e: Bootstrap starting.
I20260810 04:41:33.058720 1365 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b56636c33cce464eba3d4b547a7c704e: Neither blocks nor log segments found. Creating new log.
I20260810 04:41:33.059010 1365 log.cc:824] T 00000000000000000000000000000000 P b56636c33cce464eba3d4b547a7c704e: Log is configured to *not* fsync() on all Append() calls
I20260810 04:41:33.059674 1365 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b56636c33cce464eba3d4b547a7c704e: No bootstrap required, opened a new log
Aug 10 04:41:33 dist-test-slave-bqcl krb5kdc[1217](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786336892, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.22.167.126@KRBTEST.COM
I20260810 04:41:33.061929 1365 raft_consensus.cc:348] T 00000000000000000000000000000000 P b56636c33cce464eba3d4b547a7c704e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b56636c33cce464eba3d4b547a7c704e" member_type: VOTER last_known_addr { host: "127.22.167.126" port: 41961 } }
I20260810 04:41:33.062104 1365 raft_consensus.cc:374] T 00000000000000000000000000000000 P b56636c33cce464eba3d4b547a7c704e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 04:41:33.062180 1365 raft_consensus.cc:729] T 00000000000000000000000000000000 P b56636c33cce464eba3d4b547a7c704e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b56636c33cce464eba3d4b547a7c704e, State: Initialized, Role: FOLLOWER
I20260810 04:41:33.062320 1365 consensus_queue.cc:260] T 00000000000000000000000000000000 P b56636c33cce464eba3d4b547a7c704e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b56636c33cce464eba3d4b547a7c704e" member_type: VOTER last_known_addr { host: "127.22.167.126" port: 41961 } }
I20260810 04:41:33.062417 1365 raft_consensus.cc:388] T 00000000000000000000000000000000 P b56636c33cce464eba3d4b547a7c704e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 04:41:33.062440 1365 raft_consensus.cc:482] T 00000000000000000000000000000000 P b56636c33cce464eba3d4b547a7c704e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 04:41:33.062490 1365 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b56636c33cce464eba3d4b547a7c704e [term 0 FOLLOWER]: Advancing to term 1
I20260810 04:41:33.062968 1365 raft_consensus.cc:504] T 00000000000000000000000000000000 P b56636c33cce464eba3d4b547a7c704e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b56636c33cce464eba3d4b547a7c704e" member_type: VOTER last_known_addr { host: "127.22.167.126" port: 41961 } }
I20260810 04:41:33.063055 1365 leader_election.cc:302] T 00000000000000000000000000000000 P b56636c33cce464eba3d4b547a7c704e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b56636c33cce464eba3d4b547a7c704e; no voters:
I20260810 04:41:33.063225 1365 leader_election.cc:288] T 00000000000000000000000000000000 P b56636c33cce464eba3d4b547a7c704e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 04:41:33.063266 1370 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b56636c33cce464eba3d4b547a7c704e [term 1 FOLLOWER]: Leader election won for term 1
I20260810 04:41:33.063402 1370 raft_consensus.cc:686] T 00000000000000000000000000000000 P b56636c33cce464eba3d4b547a7c704e [term 1 LEADER]: Becoming Leader. State: Replica: b56636c33cce464eba3d4b547a7c704e, State: Running, Role: LEADER
I20260810 04:41:33.063477 1365 sys_catalog.cc:564] T 00000000000000000000000000000000 P b56636c33cce464eba3d4b547a7c704e [sys.catalog]: configured and running, proceeding with master startup.
I20260810 04:41:33.063525 1370 consensus_queue.cc:237] T 00000000000000000000000000000000 P b56636c33cce464eba3d4b547a7c704e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b56636c33cce464eba3d4b547a7c704e" member_type: VOTER last_known_addr { host: "127.22.167.126" port: 41961 } }
I20260810 04:41:33.063856 1370 sys_catalog.cc:455] T 00000000000000000000000000000000 P b56636c33cce464eba3d4b547a7c704e [sys.catalog]: SysCatalogTable state changed. Reason: New leader b56636c33cce464eba3d4b547a7c704e. Latest consensus state: current_term: 1 leader_uuid: "b56636c33cce464eba3d4b547a7c704e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b56636c33cce464eba3d4b547a7c704e" member_type: VOTER last_known_addr { host: "127.22.167.126" port: 41961 } } }
I20260810 04:41:33.063932 1370 sys_catalog.cc:458] T 00000000000000000000000000000000 P b56636c33cce464eba3d4b547a7c704e [sys.catalog]: This master's current role is: LEADER
I20260810 04:41:33.064078 1375 sys_catalog.cc:455] T 00000000000000000000000000000000 P b56636c33cce464eba3d4b547a7c704e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b56636c33cce464eba3d4b547a7c704e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b56636c33cce464eba3d4b547a7c704e" member_type: VOTER last_known_addr { host: "127.22.167.126" port: 41961 } } }
I20260810 04:41:33.064172 1375 sys_catalog.cc:458] T 00000000000000000000000000000000 P b56636c33cce464eba3d4b547a7c704e [sys.catalog]: This master's current role is: LEADER
I20260810 04:41:33.064982 1386 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 04:41:33.065466 1386 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 04:41:33.065878 1368 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:41:33.055260 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48320 (local address 127.22.167.126:41961)
0810 04:41:33.055461 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0810 04:41:33.055471 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 04:41:33.055503 (+ 32us) server_negotiation.cc:407] Connection header received
0810 04:41:33.055940 (+ 437us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:41:33.055950 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:41:33.056011 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:41:33.056157 (+ 146us) server_negotiation.cc:226] Negotiated authn=SASL
0810 04:41:33.056707 (+ 550us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:41:33.058022 (+ 1315us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:41:33.059043 (+ 1021us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:41:33.059489 (+ 446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:41:33.059582 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:41:33.062049 (+ 2467us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 04:41:33.062053 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 04:41:33.062053 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 04:41:33.062064 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0810 04:41:33.064029 (+ 1965us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 04:41:33.064432 (+ 403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 04:41:33.064435 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 04:41:33.064435 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 04:41:33.064516 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 04:41:33.064706 (+ 190us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 04:41:33.064707 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 04:41:33.064708 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 04:41:33.065003 (+ 295us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 04:41:33.065553 (+ 550us) server_negotiation.cc:1035] Waiting for connection context
0810 04:41:33.065626 (+ 73us) server_negotiation.cc:299] Negotiation successful
0810 04:41:33.065739 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":148,"thread_start_us":88,"threads_started":1}
I20260810 04:41:33.067166 1386 catalog_manager.cc:1347] Generated new cluster ID: 8f7937fec90543fcb67172c95103c8d3
I20260810 04:41:33.067229 1386 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 04:41:33.077977 1386 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 04:41:33.078683 1386 catalog_manager.cc:1504] Loading token signing keys...
I20260810 04:41:33.090674 1386 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b56636c33cce464eba3d4b547a7c704e: Generated new TSK 0
I20260810 04:41:33.090852 1386 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.22.167.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.22.167.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.22.167.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786336863388938-23197-0/krb5kdc/oryx_127.22.167.65.keytab.
Entry for principal oryx/127.22.167.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786336863388938-23197-0/krb5kdc/oryx_127.22.167.65.keytab.
I20260810 04:41:33.124562 23197 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskemv_AM/build/release/bin/kudu
/tmp/dist-test-taskemv_AM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786336863388938-23197-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786336863388938-23197-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786336863388938-23197-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786336863388938-23197-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.167.65:0
--local_ip_for_outbound_sockets=127.22.167.65
--webserver_interface=127.22.167.65
--webserver_port=0
--tserver_master_addrs=127.22.167.126:41961
--keytab_file=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786336863388938-23197-0/krb5kdc/oryx_127.22.167.65.keytab
--principal=oryx/127.22.167.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.167.84:33603
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786336863388938-23197-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786336863388938-23197-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786336863388938-23197-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 04:41:33.205101 1411 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 04:41:33.205312 1411 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 04:41:33.205344 1411 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 04:41:33.206485 1411 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 04:41:33.206529 1411 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 04:41:33.206564 1411 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.167.65
I20260810 04:41:33.207975 1411 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.167.84:33603
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786336863388938-23197-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786336863388938-23197-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786336863388938-23197-0/krb5kdc/oryx_127.22.167.65.keytab
--principal=oryx/127.22.167.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.167.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786336863388938-23197-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.22.167.65
--webserver_port=0
--tserver_master_addrs=127.22.167.126:41961
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1411
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.167.65
--log_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786336863388938-23197-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 03:54:01 UTC on ccycloud.quasar-icijmt.root.comops.site
build id 4577009
I20260810 04:41:33.208201 1411 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 04:41:33.208408 1411 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 04:41:33.213135 1425 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 04:41:33.213135 1429 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 04:41:33.213178 1424 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 04:41:33.213349 1411 server_base.cc:1034] running on GCE node
I20260810 04:41:33.213534 1411 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 04:41:33.213747 1411 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 04:41:33.214882 1411 hybrid_clock.cc:648] HybridClock initialized: now 1786336893214865 us; error 27 us; skew 500 ppm
Aug 10 04:41:33 dist-test-slave-bqcl krb5kdc[1217](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786336893, etypes {rep=17 tkt=17 ses=17}, oryx/127.22.167.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 04:41:33.217195 1411 init.cc:377] Logged in from keytab as oryx/127.22.167.65@KRBTEST.COM (short username oryx)
I20260810 04:41:33.217623 1411 webserver.cc:460] Webserver started at http://127.22.167.65:45253/ using document root <none> and password file <none>
I20260810 04:41:33.217831 1411 fs_manager.cc:359] Metadata directory not provided
I20260810 04:41:33.217880 1411 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 04:41:33.218006 1411 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 04:41:33.219041 1411 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786336863388938-23197-0/minicluster-data/ts-0/data/instance:
uuid: "5e88101ae4ec49b2a0be733b6f18ea08"
format_stamp: "Formatted at 2026-08-10 04:41:33 on dist-test-slave-bqcl"
server_key: "598bfa07cd51f858e573855e59168a93"
server_key_iv: "17fc29583466f0ffe38c4a4feafabfb6"
server_key_version: "encryptionkey@0"
I20260810 04:41:33.219369 1411 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786336863388938-23197-0/minicluster-data/ts-0/wal/instance:
uuid: "5e88101ae4ec49b2a0be733b6f18ea08"
format_stamp: "Formatted at 2026-08-10 04:41:33 on dist-test-slave-bqcl"
server_key: "598bfa07cd51f858e573855e59168a93"
server_key_iv: "17fc29583466f0ffe38c4a4feafabfb6"
server_key_version: "encryptionkey@0"
I20260810 04:41:33.220563 1411 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260810 04:41:33.221194 1439 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 04:41:33.221346 1411 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260810 04:41:33.221421 1411 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786336863388938-23197-0/minicluster-data/ts-0/data,/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786336863388938-23197-0/minicluster-data/ts-0/wal
uuid: "5e88101ae4ec49b2a0be733b6f18ea08"
format_stamp: "Formatted at 2026-08-10 04:41:33 on dist-test-slave-bqcl"
server_key: "598bfa07cd51f858e573855e59168a93"
server_key_iv: "17fc29583466f0ffe38c4a4feafabfb6"
server_key_version: "encryptionkey@0"
I20260810 04:41:33.221486 1411 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786336863388938-23197-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786336863388938-23197-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786336863388938-23197-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 04:41:33.239722 1411 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 04:41:33.241498 1411 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 04:41:33.241626 1411 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 04:41:33.241820 1411 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 04:41:33.242105 1411 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 04:41:33.242166 1411 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 04:41:33.242203 1411 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 04:41:33.242228 1411 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 04:41:33.247599 1411 rpc_server.cc:307] RPC server started. Bound to: 127.22.167.65:43639
I20260810 04:41:33.247656 1627 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.167.65:43639 every 8 connection(s)
I20260810 04:41:33.247927 1411 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786336863388938-23197-0/minicluster-data/ts-0/data/info.pb
I20260810 04:41:33.249420 23197 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskemv_AM/build/release/bin/kudu as pid 1411
I20260810 04:41:33.249538 23197 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786336863388938-23197-0/minicluster-data/ts-0/wal/instance
I20260810 04:41:33.249750 23197 external_mini_cluster.cc:1428] Setting key 73a1d02de77bd272cf59af74733ca0b9
Aug 10 04:41:33 dist-test-slave-bqcl krb5kdc[1217](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786336893, etypes {rep=17 tkt=17 ses=17}, oryx/127.22.167.65@KRBTEST.COM for oryx/127.22.167.126@KRBTEST.COM
I20260810 04:41:33.257661 1368 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:41:33.248582 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.167.65:51615 (local address 127.22.167.126:41961)
0810 04:41:33.248660 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0810 04:41:33.248662 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 04:41:33.248881 (+ 219us) server_negotiation.cc:407] Connection header received
0810 04:41:33.249467 (+ 586us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:41:33.249469 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:41:33.249482 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:41:33.249552 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0810 04:41:33.250769 (+ 1217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:41:33.251261 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:41:33.252541 (+ 1280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:41:33.252743 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:41:33.252799 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:41:33.255009 (+ 2210us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 04:41:33.255012 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 04:41:33.255012 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 04:41:33.255021 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0810 04:41:33.256425 (+ 1404us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 04:41:33.256795 (+ 370us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 04:41:33.256796 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 04:41:33.256797 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 04:41:33.256862 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 04:41:33.257061 (+ 199us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 04:41:33.257062 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 04:41:33.257063 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 04:41:33.257199 (+ 136us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 04:41:33.257271 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0810 04:41:33.257474 (+ 203us) server_negotiation.cc:299] Negotiation successful
0810 04:41:33.257570 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260810 04:41:33.257709 1631 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:41:33.248549 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.126:41961 (local address 127.22.167.65:51615)
0810 04:41:33.248747 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0810 04:41:33.248788 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0810 04:41:33.249368 (+ 580us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 04:41:33.249699 (+ 331us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 04:41:33.249703 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 04:41:33.250424 (+ 721us) client_negotiation.cc:189] Negotiated authn=SASL
0810 04:41:33.250678 (+ 254us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:41:33.250684 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:41:33.251346 (+ 662us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:41:33.251346 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:41:33.252456 (+ 1110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:41:33.252459 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:41:33.252813 (+ 354us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:41:33.252814 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:41:33.252947 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:41:33.253436 (+ 489us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 04:41:33.253446 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0810 04:41:33.254911 (+ 1465us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 04:41:33.256508 (+ 1597us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 04:41:33.256515 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 04:41:33.256517 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 04:41:33.256729 (+ 212us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 04:41:33.256921 (+ 192us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 04:41:33.256922 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 04:41:33.256922 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 04:41:33.257011 (+ 89us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 04:41:33.257258 (+ 247us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 04:41:33.257261 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 04:41:33.257366 (+ 105us) client_negotiation.cc:769] Sending connection context
0810 04:41:33.257439 (+ 73us) client_negotiation.cc:240] Negotiation successful
0810 04:41:33.257551 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":161,"thread_start_us":80,"threads_started":1}
I20260810 04:41:33.258260 1629 heartbeater.cc:347] Connected to a master server at 127.22.167.126:41961
I20260810 04:41:33.258344 1629 heartbeater.cc:464] Registering TS with master...
I20260810 04:41:33.258505 1629 heartbeater.cc:511] Master 127.22.167.126:41961 requested a full tablet report, sending...
I20260810 04:41:33.259020 1306 ts_manager.cc:194] Registered new tserver with Master: 5e88101ae4ec49b2a0be733b6f18ea08 (127.22.167.65:43639)
WARNING: no policy specified for oryx/127.22.167.66@KRBTEST.COM; defaulting to no policy
I20260810 04:41:33.259631 1306 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.22.167.65@KRBTEST.COM'} at 127.22.167.65:51615
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.22.167.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.22.167.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786336863388938-23197-0/krb5kdc/oryx_127.22.167.66.keytab.
Entry for principal oryx/127.22.167.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786336863388938-23197-0/krb5kdc/oryx_127.22.167.66.keytab.
I20260810 04:41:33.278270 23197 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskemv_AM/build/release/bin/kudu
/tmp/dist-test-taskemv_AM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786336863388938-23197-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786336863388938-23197-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786336863388938-23197-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786336863388938-23197-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.167.66:0
--local_ip_for_outbound_sockets=127.22.167.66
--webserver_interface=127.22.167.66
--webserver_port=0
--tserver_master_addrs=127.22.167.126:41961
--keytab_file=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786336863388938-23197-0/krb5kdc/oryx_127.22.167.66.keytab
--principal=oryx/127.22.167.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.167.84:33603
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786336863388938-23197-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786336863388938-23197-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786336863388938-23197-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 04:41:33.356654 1636 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 04:41:33.356848 1636 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 04:41:33.356868 1636 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 04:41:33.357944 1636 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 04:41:33.357982 1636 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 04:41:33.358011 1636 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.167.66
I20260810 04:41:33.359450 1636 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.167.84:33603
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786336863388938-23197-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786336863388938-23197-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786336863388938-23197-0/krb5kdc/oryx_127.22.167.66.keytab
--principal=oryx/127.22.167.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.167.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786336863388938-23197-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.22.167.66
--webserver_port=0
--tserver_master_addrs=127.22.167.126:41961
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1636
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.167.66
--log_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786336863388938-23197-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 03:54:01 UTC on ccycloud.quasar-icijmt.root.comops.site
build id 4577009
I20260810 04:41:33.359658 1636 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 04:41:33.359858 1636 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 04:41:33.364549 1646 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 04:41:33.364552 1644 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 04:41:33.364562 1649 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 04:41:33.364845 1636 server_base.cc:1034] running on GCE node
I20260810 04:41:33.365015 1636 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 04:41:33.365231 1636 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 04:41:33.366365 1636 hybrid_clock.cc:648] HybridClock initialized: now 1786336893366347 us; error 34 us; skew 500 ppm
Aug 10 04:41:33 dist-test-slave-bqcl krb5kdc[1217](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786336893, etypes {rep=17 tkt=17 ses=17}, oryx/127.22.167.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 04:41:33.368636 1636 init.cc:377] Logged in from keytab as oryx/127.22.167.66@KRBTEST.COM (short username oryx)
I20260810 04:41:33.369004 1636 webserver.cc:460] Webserver started at http://127.22.167.66:35811/ using document root <none> and password file <none>
I20260810 04:41:33.369165 1636 fs_manager.cc:359] Metadata directory not provided
I20260810 04:41:33.369204 1636 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 04:41:33.369293 1636 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 04:41:33.370234 1636 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786336863388938-23197-0/minicluster-data/ts-1/data/instance:
uuid: "7670dc9cfd0d4c25a8294f9115c4235b"
format_stamp: "Formatted at 2026-08-10 04:41:33 on dist-test-slave-bqcl"
server_key: "db6e40629773f391648ddcd2a6a2bfda"
server_key_iv: "7dbf250e0c88d6ab6870ddbf1159e293"
server_key_version: "encryptionkey@0"
I20260810 04:41:33.370527 1636 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786336863388938-23197-0/minicluster-data/ts-1/wal/instance:
uuid: "7670dc9cfd0d4c25a8294f9115c4235b"
format_stamp: "Formatted at 2026-08-10 04:41:33 on dist-test-slave-bqcl"
server_key: "db6e40629773f391648ddcd2a6a2bfda"
server_key_iv: "7dbf250e0c88d6ab6870ddbf1159e293"
server_key_version: "encryptionkey@0"
I20260810 04:41:33.371642 1636 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260810 04:41:33.372232 1664 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 04:41:33.372393 1636 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260810 04:41:33.372459 1636 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786336863388938-23197-0/minicluster-data/ts-1/data,/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786336863388938-23197-0/minicluster-data/ts-1/wal
uuid: "7670dc9cfd0d4c25a8294f9115c4235b"
format_stamp: "Formatted at 2026-08-10 04:41:33 on dist-test-slave-bqcl"
server_key: "db6e40629773f391648ddcd2a6a2bfda"
server_key_iv: "7dbf250e0c88d6ab6870ddbf1159e293"
server_key_version: "encryptionkey@0"
I20260810 04:41:33.372507 1636 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786336863388938-23197-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786336863388938-23197-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786336863388938-23197-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 04:41:33.381213 1636 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 04:41:33.382977 1636 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 04:41:33.383105 1636 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 04:41:33.384568 1636 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 04:41:33.384898 1636 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 04:41:33.384941 1636 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 04:41:33.384974 1636 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 04:41:33.384994 1636 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 04:41:33.390380 1636 rpc_server.cc:307] RPC server started. Bound to: 127.22.167.66:46237
I20260810 04:41:33.390436 1858 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.167.66:46237 every 8 connection(s)
I20260810 04:41:33.390684 1636 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786336863388938-23197-0/minicluster-data/ts-1/data/info.pb
I20260810 04:41:33.392670 23197 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskemv_AM/build/release/bin/kudu as pid 1636
I20260810 04:41:33.392765 23197 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786336863388938-23197-0/minicluster-data/ts-1/wal/instance
I20260810 04:41:33.392925 23197 external_mini_cluster.cc:1428] Setting key f1446a48bd59d9bb4ea7f6f88c8895f0
Aug 10 04:41:33 dist-test-slave-bqcl krb5kdc[1217](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786336893, etypes {rep=17 tkt=17 ses=17}, oryx/127.22.167.66@KRBTEST.COM for oryx/127.22.167.126@KRBTEST.COM
I20260810 04:41:33.399936 1368 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:41:33.391281 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.167.66:60785 (local address 127.22.167.126:41961)
0810 04:41:33.391329 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0810 04:41:33.391331 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 04:41:33.391497 (+ 166us) server_negotiation.cc:407] Connection header received
0810 04:41:33.391915 (+ 418us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:41:33.391915 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:41:33.391924 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:41:33.391971 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0810 04:41:33.392842 (+ 871us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:41:33.393424 (+ 582us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:41:33.394830 (+ 1406us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:41:33.395013 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:41:33.395057 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:41:33.397283 (+ 2226us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 04:41:33.397284 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 04:41:33.397285 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 04:41:33.397291 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0810 04:41:33.398721 (+ 1430us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 04:41:33.399073 (+ 352us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 04:41:33.399075 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 04:41:33.399075 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 04:41:33.399143 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 04:41:33.399348 (+ 205us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 04:41:33.399349 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 04:41:33.399350 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 04:41:33.399483 (+ 133us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 04:41:33.399554 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0810 04:41:33.399757 (+ 203us) server_negotiation.cc:299] Negotiation successful
0810 04:41:33.399845 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260810 04:41:33.399971 1862 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:41:33.391241 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.126:41961 (local address 127.22.167.66:60785)
0810 04:41:33.391384 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0810 04:41:33.391423 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0810 04:41:33.391843 (+ 420us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 04:41:33.392021 (+ 178us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 04:41:33.392024 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 04:41:33.392535 (+ 511us) client_negotiation.cc:189] Negotiated authn=SASL
0810 04:41:33.392713 (+ 178us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:41:33.392717 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:41:33.393602 (+ 885us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:41:33.393603 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:41:33.394752 (+ 1149us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:41:33.394756 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:41:33.395080 (+ 324us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:41:33.395081 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:41:33.395214 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:41:33.395677 (+ 463us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 04:41:33.395687 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0810 04:41:33.397199 (+ 1512us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 04:41:33.398796 (+ 1597us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 04:41:33.398802 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 04:41:33.398803 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 04:41:33.399004 (+ 201us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 04:41:33.399208 (+ 204us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 04:41:33.399209 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 04:41:33.399209 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 04:41:33.399293 (+ 84us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 04:41:33.399540 (+ 247us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 04:41:33.399543 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 04:41:33.399643 (+ 100us) client_negotiation.cc:769] Sending connection context
0810 04:41:33.399719 (+ 76us) client_negotiation.cc:240] Negotiation successful
0810 04:41:33.399827 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":66,"threads_started":1}
I20260810 04:41:33.400399 1860 heartbeater.cc:347] Connected to a master server at 127.22.167.126:41961
I20260810 04:41:33.400480 1860 heartbeater.cc:464] Registering TS with master...
I20260810 04:41:33.400635 1860 heartbeater.cc:511] Master 127.22.167.126:41961 requested a full tablet report, sending...
I20260810 04:41:33.400971 1306 ts_manager.cc:194] Registered new tserver with Master: 7670dc9cfd0d4c25a8294f9115c4235b (127.22.167.66:46237)
I20260810 04:41:33.401317 1306 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.22.167.66@KRBTEST.COM'} at 127.22.167.66:60785
WARNING: no policy specified for oryx/127.22.167.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.22.167.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.22.167.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786336863388938-23197-0/krb5kdc/oryx_127.22.167.67.keytab.
Entry for principal oryx/127.22.167.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786336863388938-23197-0/krb5kdc/oryx_127.22.167.67.keytab.
I20260810 04:41:33.417861 23197 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskemv_AM/build/release/bin/kudu
/tmp/dist-test-taskemv_AM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786336863388938-23197-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786336863388938-23197-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786336863388938-23197-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786336863388938-23197-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.167.67:0
--local_ip_for_outbound_sockets=127.22.167.67
--webserver_interface=127.22.167.67
--webserver_port=0
--tserver_master_addrs=127.22.167.126:41961
--keytab_file=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786336863388938-23197-0/krb5kdc/oryx_127.22.167.67.keytab
--principal=oryx/127.22.167.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.167.84:33603
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786336863388938-23197-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786336863388938-23197-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786336863388938-23197-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 04:41:33.496979 1872 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 04:41:33.497189 1872 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 04:41:33.497212 1872 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 04:41:33.498303 1872 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 04:41:33.498342 1872 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 04:41:33.498375 1872 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.167.67
I20260810 04:41:33.499775 1872 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.167.84:33603
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786336863388938-23197-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786336863388938-23197-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786336863388938-23197-0/krb5kdc/oryx_127.22.167.67.keytab
--principal=oryx/127.22.167.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.167.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786336863388938-23197-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.22.167.67
--webserver_port=0
--tserver_master_addrs=127.22.167.126:41961
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1872
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.167.67
--log_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786336863388938-23197-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 03:54:01 UTC on ccycloud.quasar-icijmt.root.comops.site
build id 4577009
I20260810 04:41:33.499995 1872 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 04:41:33.500216 1872 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 04:41:33.504917 1880 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 04:41:33.504927 1881 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 04:41:33.504935 1883 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 04:41:33.504956 1872 server_base.cc:1034] running on GCE node
I20260810 04:41:33.505293 1872 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 04:41:33.505527 1872 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 04:41:33.506664 1872 hybrid_clock.cc:648] HybridClock initialized: now 1786336893506646 us; error 33 us; skew 500 ppm
Aug 10 04:41:33 dist-test-slave-bqcl krb5kdc[1217](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786336893, etypes {rep=17 tkt=17 ses=17}, oryx/127.22.167.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 04:41:33.508898 1872 init.cc:377] Logged in from keytab as oryx/127.22.167.67@KRBTEST.COM (short username oryx)
I20260810 04:41:33.509276 1872 webserver.cc:460] Webserver started at http://127.22.167.67:44653/ using document root <none> and password file <none>
I20260810 04:41:33.509459 1872 fs_manager.cc:359] Metadata directory not provided
I20260810 04:41:33.509505 1872 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 04:41:33.509631 1872 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 04:41:33.510609 1872 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786336863388938-23197-0/minicluster-data/ts-2/data/instance:
uuid: "1f3cb452053d447290369216f3e9ed93"
format_stamp: "Formatted at 2026-08-10 04:41:33 on dist-test-slave-bqcl"
server_key: "819e4ec0ea94ad7ab88abc284ec7d62d"
server_key_iv: "2703982d6d1d801dc367043ed5ff1a30"
server_key_version: "encryptionkey@0"
I20260810 04:41:33.510936 1872 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786336863388938-23197-0/minicluster-data/ts-2/wal/instance:
uuid: "1f3cb452053d447290369216f3e9ed93"
format_stamp: "Formatted at 2026-08-10 04:41:33 on dist-test-slave-bqcl"
server_key: "819e4ec0ea94ad7ab88abc284ec7d62d"
server_key_iv: "2703982d6d1d801dc367043ed5ff1a30"
server_key_version: "encryptionkey@0"
I20260810 04:41:33.512085 1872 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260810 04:41:33.512701 1891 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 04:41:33.512843 1872 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260810 04:41:33.512923 1872 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786336863388938-23197-0/minicluster-data/ts-2/data,/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786336863388938-23197-0/minicluster-data/ts-2/wal
uuid: "1f3cb452053d447290369216f3e9ed93"
format_stamp: "Formatted at 2026-08-10 04:41:33 on dist-test-slave-bqcl"
server_key: "819e4ec0ea94ad7ab88abc284ec7d62d"
server_key_iv: "2703982d6d1d801dc367043ed5ff1a30"
server_key_version: "encryptionkey@0"
I20260810 04:41:33.512980 1872 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786336863388938-23197-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786336863388938-23197-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786336863388938-23197-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 04:41:33.561094 1872 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 04:41:33.562954 1872 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 04:41:33.563087 1872 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 04:41:33.563336 1872 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 04:41:33.563647 1872 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 04:41:33.563680 1872 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 04:41:33.563719 1872 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 04:41:33.563740 1872 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 04:41:33.569398 1872 rpc_server.cc:307] RPC server started. Bound to: 127.22.167.67:41131
I20260810 04:41:33.569463 2078 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.167.67:41131 every 8 connection(s)
I20260810 04:41:33.569751 1872 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786336863388938-23197-0/minicluster-data/ts-2/data/info.pb
I20260810 04:41:33.572468 23197 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskemv_AM/build/release/bin/kudu as pid 1872
I20260810 04:41:33.572563 23197 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786336863388938-23197-0/minicluster-data/ts-2/wal/instance
I20260810 04:41:33.572728 23197 external_mini_cluster.cc:1428] Setting key abb464eac0be875092a0960264edfc07
Aug 10 04:41:33 dist-test-slave-bqcl krb5kdc[1217](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786336893, etypes {rep=17 tkt=17 ses=17}, oryx/127.22.167.67@KRBTEST.COM for oryx/127.22.167.126@KRBTEST.COM
I20260810 04:41:33.578364 1368 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:41:33.570428 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.167.67:49307 (local address 127.22.167.126:41961)
0810 04:41:33.570478 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0810 04:41:33.570481 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 04:41:33.570676 (+ 195us) server_negotiation.cc:407] Connection header received
0810 04:41:33.571185 (+ 509us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:41:33.571186 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:41:33.571198 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:41:33.571250 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0810 04:41:33.572172 (+ 922us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:41:33.572854 (+ 682us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:41:33.573884 (+ 1030us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:41:33.574087 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:41:33.574170 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:41:33.576093 (+ 1923us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 04:41:33.576095 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 04:41:33.576095 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 04:41:33.576102 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 04:41:33.577113 (+ 1011us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 04:41:33.577462 (+ 349us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 04:41:33.577463 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 04:41:33.577463 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 04:41:33.577518 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 04:41:33.577736 (+ 218us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 04:41:33.577737 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 04:41:33.577737 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 04:41:33.577860 (+ 123us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 04:41:33.577919 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0810 04:41:33.578195 (+ 276us) server_negotiation.cc:299] Negotiation successful
0810 04:41:33.578276 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260810 04:41:33.578516 2084 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:41:33.570417 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.126:41961 (local address 127.22.167.67:49307)
0810 04:41:33.570563 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0810 04:41:33.570599 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0810 04:41:33.571083 (+ 484us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 04:41:33.571320 (+ 237us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 04:41:33.571324 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 04:41:33.571881 (+ 557us) client_negotiation.cc:189] Negotiated authn=SASL
0810 04:41:33.572073 (+ 192us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:41:33.572078 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:41:33.572942 (+ 864us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:41:33.572943 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:41:33.573806 (+ 863us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:41:33.573809 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:41:33.574175 (+ 366us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:41:33.574177 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:41:33.574291 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:41:33.574677 (+ 386us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 04:41:33.574685 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0810 04:41:33.576012 (+ 1327us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 04:41:33.577183 (+ 1171us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 04:41:33.577191 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 04:41:33.577193 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 04:41:33.577397 (+ 204us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 04:41:33.577571 (+ 174us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 04:41:33.577571 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 04:41:33.577572 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 04:41:33.577680 (+ 108us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 04:41:33.577916 (+ 236us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 04:41:33.577923 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 04:41:33.578052 (+ 129us) client_negotiation.cc:769] Sending connection context
0810 04:41:33.578165 (+ 113us) client_negotiation.cc:240] Negotiation successful
0810 04:41:33.578314 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":68,"threads_started":1}
I20260810 04:41:33.578910 2080 heartbeater.cc:347] Connected to a master server at 127.22.167.126:41961
I20260810 04:41:33.578990 2080 heartbeater.cc:464] Registering TS with master...
I20260810 04:41:33.579125 2080 heartbeater.cc:511] Master 127.22.167.126:41961 requested a full tablet report, sending...
I20260810 04:41:33.579423 1306 ts_manager.cc:194] Registered new tserver with Master: 1f3cb452053d447290369216f3e9ed93 (127.22.167.67:41131)
I20260810 04:41:33.579744 1306 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.22.167.67@KRBTEST.COM'} at 127.22.167.67:49307
I20260810 04:41:33.584199 23197 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.22.167.125@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.22.167.125@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.22.167.125 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786336863388938-23197-0/krb5kdc/oryx_127.22.167.125.keytab.
Entry for principal oryx/127.22.167.125 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786336863388938-23197-0/krb5kdc/oryx_127.22.167.125.keytab.
I20260810 04:41:33.607971 23197 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskemv_AM/build/release/bin/kudu
/tmp/dist-test-taskemv_AM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786336863388938-23197-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786336863388938-23197-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786336863388938-23197-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786336863388938-23197-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.167.125:46451
--webserver_interface=127.22.167.125
--webserver_port=0
--keytab_file=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786336863388938-23197-0/krb5kdc/oryx_127.22.167.125.keytab
--principal=oryx/127.22.167.125
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.167.84:33603
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.167.126:41961,127.22.167.125:46451
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786336863388938-23197-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786336863388938-23197-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786336863388938-23197-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 04:41:33.686410 2088 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 04:41:33.686597 2088 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 04:41:33.686625 2088 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 04:41:33.687706 2088 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 04:41:33.687747 2088 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 04:41:33.687765 2088 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 04:41:33.687781 2088 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 04:41:33.687796 2088 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 04:41:33.689193 2088 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.167.84:33603
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786336863388938-23197-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786336863388938-23197-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.22.167.126:41961,127.22.167.125:46451
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786336863388938-23197-0/krb5kdc/oryx_127.22.167.125.keytab
--principal=oryx/127.22.167.125
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.167.125:46451
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786336863388938-23197-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.22.167.125
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2088
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786336863388938-23197-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 03:54:01 UTC on ccycloud.quasar-icijmt.root.comops.site
build id 4577009
W20260810 04:41:33.689307 2088 master_options.cc:55] Only 2 masters are specified by master_addresses_flag ('127.22.167.126:41961,127.22.167.125:46451'), but minimum of 3 are required to tolerate failures of any one master. It is recommended to use at least 3 masters.
I20260810 04:41:33.689455 2088 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 04:41:33.689647 2088 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 04:41:33.694059 2093 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 04:41:33.694043 2094 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 04:41:33.694059 2097 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 04:41:33.694206 2088 server_base.cc:1034] running on GCE node
I20260810 04:41:33.694468 2088 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 04:41:33.694728 2088 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 04:41:33.695863 2088 hybrid_clock.cc:648] HybridClock initialized: now 1786336893695846 us; error 33 us; skew 500 ppm
Aug 10 04:41:33 dist-test-slave-bqcl krb5kdc[1217](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786336893, etypes {rep=17 tkt=17 ses=17}, oryx/127.22.167.125@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 04:41:33.698221 2088 init.cc:377] Logged in from keytab as oryx/127.22.167.125@KRBTEST.COM (short username oryx)
I20260810 04:41:33.698670 2088 webserver.cc:460] Webserver started at http://127.22.167.125:40871/ using document root <none> and password file <none>
I20260810 04:41:33.698925 2088 fs_manager.cc:359] Metadata directory not provided
I20260810 04:41:33.698984 2088 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 04:41:33.699116 2088 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 04:41:33.700078 2088 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786336863388938-23197-0/minicluster-data/master-1/data/instance:
uuid: "b421e727773044528cb946bc1c2b5b83"
format_stamp: "Formatted at 2026-08-10 04:41:33 on dist-test-slave-bqcl"
server_key: "10c8002e73c4e9e3a7060d755157435f"
server_key_iv: "ef2a5a9085e98b5b00b4d1249b101df8"
server_key_version: "encryptionkey@0"
I20260810 04:41:33.700402 2088 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786336863388938-23197-0/minicluster-data/master-1/wal/instance:
uuid: "b421e727773044528cb946bc1c2b5b83"
format_stamp: "Formatted at 2026-08-10 04:41:33 on dist-test-slave-bqcl"
server_key: "10c8002e73c4e9e3a7060d755157435f"
server_key_iv: "ef2a5a9085e98b5b00b4d1249b101df8"
server_key_version: "encryptionkey@0"
I20260810 04:41:33.701628 2088 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260810 04:41:33.702272 2106 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 04:41:33.702418 2088 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260810 04:41:33.702497 2088 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786336863388938-23197-0/minicluster-data/master-1/data,/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786336863388938-23197-0/minicluster-data/master-1/wal
uuid: "b421e727773044528cb946bc1c2b5b83"
format_stamp: "Formatted at 2026-08-10 04:41:33 on dist-test-slave-bqcl"
server_key: "10c8002e73c4e9e3a7060d755157435f"
server_key_iv: "ef2a5a9085e98b5b00b4d1249b101df8"
server_key_version: "encryptionkey@0"
I20260810 04:41:33.702561 2088 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786336863388938-23197-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786336863388938-23197-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786336863388938-23197-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 04:41:33.710808 2088 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 04:41:33.712314 2088 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 04:41:33.712435 2088 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 04:41:33.716115 2088 rpc_server.cc:307] RPC server started. Bound to: 127.22.167.125:46451
I20260810 04:41:33.716145 2185 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.167.125:46451 every 8 connection(s)
I20260810 04:41:33.716456 2088 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786336863388938-23197-0/minicluster-data/master-1/data/info.pb
I20260810 04:41:33.717391 2186 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 04:41:33.718818 2186 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.22.167.126" port: 41961 } has no permanent_uuid. Determining permanent_uuid...
I20260810 04:41:33.721930 23197 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskemv_AM/build/release/bin/kudu as pid 2088
Aug 10 04:41:33 dist-test-slave-bqcl krb5kdc[1217](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786336893, etypes {rep=17 tkt=17 ses=17}, oryx/127.22.167.125@KRBTEST.COM for oryx/127.22.167.126@KRBTEST.COM
Aug 10 04:41:33 dist-test-slave-bqcl krb5kdc[1217](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786336892, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.22.167.125@KRBTEST.COM
I20260810 04:41:33.727770 1368 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:41:33.719447 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48330 (local address 127.22.167.126:41961)
0810 04:41:33.719500 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0810 04:41:33.719502 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 04:41:33.719736 (+ 234us) server_negotiation.cc:407] Connection header received
0810 04:41:33.720171 (+ 435us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:41:33.720173 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:41:33.720185 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:41:33.720238 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0810 04:41:33.720796 (+ 558us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:41:33.721311 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:41:33.722550 (+ 1239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:41:33.722743 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:41:33.722776 (+ 33us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:41:33.725102 (+ 2326us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 04:41:33.725104 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 04:41:33.725104 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 04:41:33.725112 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0810 04:41:33.726481 (+ 1369us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 04:41:33.726839 (+ 358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 04:41:33.726840 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 04:41:33.726841 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 04:41:33.726896 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 04:41:33.727112 (+ 216us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 04:41:33.727113 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 04:41:33.727113 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 04:41:33.727258 (+ 145us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 04:41:33.727317 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0810 04:41:33.727586 (+ 269us) server_negotiation.cc:299] Negotiation successful
0810 04:41:33.727680 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260810 04:41:33.727918 2188 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:41:33.719416 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.126:41961 (local address 127.0.0.1:48330)
0810 04:41:33.719626 (+ 210us) negotiation.cc:107] Waiting for socket to connect
0810 04:41:33.719658 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0810 04:41:33.720078 (+ 420us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 04:41:33.720298 (+ 220us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 04:41:33.720302 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 04:41:33.720531 (+ 229us) client_negotiation.cc:189] Negotiated authn=SASL
0810 04:41:33.720709 (+ 178us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:41:33.720713 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:41:33.721391 (+ 678us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:41:33.721392 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:41:33.722438 (+ 1046us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:41:33.722441 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:41:33.722842 (+ 401us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:41:33.722842 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:41:33.722944 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:41:33.723419 (+ 475us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 04:41:33.723428 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0810 04:41:33.725014 (+ 1586us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 04:41:33.726552 (+ 1538us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 04:41:33.726559 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 04:41:33.726561 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 04:41:33.726771 (+ 210us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 04:41:33.726950 (+ 179us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 04:41:33.726951 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 04:41:33.726951 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 04:41:33.727056 (+ 105us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 04:41:33.727318 (+ 262us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 04:41:33.727324 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 04:41:33.727465 (+ 141us) client_negotiation.cc:769] Sending connection context
0810 04:41:33.727550 (+ 85us) client_negotiation.cc:240] Negotiation successful
0810 04:41:33.727688 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":172,"thread_start_us":130,"threads_started":1}
I20260810 04:41:33.728490 2186 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.22.167.125" port: 46451 } has no permanent_uuid. Determining permanent_uuid...
I20260810 04:41:33.730567 2190 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:41:33.722429 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39280 (local address 127.22.167.125:46451)
0810 04:41:33.722578 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0810 04:41:33.722582 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 04:41:33.722605 (+ 23us) server_negotiation.cc:407] Connection header received
0810 04:41:33.722652 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:41:33.722655 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:41:33.722674 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:41:33.722737 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0810 04:41:33.723311 (+ 574us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:41:33.724286 (+ 975us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:41:33.725118 (+ 832us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:41:33.725552 (+ 434us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:41:33.725618 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:41:33.727737 (+ 2119us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 04:41:33.727744 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 04:41:33.727745 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 04:41:33.727751 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0810 04:41:33.729508 (+ 1757us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 04:41:33.729817 (+ 309us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 04:41:33.729819 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 04:41:33.729820 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 04:41:33.729876 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 04:41:33.730038 (+ 162us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 04:41:33.730039 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 04:41:33.730039 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 04:41:33.730208 (+ 169us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 04:41:33.730363 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0810 04:41:33.730414 (+ 51us) server_negotiation.cc:299] Negotiation successful
0810 04:41:33.730479 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":120,"thread_start_us":79,"threads_started":1}
Aug 10 04:41:33 dist-test-slave-bqcl krb5kdc[1217](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786336893, etypes {rep=17 tkt=17 ses=17}, oryx/127.22.167.125@KRBTEST.COM for oryx/127.22.167.125@KRBTEST.COM
I20260810 04:41:33.734715 2188 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:41:33.728717 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.125:46451 (local address 127.0.0.1:39290)
0810 04:41:33.728809 (+ 92us) negotiation.cc:107] Waiting for socket to connect
0810 04:41:33.728818 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0810 04:41:33.728855 (+ 37us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 04:41:33.729011 (+ 156us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 04:41:33.729012 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 04:41:33.729145 (+ 133us) client_negotiation.cc:189] Negotiated authn=SASL
0810 04:41:33.729206 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:41:33.729208 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:41:33.729777 (+ 569us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:41:33.729778 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:41:33.730291 (+ 513us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:41:33.730293 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:41:33.730808 (+ 515us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:41:33.730809 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:41:33.730880 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:41:33.731222 (+ 342us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 04:41:33.731227 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0810 04:41:33.732217 (+ 990us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 04:41:33.734006 (+ 1789us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 04:41:33.734007 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 04:41:33.734007 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 04:41:33.734170 (+ 163us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 04:41:33.734322 (+ 152us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 04:41:33.734323 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 04:41:33.734323 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 04:41:33.734366 (+ 43us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 04:41:33.734516 (+ 150us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 04:41:33.734516 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 04:41:33.734546 (+ 30us) client_negotiation.cc:769] Sending connection context
0810 04:41:33.734588 (+ 42us) client_negotiation.cc:240] Negotiation successful
0810 04:41:33.734652 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":76}
I20260810 04:41:33.734812 2195 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:41:33.728776 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39290 (local address 127.22.167.125:46451)
0810 04:41:33.728908 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0810 04:41:33.728910 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 04:41:33.728916 (+ 6us) server_negotiation.cc:407] Connection header received
0810 04:41:33.728932 (+ 16us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:41:33.728932 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:41:33.728940 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:41:33.728993 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0810 04:41:33.729274 (+ 281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:41:33.729712 (+ 438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:41:33.730363 (+ 651us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:41:33.730568 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:41:33.730630 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:41:33.732290 (+ 1660us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 04:41:33.732291 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 04:41:33.732291 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 04:41:33.732297 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0810 04:41:33.733949 (+ 1652us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 04:41:33.734228 (+ 279us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 04:41:33.734228 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 04:41:33.734229 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 04:41:33.734275 (+ 46us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 04:41:33.734399 (+ 124us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 04:41:33.734399 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 04:41:33.734400 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 04:41:33.734478 (+ 78us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 04:41:33.734518 (+ 40us) server_negotiation.cc:1035] Waiting for connection context
0810 04:41:33.734602 (+ 84us) server_negotiation.cc:299] Negotiation successful
0810 04:41:33.734694 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":39,"server-negotiator.queue_time_us":114,"thread_start_us":80,"threads_started":1}
I20260810 04:41:33.735862 2186 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b421e727773044528cb946bc1c2b5b83: Bootstrap starting.
I20260810 04:41:33.736418 2186 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b421e727773044528cb946bc1c2b5b83: Neither blocks nor log segments found. Creating new log.
I20260810 04:41:33.736656 2186 log.cc:824] T 00000000000000000000000000000000 P b421e727773044528cb946bc1c2b5b83: Log is configured to *not* fsync() on all Append() calls
I20260810 04:41:33.737275 2186 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b421e727773044528cb946bc1c2b5b83: No bootstrap required, opened a new log
I20260810 04:41:33.738679 2186 raft_consensus.cc:348] T 00000000000000000000000000000000 P b421e727773044528cb946bc1c2b5b83 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b56636c33cce464eba3d4b547a7c704e" member_type: VOTER last_known_addr { host: "127.22.167.126" port: 41961 } } peers { permanent_uuid: "b421e727773044528cb946bc1c2b5b83" member_type: VOTER last_known_addr { host: "127.22.167.125" port: 46451 } }
I20260810 04:41:33.738804 2186 raft_consensus.cc:374] T 00000000000000000000000000000000 P b421e727773044528cb946bc1c2b5b83 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 04:41:33.738837 2186 raft_consensus.cc:729] T 00000000000000000000000000000000 P b421e727773044528cb946bc1c2b5b83 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b421e727773044528cb946bc1c2b5b83, State: Initialized, Role: FOLLOWER
I20260810 04:41:33.738955 2186 consensus_queue.cc:260] T 00000000000000000000000000000000 P b421e727773044528cb946bc1c2b5b83 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b56636c33cce464eba3d4b547a7c704e" member_type: VOTER last_known_addr { host: "127.22.167.126" port: 41961 } } peers { permanent_uuid: "b421e727773044528cb946bc1c2b5b83" member_type: VOTER last_known_addr { host: "127.22.167.125" port: 46451 } }
I20260810 04:41:33.739217 2186 sys_catalog.cc:564] T 00000000000000000000000000000000 P b421e727773044528cb946bc1c2b5b83 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 04:41:33.739181 2198 sys_catalog.cc:455] T 00000000000000000000000000000000 P b421e727773044528cb946bc1c2b5b83 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b56636c33cce464eba3d4b547a7c704e" member_type: VOTER last_known_addr { host: "127.22.167.126" port: 41961 } } peers { permanent_uuid: "b421e727773044528cb946bc1c2b5b83" member_type: VOTER last_known_addr { host: "127.22.167.125" port: 46451 } } }
I20260810 04:41:33.739267 2198 sys_catalog.cc:458] T 00000000000000000000000000000000 P b421e727773044528cb946bc1c2b5b83 [sys.catalog]: This master's current role is: FOLLOWER
W20260810 04:41:33.740675 2215 catalog_manager.cc:1558] T 00000000000000000000000000000000 P b421e727773044528cb946bc1c2b5b83: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260810 04:41:33.740724 2215 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260810 04:41:33.742221 2088 master_runner.cc:428] Detected that this master b421e727773044528cb946bc1c2b5b83 is joining an existing cluster
I20260810 04:41:33.742293 2088 master_runner.cc:432] Initiating AddMaster RPC to add 127.22.167.125:46451
I20260810 04:41:33.749115 1368 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:41:33.743149 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48334 (local address 127.22.167.126:41961)
0810 04:41:33.743194 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0810 04:41:33.743196 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 04:41:33.743321 (+ 125us) server_negotiation.cc:407] Connection header received
0810 04:41:33.743404 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:41:33.743405 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:41:33.743414 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:41:33.743455 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0810 04:41:33.743817 (+ 362us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:41:33.744560 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:41:33.745347 (+ 787us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:41:33.745541 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:41:33.745593 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:41:33.746484 (+ 891us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 04:41:33.746485 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 04:41:33.746485 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 04:41:33.746492 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 04:41:33.747981 (+ 1489us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 04:41:33.748353 (+ 372us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 04:41:33.748354 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 04:41:33.748355 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 04:41:33.748408 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 04:41:33.748645 (+ 237us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 04:41:33.748646 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 04:41:33.748646 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 04:41:33.748757 (+ 111us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 04:41:33.748811 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0810 04:41:33.748953 (+ 142us) server_negotiation.cc:299] Negotiation successful
0810 04:41:33.749026 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260810 04:41:33.749151 2223 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:41:33.743108 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.126:41961 (local address 127.0.0.1:48334)
0810 04:41:33.743256 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0810 04:41:33.743266 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0810 04:41:33.743345 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 04:41:33.743495 (+ 150us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 04:41:33.743495 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 04:41:33.743683 (+ 188us) client_negotiation.cc:189] Negotiated authn=SASL
0810 04:41:33.743747 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:41:33.743748 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:41:33.744640 (+ 892us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:41:33.744640 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:41:33.745272 (+ 632us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:41:33.745274 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:41:33.745619 (+ 345us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:41:33.745620 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:41:33.745681 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:41:33.746012 (+ 331us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 04:41:33.746016 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0810 04:41:33.746417 (+ 401us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 04:41:33.748052 (+ 1635us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 04:41:33.748053 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 04:41:33.748053 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 04:41:33.748295 (+ 242us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 04:41:33.748539 (+ 244us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 04:41:33.748539 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 04:41:33.748540 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 04:41:33.748591 (+ 51us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 04:41:33.748811 (+ 220us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 04:41:33.748811 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 04:41:33.748851 (+ 40us) client_negotiation.cc:769] Sending connection context
0810 04:41:33.748922 (+ 71us) client_negotiation.cc:240] Negotiation successful
0810 04:41:33.749059 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":125,"mutex_wait_us":29,"thread_start_us":135,"threads_started":1}
I20260810 04:41:33.750160 2195 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:41:33.743292 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39292 (local address 127.22.167.125:46451)
0810 04:41:33.743318 (+ 26us) server_negotiation.cc:206] Beginning negotiation
0810 04:41:33.743319 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0810 04:41:33.743352 (+ 33us) server_negotiation.cc:407] Connection header received
0810 04:41:33.743435 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:41:33.743436 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:41:33.743444 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:41:33.743483 (+ 39us) server_negotiation.cc:226] Negotiated authn=SASL
0810 04:41:33.743860 (+ 377us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:41:33.744551 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:41:33.745631 (+ 1080us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:41:33.745917 (+ 286us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:41:33.745957 (+ 40us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:41:33.746884 (+ 927us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 04:41:33.746885 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 04:41:33.746885 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 04:41:33.746892 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 04:41:33.748427 (+ 1535us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 04:41:33.749418 (+ 991us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 04:41:33.749419 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 04:41:33.749419 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 04:41:33.749477 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 04:41:33.749653 (+ 176us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 04:41:33.749654 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 04:41:33.749654 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 04:41:33.749771 (+ 117us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 04:41:33.749809 (+ 38us) server_negotiation.cc:1035] Waiting for connection context
0810 04:41:33.749968 (+ 159us) server_negotiation.cc:299] Negotiation successful
0810 04:41:33.750041 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13}
I20260810 04:41:33.750573 2224 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:41:33.743148 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.125:46451 (local address 127.0.0.1:39292)
0810 04:41:33.743298 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0810 04:41:33.743307 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0810 04:41:33.743365 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 04:41:33.743497 (+ 132us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 04:41:33.743498 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 04:41:33.743740 (+ 242us) client_negotiation.cc:189] Negotiated authn=SASL
0810 04:41:33.743801 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:41:33.743803 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:41:33.744627 (+ 824us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:41:33.744628 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:41:33.745539 (+ 911us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:41:33.745542 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:41:33.746021 (+ 479us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:41:33.746022 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:41:33.746077 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:41:33.746452 (+ 375us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 04:41:33.746456 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0810 04:41:33.746803 (+ 347us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 04:41:33.748509 (+ 1706us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 04:41:33.748510 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 04:41:33.748510 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 04:41:33.749080 (+ 570us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 04:41:33.749541 (+ 461us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 04:41:33.749542 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 04:41:33.749542 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 04:41:33.749593 (+ 51us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 04:41:33.749844 (+ 251us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 04:41:33.749845 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 04:41:33.749876 (+ 31us) client_negotiation.cc:769] Sending connection context
0810 04:41:33.750387 (+ 511us) client_negotiation.cc:240] Negotiation successful
0810 04:41:33.750478 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":136,"mutex_wait_us":34,"thread_start_us":60,"threads_started":1}
Aug 10 04:41:33 dist-test-slave-bqcl krb5kdc[1217](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786336893, etypes {rep=17 tkt=17 ses=17}, oryx/127.22.167.126@KRBTEST.COM for oryx/127.22.167.125@KRBTEST.COM
I20260810 04:41:33.757845 2226 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:41:33.751225 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.125:46451 (local address 127.0.0.1:39300)
0810 04:41:33.751347 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0810 04:41:33.751378 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0810 04:41:33.751435 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 04:41:33.751570 (+ 135us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 04:41:33.751572 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 04:41:33.751791 (+ 219us) client_negotiation.cc:189] Negotiated authn=SASL
0810 04:41:33.751833 (+ 42us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:41:33.751835 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:41:33.752636 (+ 801us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:41:33.752636 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:41:33.753171 (+ 535us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:41:33.753173 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:41:33.753594 (+ 421us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:41:33.753595 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:41:33.753642 (+ 47us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:41:33.754039 (+ 397us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 04:41:33.754043 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0810 04:41:33.755476 (+ 1433us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 04:41:33.756960 (+ 1484us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 04:41:33.756966 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 04:41:33.756968 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 04:41:33.757142 (+ 174us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 04:41:33.757316 (+ 174us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 04:41:33.757317 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 04:41:33.757317 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 04:41:33.757365 (+ 48us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 04:41:33.757592 (+ 227us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 04:41:33.757596 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 04:41:33.757632 (+ 36us) client_negotiation.cc:769] Sending connection context
0810 04:41:33.757688 (+ 56us) client_negotiation.cc:240] Negotiation successful
0810 04:41:33.757771 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":99,"thread_start_us":74,"threads_started":1}
I20260810 04:41:33.757866 2195 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:41:33.751430 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39300 (local address 127.22.167.125:46451)
0810 04:41:33.751465 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0810 04:41:33.751467 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 04:41:33.751475 (+ 8us) server_negotiation.cc:407] Connection header received
0810 04:41:33.751503 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:41:33.751504 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:41:33.751512 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:41:33.751561 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0810 04:41:33.751895 (+ 334us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:41:33.752569 (+ 674us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:41:33.753244 (+ 675us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:41:33.753533 (+ 289us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:41:33.753596 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:41:33.755554 (+ 1958us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 04:41:33.755555 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 04:41:33.755555 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 04:41:33.755562 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 04:41:33.756889 (+ 1327us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 04:41:33.757203 (+ 314us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 04:41:33.757204 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 04:41:33.757204 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 04:41:33.757260 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 04:41:33.757423 (+ 163us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 04:41:33.757423 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 04:41:33.757424 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 04:41:33.757538 (+ 114us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 04:41:33.757601 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0810 04:41:33.757721 (+ 120us) server_negotiation.cc:299] Negotiation successful
0810 04:41:33.757789 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260810 04:41:33.758389 1306 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.22.167.125:46451: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "b421e727773044528cb946bc1c2b5b83"
member_type: NON_VOTER
last_known_addr {
host: "127.22.167.125"
port: 46451
}
attrs {
promote: true
}
}
dest_uuid: "b56636c33cce464eba3d4b547a7c704e"
cas_config_opid_index: -1
I20260810 04:41:33.758699 1306 consensus_queue.cc:237] T 00000000000000000000000000000000 P b56636c33cce464eba3d4b547a7c704e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "b56636c33cce464eba3d4b547a7c704e" member_type: VOTER last_known_addr { host: "127.22.167.126" port: 41961 } } peers { permanent_uuid: "b421e727773044528cb946bc1c2b5b83" member_type: NON_VOTER last_known_addr { host: "127.22.167.125" port: 46451 } attrs { promote: true } }
I20260810 04:41:33.759155 2230 raft_consensus.cc:2932] T 00000000000000000000000000000000 P b56636c33cce464eba3d4b547a7c704e [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER b421e727773044528cb946bc1c2b5b83 (127.22.167.125) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "b56636c33cce464eba3d4b547a7c704e" member_type: VOTER last_known_addr { host: "127.22.167.126" port: 41961 } } peers { permanent_uuid: "b421e727773044528cb946bc1c2b5b83" member_type: NON_VOTER last_known_addr { host: "127.22.167.125" port: 46451 } attrs { promote: true } } }
I20260810 04:41:33.759424 2149 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b421e727773044528cb946bc1c2b5b83 [term 0 FOLLOWER]: Advancing to term 1
I20260810 04:41:33.760082 2149 raft_consensus.cc:1261] T 00000000000000000000000000000000 P b421e727773044528cb946bc1c2b5b83 [term 1 FOLLOWER]: Refusing update from remote peer b56636c33cce464eba3d4b547a7c704e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20260810 04:41:33.760134 2230 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.22.167.125:46451
I20260810 04:41:33.760289 2227 sys_catalog.cc:455] T 00000000000000000000000000000000 P b56636c33cce464eba3d4b547a7c704e [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "b56636c33cce464eba3d4b547a7c704e" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "b56636c33cce464eba3d4b547a7c704e" member_type: VOTER last_known_addr { host: "127.22.167.126" port: 41961 } } peers { permanent_uuid: "b421e727773044528cb946bc1c2b5b83" member_type: NON_VOTER last_known_addr { host: "127.22.167.125" port: 46451 } attrs { promote: true } } }
I20260810 04:41:33.760383 2227 sys_catalog.cc:458] T 00000000000000000000000000000000 P b56636c33cce464eba3d4b547a7c704e [sys.catalog]: This master's current role is: LEADER
I20260810 04:41:33.760421 2088 master.cc:537] Master@127.22.167.125:46451 shutting down...
I20260810 04:41:33.760633 2227 consensus_queue.cc:1035] T 00000000000000000000000000000000 P b56636c33cce464eba3d4b547a7c704e [LEADER]: Connected to new peer: Peer: permanent_uuid: "b421e727773044528cb946bc1c2b5b83" member_type: NON_VOTER last_known_addr { host: "127.22.167.125" port: 46451 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 0, Time since last communication: 0.000s
W20260810 04:41:33.761276 1280 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master
W20260810 04:41:33.761674 1280 consensus_peers.cc:487] T 00000000000000000000000000000000 P b56636c33cce464eba3d4b547a7c704e -> Peer b421e727773044528cb946bc1c2b5b83 (127.22.167.125:46451): Couldn't send request to peer b421e727773044528cb946bc1c2b5b83. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260810 04:41:33.762914 2088 raft_consensus.cc:2229] T 00000000000000000000000000000000 P b421e727773044528cb946bc1c2b5b83 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 04:41:33.763001 2088 raft_consensus.cc:2258] T 00000000000000000000000000000000 P b421e727773044528cb946bc1c2b5b83 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 04:41:33.763051 2088 tablet_replica.cc:330] T 00000000000000000000000000000000 P b421e727773044528cb946bc1c2b5b83: stopping tablet replica
I20260810 04:41:33.775382 2088 master.cc:552] Master@127.22.167.125:46451 shutdown complete.
I20260810 04:41:33.775419 2088 master_runner.cc:305] Clearing existing system tablet
I20260810 04:41:33.775530 2088 fs_manager.cc:359] Metadata directory not provided
I20260810 04:41:33.775569 2088 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 04:41:33.776093 2088 fs_manager.cc:709] Time spent opening directory manager: real 0.000s user 0.000s sys 0.001s
I20260810 04:41:33.776492 2236 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 04:41:33.776633 2088 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786336863388938-23197-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 04:41:33.776688 2088 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260810 04:41:33.776719 2088 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786336863388938-23197-0/minicluster-data/master-1/data,/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786336863388938-23197-0/minicluster-data/master-1/wal
uuid: "b421e727773044528cb946bc1c2b5b83"
format_stamp: "Formatted at 2026-08-10 04:41:33 on dist-test-slave-bqcl"
server_key: "10c8002e73c4e9e3a7060d755157435f"
server_key_iv: "ef2a5a9085e98b5b00b4d1249b101df8"
server_key_version: "encryptionkey@0"
I20260810 04:41:33.777359 2088 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P b421e727773044528cb946bc1c2b5b83: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 04:41:33.778669 2088 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P b421e727773044528cb946bc1c2b5b83: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260810 04:41:33.778730 2088 log.cc:1196] T 00000000000000000000000000000000 P b421e727773044528cb946bc1c2b5b83: Deleting WAL directory at /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786336863388938-23197-0/minicluster-data/master-1/wal/wals/00000000000000000000000000000000
I20260810 04:41:33.778895 2088 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P b421e727773044528cb946bc1c2b5b83: Deleting consensus metadata
I20260810 04:41:33.778995 2088 master_runner.cc:315] Copying system tablet from 127.22.167.126:41961
I20260810 04:41:33.779520 2088 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P b421e727773044528cb946bc1c2b5b83: tablet copy: Beginning tablet copy session from remote peer at address 127.22.167.126:41961
I20260810 04:41:33.785072 2243 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:41:33.779798 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.126:41961 (local address 127.0.0.1:48346)
0810 04:41:33.779911 (+ 113us) negotiation.cc:107] Waiting for socket to connect
0810 04:41:33.779922 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0810 04:41:33.779994 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 04:41:33.780118 (+ 124us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 04:41:33.780119 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 04:41:33.780271 (+ 152us) client_negotiation.cc:189] Negotiated authn=SASL
0810 04:41:33.780323 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:41:33.780324 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:41:33.780930 (+ 606us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:41:33.780931 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:41:33.781541 (+ 610us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:41:33.781543 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:41:33.781926 (+ 383us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:41:33.781926 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:41:33.781973 (+ 47us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:41:33.782394 (+ 421us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 04:41:33.782399 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0810 04:41:33.782670 (+ 271us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 04:41:33.784261 (+ 1591us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 04:41:33.784262 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 04:41:33.784262 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 04:41:33.784413 (+ 151us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 04:41:33.784600 (+ 187us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 04:41:33.784601 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 04:41:33.784601 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 04:41:33.784643 (+ 42us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 04:41:33.784862 (+ 219us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 04:41:33.784862 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 04:41:33.784894 (+ 32us) client_negotiation.cc:769] Sending connection context
0810 04:41:33.784938 (+ 44us) client_negotiation.cc:240] Negotiation successful
0810 04:41:33.785003 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":92,"thread_start_us":68,"threads_started":1}
I20260810 04:41:33.785126 1368 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:41:33.779846 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48346 (local address 127.22.167.126:41961)
0810 04:41:33.779882 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0810 04:41:33.779884 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 04:41:33.779976 (+ 92us) server_negotiation.cc:407] Connection header received
0810 04:41:33.780050 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:41:33.780051 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:41:33.780060 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:41:33.780112 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0810 04:41:33.780389 (+ 277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:41:33.780864 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:41:33.781632 (+ 768us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:41:33.781835 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:41:33.781866 (+ 31us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:41:33.782764 (+ 898us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 04:41:33.782766 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 04:41:33.782767 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 04:41:33.782774 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 04:41:33.784185 (+ 1411us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 04:41:33.784474 (+ 289us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 04:41:33.784475 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 04:41:33.784475 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 04:41:33.784544 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 04:41:33.784687 (+ 143us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 04:41:33.784688 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 04:41:33.784688 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 04:41:33.784811 (+ 123us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 04:41:33.784874 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0810 04:41:33.784961 (+ 87us) server_negotiation.cc:299] Negotiation successful
0810 04:41:33.785035 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260810 04:41:33.785420 1340 tablet_copy_service.cc:140] P b56636c33cce464eba3d4b547a7c704e: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer b421e727773044528cb946bc1c2b5b83 ({username='oryx', principal='oryx/127.22.167.125@KRBTEST.COM'} at 127.0.0.1:48346)
I20260810 04:41:33.785508 1340 tablet_copy_service.cc:161] P b56636c33cce464eba3d4b547a7c704e: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer b421e727773044528cb946bc1c2b5b83 at {username='oryx', principal='oryx/127.22.167.125@KRBTEST.COM'} at 127.0.0.1:48346: session id = b421e727773044528cb946bc1c2b5b83-00000000000000000000000000000000
I20260810 04:41:33.786381 1340 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P b56636c33cce464eba3d4b547a7c704e: Tablet Copy: opened 0 blocks and 1 log segments
I20260810 04:41:33.786919 2088 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260810 04:41:33.787681 2088 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P b421e727773044528cb946bc1c2b5b83: tablet copy: Starting download of 0 data blocks...
I20260810 04:41:33.787813 2088 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P b421e727773044528cb946bc1c2b5b83: tablet copy: Starting download of 1 WAL segments...
I20260810 04:41:33.788616 2088 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P b421e727773044528cb946bc1c2b5b83: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260810 04:41:33.789294 1340 tablet_copy_service.cc:342] P b56636c33cce464eba3d4b547a7c704e: Request end of tablet copy session b421e727773044528cb946bc1c2b5b83-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.22.167.125@KRBTEST.COM'} at 127.0.0.1:48346
I20260810 04:41:33.789341 1340 tablet_copy_service.cc:433] P b56636c33cce464eba3d4b547a7c704e: ending tablet copy session b421e727773044528cb946bc1c2b5b83-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer b421e727773044528cb946bc1c2b5b83
I20260810 04:41:33.791826 2088 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 04:41:33.792809 2245 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 04:41:33.792872 2244 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 04:41:33.794925 2088 server_base.cc:1034] running on GCE node
W20260810 04:41:33.794931 2250 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 04:41:33.795110 2088 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 04:41:33.795245 2088 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 04:41:33.796367 2088 hybrid_clock.cc:648] HybridClock initialized: now 1786336893796353 us; error 27 us; skew 500 ppm
Aug 10 04:41:33 dist-test-slave-bqcl krb5kdc[1217](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786336893, etypes {rep=17 tkt=17 ses=17}, oryx/127.22.167.125@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 04:41:33.797643 2088 init.cc:377] Logged in from keytab as oryx/127.22.167.125@KRBTEST.COM (short username oryx)
I20260810 04:41:33.797897 2088 webserver.cc:460] Webserver started at http://127.22.167.125:34951/ using document root <none> and password file <none>
I20260810 04:41:33.798005 2088 fs_manager.cc:359] Metadata directory not provided
I20260810 04:41:33.798038 2088 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 04:41:33.798694 2088 fs_manager.cc:709] Time spent opening directory manager: real 0.000s user 0.000s sys 0.000s
I20260810 04:41:33.799181 2269 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 04:41:33.799290 2088 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260810 04:41:33.799341 2088 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786336863388938-23197-0/minicluster-data/master-1/data,/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786336863388938-23197-0/minicluster-data/master-1/wal
uuid: "b421e727773044528cb946bc1c2b5b83"
format_stamp: "Formatted at 2026-08-10 04:41:33 on dist-test-slave-bqcl"
server_key: "10c8002e73c4e9e3a7060d755157435f"
server_key_iv: "ef2a5a9085e98b5b00b4d1249b101df8"
server_key_version: "encryptionkey@0"
I20260810 04:41:33.799391 2088 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786336863388938-23197-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786336863388938-23197-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786336863388938-23197-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 04:41:33.816145 2088 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 04:41:33.817608 2088 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 04:41:33.821069 2088 rpc_server.cc:307] RPC server started. Bound to: 127.22.167.125:46451
I20260810 04:41:33.821257 2347 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.167.125:46451 every 8 connection(s)
I20260810 04:41:33.821441 2088 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786336863388938-23197-0/minicluster-data/master-1/data/info.pb
I20260810 04:41:33.822239 2348 sys_catalog.cc:263] Verifying existing consensus state
I20260810 04:41:33.822625 2348 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b421e727773044528cb946bc1c2b5b83: Bootstrap starting.
I20260810 04:41:33.824790 2348 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b421e727773044528cb946bc1c2b5b83: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260810 04:41:33.825031 2348 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b421e727773044528cb946bc1c2b5b83: Bootstrap complete.
I20260810 04:41:33.825222 2348 raft_consensus.cc:348] T 00000000000000000000000000000000 P b421e727773044528cb946bc1c2b5b83 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "b56636c33cce464eba3d4b547a7c704e" member_type: VOTER last_known_addr { host: "127.22.167.126" port: 41961 } } peers { permanent_uuid: "b421e727773044528cb946bc1c2b5b83" member_type: NON_VOTER last_known_addr { host: "127.22.167.125" port: 46451 } attrs { promote: true } }
I20260810 04:41:33.825299 2348 raft_consensus.cc:729] T 00000000000000000000000000000000 P b421e727773044528cb946bc1c2b5b83 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: b421e727773044528cb946bc1c2b5b83, State: Initialized, Role: LEARNER
I20260810 04:41:33.825335 2348 consensus_queue.cc:260] T 00000000000000000000000000000000 P b421e727773044528cb946bc1c2b5b83 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "b56636c33cce464eba3d4b547a7c704e" member_type: VOTER last_known_addr { host: "127.22.167.126" port: 41961 } } peers { permanent_uuid: "b421e727773044528cb946bc1c2b5b83" member_type: NON_VOTER last_known_addr { host: "127.22.167.125" port: 46451 } attrs { promote: true } }
I20260810 04:41:33.825500 2348 sys_catalog.cc:564] T 00000000000000000000000000000000 P b421e727773044528cb946bc1c2b5b83 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 04:41:33.825484 2352 sys_catalog.cc:455] T 00000000000000000000000000000000 P b421e727773044528cb946bc1c2b5b83 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "b56636c33cce464eba3d4b547a7c704e" member_type: VOTER last_known_addr { host: "127.22.167.126" port: 41961 } } peers { permanent_uuid: "b421e727773044528cb946bc1c2b5b83" member_type: NON_VOTER last_known_addr { host: "127.22.167.125" port: 46451 } attrs { promote: true } } }
I20260810 04:41:33.825568 2352 sys_catalog.cc:458] T 00000000000000000000000000000000 P b421e727773044528cb946bc1c2b5b83 [sys.catalog]: This master's current role is: LEARNER
I20260810 04:41:33.826793 2363 catalog_manager.cc:1259] Loaded cluster ID: 8f7937fec90543fcb67172c95103c8d3
I20260810 04:41:33.826833 2363 catalog_manager.cc:1552] T 00000000000000000000000000000000 P b421e727773044528cb946bc1c2b5b83: loading cluster ID for follower catalog manager: success
I20260810 04:41:33.827636 2363 catalog_manager.cc:1574] T 00000000000000000000000000000000 P b421e727773044528cb946bc1c2b5b83: acquiring CA information for follower catalog manager: success
I20260810 04:41:33.827877 2363 catalog_manager.cc:1602] T 00000000000000000000000000000000 P b421e727773044528cb946bc1c2b5b83: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260810 04:41:33.837782 2364 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:41:33.832002 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39312 (local address 127.22.167.125:46451)
0810 04:41:33.832102 (+ 100us) server_negotiation.cc:206] Beginning negotiation
0810 04:41:33.832104 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 04:41:33.832111 (+ 7us) server_negotiation.cc:407] Connection header received
0810 04:41:33.832131 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:41:33.832132 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:41:33.832142 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:41:33.832195 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0810 04:41:33.832700 (+ 505us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:41:33.833323 (+ 623us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:41:33.833997 (+ 674us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:41:33.834251 (+ 254us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:41:33.834302 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:41:33.835340 (+ 1038us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 04:41:33.835341 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 04:41:33.835342 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 04:41:33.835348 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0810 04:41:33.836729 (+ 1381us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 04:41:33.837046 (+ 317us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 04:41:33.837047 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 04:41:33.837048 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 04:41:33.837104 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 04:41:33.837270 (+ 166us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 04:41:33.837271 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 04:41:33.837271 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 04:41:33.837374 (+ 103us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 04:41:33.837461 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0810 04:41:33.837593 (+ 132us) server_negotiation.cc:299] Negotiation successful
0810 04:41:33.837682 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83,"thread_start_us":48,"threads_started":1}
WARNING: no policy specified for oryx/127.22.167.124@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.22.167.124@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.22.167.124 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786336863388938-23197-0/krb5kdc/oryx_127.22.167.124.keytab.
Entry for principal oryx/127.22.167.124 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786336863388938-23197-0/krb5kdc/oryx_127.22.167.124.keytab.
I20260810 04:41:33.864311 23197 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskemv_AM/build/release/bin/kudu
/tmp/dist-test-taskemv_AM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786336863388938-23197-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786336863388938-23197-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786336863388938-23197-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786336863388938-23197-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.167.124:39887
--webserver_interface=127.22.167.124
--webserver_port=0
--keytab_file=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786336863388938-23197-0/krb5kdc/oryx_127.22.167.124.keytab
--principal=oryx/127.22.167.124
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.167.84:33603
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.167.126:41961,127.22.167.125:46451,127.22.167.124:39887
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786336863388938-23197-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786336863388938-23197-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786336863388938-23197-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 04:41:33.952389 2369 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 04:41:33.952585 2369 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 04:41:33.952606 2369 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 04:41:33.953699 2369 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 04:41:33.953739 2369 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 04:41:33.953752 2369 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 04:41:33.953763 2369 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 04:41:33.953786 2369 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 04:41:33.955253 2369 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.167.84:33603
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786336863388938-23197-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786336863388938-23197-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.22.167.126:41961,127.22.167.125:46451,127.22.167.124:39887
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786336863388938-23197-0/krb5kdc/oryx_127.22.167.124.keytab
--principal=oryx/127.22.167.124
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.167.124:39887
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786336863388938-23197-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.22.167.124
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2369
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786336863388938-23197-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 03:54:01 UTC on ccycloud.quasar-icijmt.root.comops.site
build id 4577009
I20260810 04:41:33.955473 2369 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 04:41:33.955719 2369 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 04:41:33.960253 2377 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 04:41:33.960260 2381 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 04:41:33.960354 2369 server_base.cc:1034] running on GCE node
W20260810 04:41:33.960265 2379 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 04:41:33.960696 2369 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 04:41:33.960978 2369 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 04:41:33.962113 2369 hybrid_clock.cc:648] HybridClock initialized: now 1786336893962098 us; error 27 us; skew 500 ppm
Aug 10 04:41:33 dist-test-slave-bqcl krb5kdc[1217](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786336893, etypes {rep=17 tkt=17 ses=17}, oryx/127.22.167.124@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 04:41:33.964569 2369 init.cc:377] Logged in from keytab as oryx/127.22.167.124@KRBTEST.COM (short username oryx)
I20260810 04:41:33.964951 2369 webserver.cc:460] Webserver started at http://127.22.167.124:34357/ using document root <none> and password file <none>
I20260810 04:41:33.965154 2369 fs_manager.cc:359] Metadata directory not provided
I20260810 04:41:33.965194 2369 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 04:41:33.965298 2369 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 04:41:33.966282 2369 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786336863388938-23197-0/minicluster-data/master-2/data/instance:
uuid: "079feed1fb674f7fae482e7e9985a560"
format_stamp: "Formatted at 2026-08-10 04:41:33 on dist-test-slave-bqcl"
server_key: "f85cac7399b29515528b419b4ddfaf2e"
server_key_iv: "3c82f9ac2e00bb4cce3fb8917d9fc5ae"
server_key_version: "encryptionkey@0"
I20260810 04:41:33.966593 2369 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786336863388938-23197-0/minicluster-data/master-2/wal/instance:
uuid: "079feed1fb674f7fae482e7e9985a560"
format_stamp: "Formatted at 2026-08-10 04:41:33 on dist-test-slave-bqcl"
server_key: "f85cac7399b29515528b419b4ddfaf2e"
server_key_iv: "3c82f9ac2e00bb4cce3fb8917d9fc5ae"
server_key_version: "encryptionkey@0"
I20260810 04:41:33.967706 2369 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260810 04:41:33.968309 2395 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 04:41:33.968461 2369 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260810 04:41:33.968539 2369 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786336863388938-23197-0/minicluster-data/master-2/data,/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786336863388938-23197-0/minicluster-data/master-2/wal
uuid: "079feed1fb674f7fae482e7e9985a560"
format_stamp: "Formatted at 2026-08-10 04:41:33 on dist-test-slave-bqcl"
server_key: "f85cac7399b29515528b419b4ddfaf2e"
server_key_iv: "3c82f9ac2e00bb4cce3fb8917d9fc5ae"
server_key_version: "encryptionkey@0"
I20260810 04:41:33.968592 2369 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786336863388938-23197-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786336863388938-23197-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786336863388938-23197-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 04:41:33.975347 2369 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 04:41:33.976959 2369 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 04:41:33.977059 2369 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 04:41:33.980784 2369 rpc_server.cc:307] RPC server started. Bound to: 127.22.167.124:39887
I20260810 04:41:33.980818 2482 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.167.124:39887 every 8 connection(s)
I20260810 04:41:33.981117 2369 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786336863388938-23197-0/minicluster-data/master-2/data/info.pb
I20260810 04:41:33.982061 2484 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 04:41:33.983563 2484 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.22.167.126" port: 41961 } has no permanent_uuid. Determining permanent_uuid...
Aug 10 04:41:33 dist-test-slave-bqcl krb5kdc[1217](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786336893, etypes {rep=17 tkt=17 ses=17}, oryx/127.22.167.124@KRBTEST.COM for oryx/127.22.167.126@KRBTEST.COM
I20260810 04:41:33.989238 23197 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskemv_AM/build/release/bin/kudu as pid 2369
I20260810 04:41:33.992012 1368 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:41:33.984430 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48360 (local address 127.22.167.126:41961)
0810 04:41:33.984509 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0810 04:41:33.984511 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 04:41:33.984521 (+ 10us) server_negotiation.cc:407] Connection header received
0810 04:41:33.984880 (+ 359us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:41:33.984881 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:41:33.984890 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:41:33.984940 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0810 04:41:33.985497 (+ 557us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:41:33.985988 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:41:33.986929 (+ 941us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:41:33.987127 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:41:33.987181 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:41:33.989071 (+ 1890us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 04:41:33.989072 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 04:41:33.989089 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 04:41:33.989096 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 04:41:33.990797 (+ 1701us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 04:41:33.991150 (+ 353us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 04:41:33.991152 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 04:41:33.991152 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 04:41:33.991209 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 04:41:33.991419 (+ 210us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 04:41:33.991420 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 04:41:33.991421 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 04:41:33.991549 (+ 128us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 04:41:33.991614 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0810 04:41:33.991841 (+ 227us) server_negotiation.cc:299] Negotiation successful
0810 04:41:33.991919 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260810 04:41:33.992061 2488 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:41:33.984166 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.126:41961 (local address 127.0.0.1:48360)
0810 04:41:33.984339 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0810 04:41:33.984386 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0810 04:41:33.984803 (+ 417us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 04:41:33.985001 (+ 198us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 04:41:33.985004 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 04:41:33.985252 (+ 248us) client_negotiation.cc:189] Negotiated authn=SASL
0810 04:41:33.985435 (+ 183us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:41:33.985439 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:41:33.986046 (+ 607us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:41:33.986047 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:41:33.986861 (+ 814us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:41:33.986863 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:41:33.987195 (+ 332us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:41:33.987203 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:41:33.987307 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:41:33.987703 (+ 396us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 04:41:33.987711 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0810 04:41:33.988987 (+ 1276us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 04:41:33.990863 (+ 1876us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 04:41:33.990869 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 04:41:33.990871 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 04:41:33.991084 (+ 213us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 04:41:33.991262 (+ 178us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 04:41:33.991263 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 04:41:33.991263 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 04:41:33.991364 (+ 101us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 04:41:33.991609 (+ 245us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 04:41:33.991612 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 04:41:33.991726 (+ 114us) client_negotiation.cc:769] Sending connection context
0810 04:41:33.991804 (+ 78us) client_negotiation.cc:240] Negotiation successful
0810 04:41:33.991915 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":76,"threads_started":1}
I20260810 04:41:33.992519 2484 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.22.167.125" port: 46451 } has no permanent_uuid. Determining permanent_uuid...
Aug 10 04:41:33 dist-test-slave-bqcl krb5kdc[1217](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786336892, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.22.167.124@KRBTEST.COM
Aug 10 04:41:33 dist-test-slave-bqcl krb5kdc[1217](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786336893, etypes {rep=17 tkt=17 ses=17}, oryx/127.22.167.124@KRBTEST.COM for oryx/127.22.167.125@KRBTEST.COM
I20260810 04:41:33.997560 2489 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:41:33.989677 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38920 (local address 127.22.167.124:39887)
0810 04:41:33.989796 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0810 04:41:33.989801 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 04:41:33.989819 (+ 18us) server_negotiation.cc:407] Connection header received
0810 04:41:33.989837 (+ 18us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:41:33.989838 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:41:33.989848 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:41:33.989907 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0810 04:41:33.990646 (+ 739us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:41:33.991386 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:41:33.992302 (+ 916us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:41:33.992710 (+ 408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:41:33.992770 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:41:33.994989 (+ 2219us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 04:41:33.994997 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 04:41:33.994997 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 04:41:33.995003 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0810 04:41:33.996435 (+ 1432us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 04:41:33.996804 (+ 369us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 04:41:33.996807 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 04:41:33.996807 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 04:41:33.996860 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 04:41:33.997062 (+ 202us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 04:41:33.997063 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 04:41:33.997064 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 04:41:33.997189 (+ 125us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 04:41:33.997310 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0810 04:41:33.997392 (+ 82us) server_negotiation.cc:299] Negotiation successful
0810 04:41:33.997457 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":94,"thread_start_us":62,"threads_started":1}
I20260810 04:41:33.999009 2488 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:41:33.992793 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.125:46451 (local address 127.0.0.1:39318)
0810 04:41:33.992837 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0810 04:41:33.992846 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0810 04:41:33.992875 (+ 29us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 04:41:33.993156 (+ 281us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 04:41:33.993156 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 04:41:33.993288 (+ 132us) client_negotiation.cc:189] Negotiated authn=SASL
0810 04:41:33.993340 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:41:33.993342 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:41:33.993961 (+ 619us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:41:33.993962 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:41:33.994521 (+ 559us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:41:33.994523 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:41:33.994912 (+ 389us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:41:33.994913 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:41:33.994984 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:41:33.995296 (+ 312us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 04:41:33.995299 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0810 04:41:33.996460 (+ 1161us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 04:41:33.998100 (+ 1640us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 04:41:33.998101 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 04:41:33.998101 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 04:41:33.998292 (+ 191us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 04:41:33.998456 (+ 164us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 04:41:33.998457 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 04:41:33.998457 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 04:41:33.998511 (+ 54us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 04:41:33.998741 (+ 230us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 04:41:33.998742 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 04:41:33.998781 (+ 39us) client_negotiation.cc:769] Sending connection context
0810 04:41:33.998841 (+ 60us) client_negotiation.cc:240] Negotiation successful
0810 04:41:33.998924 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":31}
I20260810 04:41:33.999027 2364 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:41:33.992968 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39318 (local address 127.22.167.125:46451)
0810 04:41:33.993035 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0810 04:41:33.993037 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 04:41:33.993045 (+ 8us) server_negotiation.cc:407] Connection header received
0810 04:41:33.993072 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:41:33.993073 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:41:33.993085 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:41:33.993143 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0810 04:41:33.993411 (+ 268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:41:33.993877 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:41:33.994596 (+ 719us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:41:33.994842 (+ 246us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:41:33.994903 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:41:33.996682 (+ 1779us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 04:41:33.996683 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 04:41:33.996683 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 04:41:33.996691 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0810 04:41:33.998038 (+ 1347us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 04:41:33.998353 (+ 315us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 04:41:33.998354 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 04:41:33.998354 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 04:41:33.998405 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 04:41:33.998559 (+ 154us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 04:41:33.998560 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 04:41:33.998560 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 04:41:33.998676 (+ 116us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 04:41:33.998741 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0810 04:41:33.998870 (+ 129us) server_negotiation.cc:299] Negotiation successful
0810 04:41:33.998942 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260810 04:41:33.999429 2484 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.22.167.124" port: 39887 } has no permanent_uuid. Determining permanent_uuid...
Aug 10 04:41:33 dist-test-slave-bqcl krb5kdc[1217](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786336893, etypes {rep=17 tkt=17 ses=17}, oryx/127.22.167.124@KRBTEST.COM for oryx/127.22.167.124@KRBTEST.COM
I20260810 04:41:34.004590 2488 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:41:33.999622 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.124:39887 (local address 127.0.0.1:38934)
0810 04:41:33.999661 (+ 39us) negotiation.cc:107] Waiting for socket to connect
0810 04:41:33.999670 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0810 04:41:33.999706 (+ 36us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 04:41:33.999795 (+ 89us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 04:41:33.999796 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 04:41:33.999891 (+ 95us) client_negotiation.cc:189] Negotiated authn=SASL
0810 04:41:33.999915 (+ 24us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:41:33.999916 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:41:34.000511 (+ 595us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:41:34.000512 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:41:34.000959 (+ 447us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:41:34.000961 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:41:34.001300 (+ 339us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:41:34.001301 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:41:34.001333 (+ 32us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:41:34.001628 (+ 295us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 04:41:34.001631 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0810 04:41:34.002638 (+ 1007us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 04:41:34.003961 (+ 1323us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 04:41:34.003962 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 04:41:34.003962 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 04:41:34.004091 (+ 129us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 04:41:34.004225 (+ 134us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 04:41:34.004225 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 04:41:34.004226 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 04:41:34.004265 (+ 39us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 04:41:34.004414 (+ 149us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 04:41:34.004414 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 04:41:34.004442 (+ 28us) client_negotiation.cc:769] Sending connection context
0810 04:41:34.004484 (+ 42us) client_negotiation.cc:240] Negotiation successful
0810 04:41:34.004537 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":26}
I20260810 04:41:34.004668 2489 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:41:33.999651 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38934 (local address 127.22.167.124:39887)
0810 04:41:33.999708 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0810 04:41:33.999710 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 04:41:33.999737 (+ 27us) server_negotiation.cc:407] Connection header received
0810 04:41:33.999751 (+ 14us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:41:33.999751 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:41:33.999759 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:41:33.999792 (+ 33us) server_negotiation.cc:226] Negotiated authn=SASL
0810 04:41:33.999983 (+ 191us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:41:34.000451 (+ 468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:41:34.001046 (+ 595us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:41:34.001242 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:41:34.001298 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:41:34.002716 (+ 1418us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 04:41:34.002717 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 04:41:34.002717 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 04:41:34.002723 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0810 04:41:34.003904 (+ 1181us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 04:41:34.004138 (+ 234us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 04:41:34.004139 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 04:41:34.004139 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 04:41:34.004185 (+ 46us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 04:41:34.004300 (+ 115us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 04:41:34.004300 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 04:41:34.004300 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 04:41:34.004377 (+ 77us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 04:41:34.004416 (+ 39us) server_negotiation.cc:1035] Waiting for connection context
0810 04:41:34.004497 (+ 81us) server_negotiation.cc:299] Negotiation successful
0810 04:41:34.004579 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":7,"server-negotiator.queue_time_us":41}
I20260810 04:41:34.005718 2484 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 079feed1fb674f7fae482e7e9985a560: Bootstrap starting.
I20260810 04:41:34.006481 2484 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 079feed1fb674f7fae482e7e9985a560: Neither blocks nor log segments found. Creating new log.
I20260810 04:41:34.006735 2484 log.cc:824] T 00000000000000000000000000000000 P 079feed1fb674f7fae482e7e9985a560: Log is configured to *not* fsync() on all Append() calls
I20260810 04:41:34.007350 2484 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 079feed1fb674f7fae482e7e9985a560: No bootstrap required, opened a new log
I20260810 04:41:34.008705 2484 raft_consensus.cc:348] T 00000000000000000000000000000000 P 079feed1fb674f7fae482e7e9985a560 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b56636c33cce464eba3d4b547a7c704e" member_type: VOTER last_known_addr { host: "127.22.167.126" port: 41961 } } peers { permanent_uuid: "b421e727773044528cb946bc1c2b5b83" member_type: VOTER last_known_addr { host: "127.22.167.125" port: 46451 } } peers { permanent_uuid: "079feed1fb674f7fae482e7e9985a560" member_type: VOTER last_known_addr { host: "127.22.167.124" port: 39887 } }
I20260810 04:41:34.008821 2484 raft_consensus.cc:374] T 00000000000000000000000000000000 P 079feed1fb674f7fae482e7e9985a560 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 04:41:34.008852 2484 raft_consensus.cc:729] T 00000000000000000000000000000000 P 079feed1fb674f7fae482e7e9985a560 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 079feed1fb674f7fae482e7e9985a560, State: Initialized, Role: FOLLOWER
I20260810 04:41:34.008965 2484 consensus_queue.cc:260] T 00000000000000000000000000000000 P 079feed1fb674f7fae482e7e9985a560 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b56636c33cce464eba3d4b547a7c704e" member_type: VOTER last_known_addr { host: "127.22.167.126" port: 41961 } } peers { permanent_uuid: "b421e727773044528cb946bc1c2b5b83" member_type: VOTER last_known_addr { host: "127.22.167.125" port: 46451 } } peers { permanent_uuid: "079feed1fb674f7fae482e7e9985a560" member_type: VOTER last_known_addr { host: "127.22.167.124" port: 39887 } }
I20260810 04:41:34.009217 2484 sys_catalog.cc:564] T 00000000000000000000000000000000 P 079feed1fb674f7fae482e7e9985a560 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 04:41:34.009184 2491 sys_catalog.cc:455] T 00000000000000000000000000000000 P 079feed1fb674f7fae482e7e9985a560 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b56636c33cce464eba3d4b547a7c704e" member_type: VOTER last_known_addr { host: "127.22.167.126" port: 41961 } } peers { permanent_uuid: "b421e727773044528cb946bc1c2b5b83" member_type: VOTER last_known_addr { host: "127.22.167.125" port: 46451 } } peers { permanent_uuid: "079feed1fb674f7fae482e7e9985a560" member_type: VOTER last_known_addr { host: "127.22.167.124" port: 39887 } } }
I20260810 04:41:34.009261 2491 sys_catalog.cc:458] T 00000000000000000000000000000000 P 079feed1fb674f7fae482e7e9985a560 [sys.catalog]: This master's current role is: FOLLOWER
W20260810 04:41:34.010716 2508 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 079feed1fb674f7fae482e7e9985a560: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260810 04:41:34.010771 2508 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260810 04:41:34.012749 2369 master_runner.cc:428] Detected that this master 079feed1fb674f7fae482e7e9985a560 is joining an existing cluster
I20260810 04:41:34.012797 2369 master_runner.cc:432] Initiating AddMaster RPC to add 127.22.167.124:39887
I20260810 04:41:34.020186 1368 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:41:34.014042 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48372 (local address 127.22.167.126:41961)
0810 04:41:34.014140 (+ 98us) server_negotiation.cc:206] Beginning negotiation
0810 04:41:34.014143 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 04:41:34.014636 (+ 493us) server_negotiation.cc:407] Connection header received
0810 04:41:34.014757 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:41:34.014758 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:41:34.014769 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:41:34.014798 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0810 04:41:34.015218 (+ 420us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:41:34.015674 (+ 456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:41:34.016415 (+ 741us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:41:34.016601 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:41:34.016653 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:41:34.017639 (+ 986us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 04:41:34.017641 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 04:41:34.017641 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 04:41:34.017647 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0810 04:41:34.018956 (+ 1309us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 04:41:34.019436 (+ 480us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 04:41:34.019437 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 04:41:34.019437 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 04:41:34.019495 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 04:41:34.019701 (+ 206us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 04:41:34.019702 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 04:41:34.019702 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 04:41:34.019808 (+ 106us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 04:41:34.019841 (+ 33us) server_negotiation.cc:1035] Waiting for connection context
0810 04:41:34.020036 (+ 195us) server_negotiation.cc:299] Negotiation successful
0810 04:41:34.020102 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260810 04:41:34.020246 2518 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:41:34.014296 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.126:41961 (local address 127.0.0.1:48372)
0810 04:41:34.014580 (+ 284us) negotiation.cc:107] Waiting for socket to connect
0810 04:41:34.014589 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0810 04:41:34.014670 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 04:41:34.014876 (+ 206us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 04:41:34.014877 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 04:41:34.015080 (+ 203us) client_negotiation.cc:189] Negotiated authn=SASL
0810 04:41:34.015138 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:41:34.015139 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:41:34.015743 (+ 604us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:41:34.015744 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:41:34.016343 (+ 599us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:41:34.016345 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:41:34.016670 (+ 325us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:41:34.016670 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:41:34.016742 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:41:34.017105 (+ 363us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 04:41:34.017110 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0810 04:41:34.017546 (+ 436us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 04:41:34.019199 (+ 1653us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 04:41:34.019200 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 04:41:34.019201 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 04:41:34.019368 (+ 167us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 04:41:34.019565 (+ 197us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 04:41:34.019565 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 04:41:34.019566 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 04:41:34.019629 (+ 63us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 04:41:34.019875 (+ 246us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 04:41:34.019876 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 04:41:34.019938 (+ 62us) client_negotiation.cc:769] Sending connection context
0810 04:41:34.019991 (+ 53us) client_negotiation.cc:240] Negotiation successful
0810 04:41:34.020084 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":270,"thread_start_us":126,"threads_started":1}
I20260810 04:41:34.021521 2364 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:41:34.014424 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39326 (local address 127.22.167.125:46451)
0810 04:41:34.014464 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0810 04:41:34.014465 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0810 04:41:34.014632 (+ 167us) server_negotiation.cc:407] Connection header received
0810 04:41:34.014724 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:41:34.014726 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:41:34.014734 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:41:34.014762 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0810 04:41:34.015429 (+ 667us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:41:34.016142 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:41:34.017184 (+ 1042us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:41:34.017469 (+ 285us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:41:34.017538 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:41:34.018786 (+ 1248us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 04:41:34.018787 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 04:41:34.018787 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 04:41:34.018794 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 04:41:34.020297 (+ 1503us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 04:41:34.020702 (+ 405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 04:41:34.020703 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 04:41:34.020703 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 04:41:34.020764 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 04:41:34.020964 (+ 200us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 04:41:34.020964 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 04:41:34.020965 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 04:41:34.021096 (+ 131us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 04:41:34.021141 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0810 04:41:34.021346 (+ 205us) server_negotiation.cc:299] Negotiation successful
0810 04:41:34.021432 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260810 04:41:34.022197 2517 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:41:34.014244 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.125:46451 (local address 127.0.0.1:39326)
0810 04:41:34.014564 (+ 320us) negotiation.cc:107] Waiting for socket to connect
0810 04:41:34.014575 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0810 04:41:34.014657 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 04:41:34.014820 (+ 163us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 04:41:34.014821 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 04:41:34.015211 (+ 390us) client_negotiation.cc:189] Negotiated authn=SASL
0810 04:41:34.015332 (+ 121us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:41:34.015334 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:41:34.016236 (+ 902us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:41:34.016237 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:41:34.017081 (+ 844us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:41:34.017084 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:41:34.017548 (+ 464us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:41:34.017550 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:41:34.017631 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:41:34.018275 (+ 644us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 04:41:34.018280 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0810 04:41:34.018684 (+ 404us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 04:41:34.020388 (+ 1704us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 04:41:34.020389 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 04:41:34.020389 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 04:41:34.020616 (+ 227us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 04:41:34.020838 (+ 222us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 04:41:34.020839 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 04:41:34.020839 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 04:41:34.020897 (+ 58us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 04:41:34.021194 (+ 297us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 04:41:34.021195 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 04:41:34.021236 (+ 41us) client_negotiation.cc:769] Sending connection context
0810 04:41:34.021944 (+ 708us) client_negotiation.cc:240] Negotiation successful
0810 04:41:34.022062 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":270,"thread_start_us":271,"threads_started":1}
Aug 10 04:41:34 dist-test-slave-bqcl krb5kdc[1217](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786336893, etypes {rep=17 tkt=17 ses=17}, oryx/127.22.167.126@KRBTEST.COM for oryx/127.22.167.124@KRBTEST.COM
I20260810 04:41:34.045082 2226 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:41:34.023175 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.124:39887 (local address 127.0.0.1:38946)
0810 04:41:34.023221 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0810 04:41:34.023229 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0810 04:41:34.023264 (+ 35us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 04:41:34.023619 (+ 355us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 04:41:34.023621 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 04:41:34.023756 (+ 135us) client_negotiation.cc:189] Negotiated authn=SASL
0810 04:41:34.023788 (+ 32us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:41:34.023789 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:41:34.024799 (+ 1010us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:41:34.024800 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:41:34.025440 (+ 640us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:41:34.025442 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:41:34.025845 (+ 403us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:41:34.025847 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:41:34.025913 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:41:34.026322 (+ 409us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 04:41:34.026326 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0810 04:41:34.027516 (+ 1190us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 04:41:34.043939 (+ 16423us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 04:41:34.043940 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 04:41:34.043941 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 04:41:34.044105 (+ 164us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 04:41:34.044294 (+ 189us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 04:41:34.044295 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 04:41:34.044295 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 04:41:34.044372 (+ 77us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 04:41:34.044740 (+ 368us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 04:41:34.044741 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 04:41:34.044793 (+ 52us) client_negotiation.cc:769] Sending connection context
0810 04:41:34.044858 (+ 65us) client_negotiation.cc:240] Negotiation successful
0810 04:41:34.044966 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":33}
I20260810 04:41:34.045184 2489 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:41:34.014667 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38936 (local address 127.22.167.124:39887)
0810 04:41:34.014696 (+ 29us) server_negotiation.cc:206] Beginning negotiation
0810 04:41:34.014698 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 04:41:34.020351 (+ 5653us) server_negotiation.cc:407] Connection header received
0810 04:41:34.020489 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:41:34.020490 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:41:34.020497 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:41:34.020519 (+ 22us) server_negotiation.cc:226] Negotiated authn=SASL
0810 04:41:34.020870 (+ 351us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:41:34.021478 (+ 608us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:41:34.022291 (+ 813us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:41:34.022555 (+ 264us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:41:34.022591 (+ 36us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:41:34.042498 (+ 19907us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 04:41:34.042499 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 04:41:34.042499 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 04:41:34.042506 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 04:41:34.044099 (+ 1593us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 04:41:34.044435 (+ 336us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 04:41:34.044436 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 04:41:34.044436 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 04:41:34.044496 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 04:41:34.044695 (+ 199us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 04:41:34.044703 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 04:41:34.044704 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 04:41:34.044805 (+ 101us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 04:41:34.044855 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0810 04:41:34.044982 (+ 127us) server_negotiation.cc:299] Negotiation successful
0810 04:41:34.045091 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":47,"server-negotiator.queue_time_us":17}
I20260810 04:41:34.045334 2518 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:41:34.014778 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.124:39887 (local address 127.0.0.1:38936)
0810 04:41:34.020299 (+ 5521us) negotiation.cc:107] Waiting for socket to connect
0810 04:41:34.020310 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0810 04:41:34.020434 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 04:41:34.020558 (+ 124us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 04:41:34.020559 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 04:41:34.020732 (+ 173us) client_negotiation.cc:189] Negotiated authn=SASL
0810 04:41:34.020799 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:41:34.020801 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:41:34.021543 (+ 742us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:41:34.021543 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:41:34.022204 (+ 661us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:41:34.022207 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:41:34.022648 (+ 441us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:41:34.022648 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:41:34.022722 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:41:34.023120 (+ 398us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 04:41:34.023125 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0810 04:41:34.042403 (+ 19278us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 04:41:34.044193 (+ 1790us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 04:41:34.044194 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 04:41:34.044194 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 04:41:34.044370 (+ 176us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 04:41:34.044558 (+ 188us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 04:41:34.044559 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 04:41:34.044559 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 04:41:34.044625 (+ 66us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 04:41:34.044858 (+ 233us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 04:41:34.044858 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 04:41:34.044893 (+ 35us) client_negotiation.cc:769] Sending connection context
0810 04:41:34.044947 (+ 54us) client_negotiation.cc:240] Negotiation successful
0810 04:41:34.045041 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5510,"thread_start_us":204,"threads_started":1}
I20260810 04:41:34.045367 2521 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:41:34.023230 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38946 (local address 127.22.167.124:39887)
0810 04:41:34.023495 (+ 265us) server_negotiation.cc:206] Beginning negotiation
0810 04:41:34.023497 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 04:41:34.023503 (+ 6us) server_negotiation.cc:407] Connection header received
0810 04:41:34.023535 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:41:34.023535 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:41:34.023543 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:41:34.023597 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0810 04:41:34.023877 (+ 280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:41:34.024729 (+ 852us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:41:34.025507 (+ 778us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:41:34.025744 (+ 237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:41:34.025840 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:41:34.027680 (+ 1840us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 04:41:34.027681 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 04:41:34.027681 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 04:41:34.027687 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0810 04:41:34.043865 (+ 16178us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 04:41:34.044163 (+ 298us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 04:41:34.044164 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 04:41:34.044164 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 04:41:34.044221 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 04:41:34.044439 (+ 218us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 04:41:34.044440 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 04:41:34.044440 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 04:41:34.044671 (+ 231us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 04:41:34.044729 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0810 04:41:34.045180 (+ 451us) server_negotiation.cc:299] Negotiation successful
0810 04:41:34.045263 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":185,"thread_start_us":137,"threads_started":1}
I20260810 04:41:34.045879 1306 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.22.167.124:39887: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "079feed1fb674f7fae482e7e9985a560"
member_type: NON_VOTER
last_known_addr {
host: "127.22.167.124"
port: 39887
}
attrs {
promote: true
}
}
dest_uuid: "b56636c33cce464eba3d4b547a7c704e"
cas_config_opid_index: 5
I20260810 04:41:34.046168 1306 consensus_queue.cc:237] T 00000000000000000000000000000000 P b56636c33cce464eba3d4b547a7c704e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "b56636c33cce464eba3d4b547a7c704e" member_type: VOTER last_known_addr { host: "127.22.167.126" port: 41961 } } peers { permanent_uuid: "b421e727773044528cb946bc1c2b5b83" member_type: NON_VOTER last_known_addr { host: "127.22.167.125" port: 46451 } attrs { promote: true } } peers { permanent_uuid: "079feed1fb674f7fae482e7e9985a560" member_type: NON_VOTER last_known_addr { host: "127.22.167.124" port: 39887 } attrs { promote: true } }
I20260810 04:41:34.046526 2230 raft_consensus.cc:2932] T 00000000000000000000000000000000 P b56636c33cce464eba3d4b547a7c704e [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER 079feed1fb674f7fae482e7e9985a560 (127.22.167.124) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "b56636c33cce464eba3d4b547a7c704e" member_type: VOTER last_known_addr { host: "127.22.167.126" port: 41961 } } peers { permanent_uuid: "b421e727773044528cb946bc1c2b5b83" member_type: NON_VOTER last_known_addr { host: "127.22.167.125" port: 46451 } attrs { promote: true } } peers { permanent_uuid: "079feed1fb674f7fae482e7e9985a560" member_type: NON_VOTER last_known_addr { host: "127.22.167.124" port: 39887 } attrs { promote: true } } }
I20260810 04:41:34.046734 2436 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 079feed1fb674f7fae482e7e9985a560 [term 0 FOLLOWER]: Advancing to term 1
I20260810 04:41:34.047601 2230 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.22.167.124:39887
I20260810 04:41:34.047691 2436 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 079feed1fb674f7fae482e7e9985a560 [term 1 FOLLOWER]: Refusing update from remote peer b56636c33cce464eba3d4b547a7c704e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260810 04:41:34.047683 2230 sys_catalog.cc:455] T 00000000000000000000000000000000 P b56636c33cce464eba3d4b547a7c704e [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "b56636c33cce464eba3d4b547a7c704e" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "b56636c33cce464eba3d4b547a7c704e" member_type: VOTER last_known_addr { host: "127.22.167.126" port: 41961 } } peers { permanent_uuid: "b421e727773044528cb946bc1c2b5b83" member_type: NON_VOTER last_known_addr { host: "127.22.167.125" port: 46451 } attrs { promote: true } } peers { permanent_uuid: "079feed1fb674f7fae482e7e9985a560" member_type: NON_VOTER last_known_addr { host: "127.22.167.124" port: 39887 } attrs { promote: true } } }
I20260810 04:41:34.047742 2230 sys_catalog.cc:458] T 00000000000000000000000000000000 P b56636c33cce464eba3d4b547a7c704e [sys.catalog]: This master's current role is: LEADER
I20260810 04:41:34.047839 2369 master.cc:537] Master@127.22.167.124:39887 shutting down...
I20260810 04:41:34.047967 2227 consensus_queue.cc:1035] T 00000000000000000000000000000000 P b56636c33cce464eba3d4b547a7c704e [LEADER]: Connected to new peer: Peer: permanent_uuid: "079feed1fb674f7fae482e7e9985a560" member_type: NON_VOTER last_known_addr { host: "127.22.167.124" port: 39887 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 0, Time since last communication: 0.000s
W20260810 04:41:34.048852 1281 consensus_peers.cc:487] T 00000000000000000000000000000000 P b56636c33cce464eba3d4b547a7c704e -> Peer 079feed1fb674f7fae482e7e9985a560 (127.22.167.124:39887): Couldn't send request to peer 079feed1fb674f7fae482e7e9985a560. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260810 04:41:34.051123 2369 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 079feed1fb674f7fae482e7e9985a560 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 04:41:34.051218 2369 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 079feed1fb674f7fae482e7e9985a560 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 04:41:34.051270 2369 tablet_replica.cc:330] T 00000000000000000000000000000000 P 079feed1fb674f7fae482e7e9985a560: stopping tablet replica
I20260810 04:41:34.052716 2364 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:41:34.046676 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39342 (local address 127.22.167.125:46451)
0810 04:41:34.048992 (+ 2316us) server_negotiation.cc:206] Beginning negotiation
0810 04:41:34.048994 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 04:41:34.049013 (+ 19us) server_negotiation.cc:407] Connection header received
0810 04:41:34.049036 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:41:34.049037 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:41:34.049062 (+ 25us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:41:34.049119 (+ 57us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 04:41:34.049376 (+ 257us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:41:34.050076 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:41:34.051644 (+ 1568us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:41:34.052316 (+ 672us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:41:34.052382 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:41:34.052448 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0810 04:41:34.052621 (+ 173us) server_negotiation.cc:299] Negotiation successful
0810 04:41:34.052655 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2291}
I20260810 04:41:34.052716 2226 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:41:34.046581 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.125:46451 (local address 127.0.0.1:39342)
0810 04:41:34.046611 (+ 30us) negotiation.cc:107] Waiting for socket to connect
0810 04:41:34.046622 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0810 04:41:34.046658 (+ 36us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 04:41:34.049198 (+ 2540us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 04:41:34.049199 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 04:41:34.049201 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 04:41:34.049253 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:41:34.049255 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:41:34.050183 (+ 928us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:41:34.050184 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:41:34.051549 (+ 1365us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:41:34.051553 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:41:34.052478 (+ 925us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:41:34.052479 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:41:34.052540 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:41:34.052564 (+ 24us) client_negotiation.cc:769] Sending connection context
0810 04:41:34.052615 (+ 51us) client_negotiation.cc:240] Negotiation successful
0810 04:41:34.052650 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":16}
I20260810 04:41:34.053164 2308 raft_consensus.cc:1261] T 00000000000000000000000000000000 P b421e727773044528cb946bc1c2b5b83 [term 1 LEARNER]: Refusing update from remote peer b56636c33cce464eba3d4b547a7c704e: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260810 04:41:34.053359 2227 consensus_queue.cc:1035] T 00000000000000000000000000000000 P b56636c33cce464eba3d4b547a7c704e [LEADER]: Connected to new peer: Peer: permanent_uuid: "b421e727773044528cb946bc1c2b5b83" member_type: NON_VOTER last_known_addr { host: "127.22.167.125" port: 46451 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260810 04:41:34.053876 2308 raft_consensus.cc:2932] T 00000000000000000000000000000000 P b421e727773044528cb946bc1c2b5b83 [term 1 LEARNER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER 079feed1fb674f7fae482e7e9985a560 (127.22.167.124) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "b56636c33cce464eba3d4b547a7c704e" member_type: VOTER last_known_addr { host: "127.22.167.126" port: 41961 } } peers { permanent_uuid: "b421e727773044528cb946bc1c2b5b83" member_type: NON_VOTER last_known_addr { host: "127.22.167.125" port: 46451 } attrs { promote: true } } peers { permanent_uuid: "079feed1fb674f7fae482e7e9985a560" member_type: NON_VOTER last_known_addr { host: "127.22.167.124" port: 39887 } attrs { promote: true } } }
I20260810 04:41:34.054973 2227 raft_consensus.cc:1050] T 00000000000000000000000000000000 P b56636c33cce464eba3d4b547a7c704e: attempting to promote NON_VOTER b421e727773044528cb946bc1c2b5b83 to VOTER
I20260810 04:41:34.055023 2352 sys_catalog.cc:455] T 00000000000000000000000000000000 P b421e727773044528cb946bc1c2b5b83 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b56636c33cce464eba3d4b547a7c704e. Latest consensus state: current_term: 1 leader_uuid: "b56636c33cce464eba3d4b547a7c704e" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "b56636c33cce464eba3d4b547a7c704e" member_type: VOTER last_known_addr { host: "127.22.167.126" port: 41961 } } peers { permanent_uuid: "b421e727773044528cb946bc1c2b5b83" member_type: NON_VOTER last_known_addr { host: "127.22.167.125" port: 46451 } attrs { promote: true } } peers { permanent_uuid: "079feed1fb674f7fae482e7e9985a560" member_type: NON_VOTER last_known_addr { host: "127.22.167.124" port: 39887 } attrs { promote: true } } }
I20260810 04:41:34.055099 2352 sys_catalog.cc:458] T 00000000000000000000000000000000 P b421e727773044528cb946bc1c2b5b83 [sys.catalog]: This master's current role is: LEARNER
I20260810 04:41:34.055133 2352 sys_catalog.cc:455] T 00000000000000000000000000000000 P b421e727773044528cb946bc1c2b5b83 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "b56636c33cce464eba3d4b547a7c704e" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "b56636c33cce464eba3d4b547a7c704e" member_type: VOTER last_known_addr { host: "127.22.167.126" port: 41961 } } peers { permanent_uuid: "b421e727773044528cb946bc1c2b5b83" member_type: NON_VOTER last_known_addr { host: "127.22.167.125" port: 46451 } attrs { promote: true } } peers { permanent_uuid: "079feed1fb674f7fae482e7e9985a560" member_type: NON_VOTER last_known_addr { host: "127.22.167.124" port: 39887 } attrs { promote: true } } }
I20260810 04:41:34.055171 2352 sys_catalog.cc:458] T 00000000000000000000000000000000 P b421e727773044528cb946bc1c2b5b83 [sys.catalog]: This master's current role is: LEARNER
I20260810 04:41:34.055152 2227 consensus_queue.cc:237] T 00000000000000000000000000000000 P b56636c33cce464eba3d4b547a7c704e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "b56636c33cce464eba3d4b547a7c704e" member_type: VOTER last_known_addr { host: "127.22.167.126" port: 41961 } } peers { permanent_uuid: "b421e727773044528cb946bc1c2b5b83" member_type: VOTER last_known_addr { host: "127.22.167.125" port: 46451 } attrs { promote: false } } peers { permanent_uuid: "079feed1fb674f7fae482e7e9985a560" member_type: NON_VOTER last_known_addr { host: "127.22.167.124" port: 39887 } attrs { promote: true } }
I20260810 04:41:34.055574 2308 raft_consensus.cc:1261] T 00000000000000000000000000000000 P b421e727773044528cb946bc1c2b5b83 [term 1 LEARNER]: Refusing update from remote peer b56636c33cce464eba3d4b547a7c704e: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20260810 04:41:34.055634 2226 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:41:34.055534 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.124:39887 (local address 127.0.0.1:38962)
0810 04:41:34.055571 (+ 37us) negotiation.cc:107] Waiting for socket to connect
0810 04:41:34.055612 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.167.124:39887: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":26}
I20260810 04:41:34.055790 2227 consensus_queue.cc:1035] T 00000000000000000000000000000000 P b56636c33cce464eba3d4b547a7c704e [LEADER]: Connected to new peer: Peer: permanent_uuid: "b421e727773044528cb946bc1c2b5b83" member_type: VOTER last_known_addr { host: "127.22.167.125" port: 46451 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260810 04:41:34.055989 2226 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:41:34.055911 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.124:39887 (local address 127.0.0.1:38964)
0810 04:41:34.055945 (+ 34us) negotiation.cc:107] Waiting for socket to connect
0810 04:41:34.055967 (+ 22us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.167.124:39887: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":23}
W20260810 04:41:34.056041 1281 consensus_peers.cc:487] T 00000000000000000000000000000000 P b56636c33cce464eba3d4b547a7c704e -> Peer 079feed1fb674f7fae482e7e9985a560 (127.22.167.124:39887): Couldn't send request to peer 079feed1fb674f7fae482e7e9985a560. Status: Network error: Client connection negotiation failed: client connection to 127.22.167.124:39887: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260810 04:41:34.056352 2230 raft_consensus.cc:2932] T 00000000000000000000000000000000 P b56636c33cce464eba3d4b547a7c704e [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, b421e727773044528cb946bc1c2b5b83 (127.22.167.125) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "b56636c33cce464eba3d4b547a7c704e" member_type: VOTER last_known_addr { host: "127.22.167.126" port: 41961 } } peers { permanent_uuid: "b421e727773044528cb946bc1c2b5b83" member_type: VOTER last_known_addr { host: "127.22.167.125" port: 46451 } attrs { promote: false } } peers { permanent_uuid: "079feed1fb674f7fae482e7e9985a560" member_type: NON_VOTER last_known_addr { host: "127.22.167.124" port: 39887 } attrs { promote: true } } }
I20260810 04:41:34.056989 2230 sys_catalog.cc:455] T 00000000000000000000000000000000 P b56636c33cce464eba3d4b547a7c704e [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "b56636c33cce464eba3d4b547a7c704e" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "b56636c33cce464eba3d4b547a7c704e" member_type: VOTER last_known_addr { host: "127.22.167.126" port: 41961 } } peers { permanent_uuid: "b421e727773044528cb946bc1c2b5b83" member_type: VOTER last_known_addr { host: "127.22.167.125" port: 46451 } attrs { promote: false } } peers { permanent_uuid: "079feed1fb674f7fae482e7e9985a560" member_type: NON_VOTER last_known_addr { host: "127.22.167.124" port: 39887 } attrs { promote: true } } }
I20260810 04:41:34.057066 2230 sys_catalog.cc:458] T 00000000000000000000000000000000 P b56636c33cce464eba3d4b547a7c704e [sys.catalog]: This master's current role is: LEADER
I20260810 04:41:34.057016 2527 sys_catalog.cc:455] T 00000000000000000000000000000000 P b56636c33cce464eba3d4b547a7c704e [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "b56636c33cce464eba3d4b547a7c704e" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "b56636c33cce464eba3d4b547a7c704e" member_type: VOTER last_known_addr { host: "127.22.167.126" port: 41961 } } peers { permanent_uuid: "b421e727773044528cb946bc1c2b5b83" member_type: VOTER last_known_addr { host: "127.22.167.125" port: 46451 } attrs { promote: false } } peers { permanent_uuid: "079feed1fb674f7fae482e7e9985a560" member_type: NON_VOTER last_known_addr { host: "127.22.167.124" port: 39887 } attrs { promote: true } } }
I20260810 04:41:34.057109 2527 sys_catalog.cc:458] T 00000000000000000000000000000000 P b56636c33cce464eba3d4b547a7c704e [sys.catalog]: This master's current role is: LEADER
I20260810 04:41:34.057283 2308 raft_consensus.cc:2932] T 00000000000000000000000000000000 P b421e727773044528cb946bc1c2b5b83 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, b421e727773044528cb946bc1c2b5b83 (127.22.167.125) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "b56636c33cce464eba3d4b547a7c704e" member_type: VOTER last_known_addr { host: "127.22.167.126" port: 41961 } } peers { permanent_uuid: "b421e727773044528cb946bc1c2b5b83" member_type: VOTER last_known_addr { host: "127.22.167.125" port: 46451 } attrs { promote: false } } peers { permanent_uuid: "079feed1fb674f7fae482e7e9985a560" member_type: NON_VOTER last_known_addr { host: "127.22.167.124" port: 39887 } attrs { promote: true } } }
I20260810 04:41:34.057931 2523 sys_catalog.cc:455] T 00000000000000000000000000000000 P b421e727773044528cb946bc1c2b5b83 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "b56636c33cce464eba3d4b547a7c704e" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "b56636c33cce464eba3d4b547a7c704e" member_type: VOTER last_known_addr { host: "127.22.167.126" port: 41961 } } peers { permanent_uuid: "b421e727773044528cb946bc1c2b5b83" member_type: VOTER last_known_addr { host: "127.22.167.125" port: 46451 } attrs { promote: false } } peers { permanent_uuid: "079feed1fb674f7fae482e7e9985a560" member_type: NON_VOTER last_known_addr { host: "127.22.167.124" port: 39887 } attrs { promote: true } } }
I20260810 04:41:34.058048 2523 sys_catalog.cc:458] T 00000000000000000000000000000000 P b421e727773044528cb946bc1c2b5b83 [sys.catalog]: This master's current role is: FOLLOWER
I20260810 04:41:34.064002 2369 master.cc:552] Master@127.22.167.124:39887 shutdown complete.
I20260810 04:41:34.064040 2369 master_runner.cc:305] Clearing existing system tablet
I20260810 04:41:34.064147 2369 fs_manager.cc:359] Metadata directory not provided
I20260810 04:41:34.064191 2369 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 04:41:34.064831 2369 fs_manager.cc:709] Time spent opening directory manager: real 0.000s user 0.001s sys 0.000s
I20260810 04:41:34.065299 2529 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 04:41:34.065413 2369 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786336863388938-23197-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 04:41:34.065480 2369 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260810 04:41:34.065506 2369 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786336863388938-23197-0/minicluster-data/master-2/data,/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786336863388938-23197-0/minicluster-data/master-2/wal
uuid: "079feed1fb674f7fae482e7e9985a560"
format_stamp: "Formatted at 2026-08-10 04:41:33 on dist-test-slave-bqcl"
server_key: "f85cac7399b29515528b419b4ddfaf2e"
server_key_iv: "3c82f9ac2e00bb4cce3fb8917d9fc5ae"
server_key_version: "encryptionkey@0"
I20260810 04:41:34.066267 2369 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P 079feed1fb674f7fae482e7e9985a560: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 04:41:34.067533 2369 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P 079feed1fb674f7fae482e7e9985a560: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260810 04:41:34.067603 2369 log.cc:1196] T 00000000000000000000000000000000 P 079feed1fb674f7fae482e7e9985a560: Deleting WAL directory at /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786336863388938-23197-0/minicluster-data/master-2/wal/wals/00000000000000000000000000000000
I20260810 04:41:34.067799 2369 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P 079feed1fb674f7fae482e7e9985a560: Deleting consensus metadata
I20260810 04:41:34.067932 2369 master_runner.cc:315] Copying system tablet from 127.22.167.126:41961
I20260810 04:41:34.068512 2369 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P 079feed1fb674f7fae482e7e9985a560: tablet copy: Beginning tablet copy session from remote peer at address 127.22.167.126:41961
I20260810 04:41:34.074409 1368 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:41:34.068806 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48388 (local address 127.22.167.126:41961)
0810 04:41:34.068847 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0810 04:41:34.068850 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 04:41:34.068953 (+ 103us) server_negotiation.cc:407] Connection header received
0810 04:41:34.069016 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:41:34.069016 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:41:34.069024 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:41:34.069067 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0810 04:41:34.069420 (+ 353us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:41:34.069893 (+ 473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:41:34.071031 (+ 1138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:41:34.071210 (+ 179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:41:34.071260 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:41:34.072183 (+ 923us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 04:41:34.072184 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 04:41:34.072184 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 04:41:34.072190 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0810 04:41:34.073435 (+ 1245us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 04:41:34.073792 (+ 357us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 04:41:34.073792 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 04:41:34.073793 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 04:41:34.073837 (+ 44us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 04:41:34.073998 (+ 161us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 04:41:34.073998 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 04:41:34.073998 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 04:41:34.074092 (+ 94us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 04:41:34.074169 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0810 04:41:34.074294 (+ 125us) server_negotiation.cc:299] Negotiation successful
0810 04:41:34.074353 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260810 04:41:34.074476 2535 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:41:34.068741 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.126:41961 (local address 127.0.0.1:48388)
0810 04:41:34.068882 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0810 04:41:34.068892 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0810 04:41:34.068955 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 04:41:34.069074 (+ 119us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 04:41:34.069076 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 04:41:34.069284 (+ 208us) client_negotiation.cc:189] Negotiated authn=SASL
0810 04:41:34.069354 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:41:34.069356 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:41:34.069964 (+ 608us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:41:34.069965 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:41:34.070953 (+ 988us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:41:34.070957 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:41:34.071274 (+ 317us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:41:34.071275 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:41:34.071343 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:41:34.071731 (+ 388us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 04:41:34.071736 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0810 04:41:34.072108 (+ 372us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 04:41:34.073508 (+ 1400us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 04:41:34.073509 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 04:41:34.073509 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 04:41:34.073729 (+ 220us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 04:41:34.073886 (+ 157us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 04:41:34.073886 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 04:41:34.073887 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 04:41:34.073950 (+ 63us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 04:41:34.074155 (+ 205us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 04:41:34.074156 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 04:41:34.074200 (+ 44us) client_negotiation.cc:769] Sending connection context
0810 04:41:34.074273 (+ 73us) client_negotiation.cc:240] Negotiation successful
0810 04:41:34.074369 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":61,"threads_started":1}
I20260810 04:41:34.074680 1340 tablet_copy_service.cc:140] P b56636c33cce464eba3d4b547a7c704e: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 079feed1fb674f7fae482e7e9985a560 ({username='oryx', principal='oryx/127.22.167.124@KRBTEST.COM'} at 127.0.0.1:48388)
I20260810 04:41:34.074736 1340 tablet_copy_service.cc:161] P b56636c33cce464eba3d4b547a7c704e: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 079feed1fb674f7fae482e7e9985a560 at {username='oryx', principal='oryx/127.22.167.124@KRBTEST.COM'} at 127.0.0.1:48388: session id = 079feed1fb674f7fae482e7e9985a560-00000000000000000000000000000000
I20260810 04:41:34.075160 1340 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P b56636c33cce464eba3d4b547a7c704e: Tablet Copy: opened 0 blocks and 1 log segments
I20260810 04:41:34.075591 2369 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260810 04:41:34.076678 2369 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P 079feed1fb674f7fae482e7e9985a560: tablet copy: Starting download of 0 data blocks...
I20260810 04:41:34.076828 2369 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P 079feed1fb674f7fae482e7e9985a560: tablet copy: Starting download of 1 WAL segments...
I20260810 04:41:34.077529 2369 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P 079feed1fb674f7fae482e7e9985a560: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260810 04:41:34.078492 1340 tablet_copy_service.cc:342] P b56636c33cce464eba3d4b547a7c704e: Request end of tablet copy session 079feed1fb674f7fae482e7e9985a560-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.22.167.124@KRBTEST.COM'} at 127.0.0.1:48388
I20260810 04:41:34.078547 1340 tablet_copy_service.cc:433] P b56636c33cce464eba3d4b547a7c704e: ending tablet copy session 079feed1fb674f7fae482e7e9985a560-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 079feed1fb674f7fae482e7e9985a560
I20260810 04:41:34.080724 2369 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 04:41:34.081689 2536 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 04:41:34.081809 2541 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 04:41:34.081923 2369 server_base.cc:1034] running on GCE node
W20260810 04:41:34.081985 2537 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 04:41:34.082177 2369 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 04:41:34.082307 2369 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 04:41:34.083423 2369 hybrid_clock.cc:648] HybridClock initialized: now 1786336894083416 us; error 24 us; skew 500 ppm
Aug 10 04:41:34 dist-test-slave-bqcl krb5kdc[1217](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786336894, etypes {rep=17 tkt=17 ses=17}, oryx/127.22.167.124@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 04:41:34.084738 2369 init.cc:377] Logged in from keytab as oryx/127.22.167.124@KRBTEST.COM (short username oryx)
I20260810 04:41:34.084988 2369 webserver.cc:460] Webserver started at http://127.22.167.124:42789/ using document root <none> and password file <none>
I20260810 04:41:34.085093 2369 fs_manager.cc:359] Metadata directory not provided
I20260810 04:41:34.085126 2369 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 04:41:34.085772 2369 fs_manager.cc:709] Time spent opening directory manager: real 0.000s user 0.000s sys 0.001s
I20260810 04:41:34.086323 2550 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 04:41:34.086447 2369 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260810 04:41:34.086505 2369 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786336863388938-23197-0/minicluster-data/master-2/data,/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786336863388938-23197-0/minicluster-data/master-2/wal
uuid: "079feed1fb674f7fae482e7e9985a560"
format_stamp: "Formatted at 2026-08-10 04:41:33 on dist-test-slave-bqcl"
server_key: "f85cac7399b29515528b419b4ddfaf2e"
server_key_iv: "3c82f9ac2e00bb4cce3fb8917d9fc5ae"
server_key_version: "encryptionkey@0"
I20260810 04:41:34.086558 2369 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786336863388938-23197-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786336863388938-23197-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786336863388938-23197-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 04:41:34.114207 2369 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 04:41:34.115881 2369 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 04:41:34.122401 2369 rpc_server.cc:307] RPC server started. Bound to: 127.22.167.124:39887
I20260810 04:41:34.122437 2637 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.167.124:39887 every 8 connection(s)
I20260810 04:41:34.122788 2369 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786336863388938-23197-0/minicluster-data/master-2/data/info.pb
I20260810 04:41:34.123566 2638 sys_catalog.cc:263] Verifying existing consensus state
I20260810 04:41:34.123981 2638 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 079feed1fb674f7fae482e7e9985a560: Bootstrap starting.
I20260810 04:41:34.126562 2638 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 079feed1fb674f7fae482e7e9985a560: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260810 04:41:34.126754 2638 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 079feed1fb674f7fae482e7e9985a560: Bootstrap complete.
I20260810 04:41:34.126919 2638 raft_consensus.cc:348] T 00000000000000000000000000000000 P 079feed1fb674f7fae482e7e9985a560 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "b56636c33cce464eba3d4b547a7c704e" member_type: VOTER last_known_addr { host: "127.22.167.126" port: 41961 } } peers { permanent_uuid: "b421e727773044528cb946bc1c2b5b83" member_type: VOTER last_known_addr { host: "127.22.167.125" port: 46451 } attrs { promote: false } } peers { permanent_uuid: "079feed1fb674f7fae482e7e9985a560" member_type: NON_VOTER last_known_addr { host: "127.22.167.124" port: 39887 } attrs { promote: true } }
I20260810 04:41:34.126987 2638 raft_consensus.cc:729] T 00000000000000000000000000000000 P 079feed1fb674f7fae482e7e9985a560 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 079feed1fb674f7fae482e7e9985a560, State: Initialized, Role: LEARNER
I20260810 04:41:34.127027 2638 consensus_queue.cc:260] T 00000000000000000000000000000000 P 079feed1fb674f7fae482e7e9985a560 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "b56636c33cce464eba3d4b547a7c704e" member_type: VOTER last_known_addr { host: "127.22.167.126" port: 41961 } } peers { permanent_uuid: "b421e727773044528cb946bc1c2b5b83" member_type: VOTER last_known_addr { host: "127.22.167.125" port: 46451 } attrs { promote: false } } peers { permanent_uuid: "079feed1fb674f7fae482e7e9985a560" member_type: NON_VOTER last_known_addr { host: "127.22.167.124" port: 39887 } attrs { promote: true } }
I20260810 04:41:34.127200 2638 sys_catalog.cc:564] T 00000000000000000000000000000000 P 079feed1fb674f7fae482e7e9985a560 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 04:41:34.127185 2644 sys_catalog.cc:455] T 00000000000000000000000000000000 P 079feed1fb674f7fae482e7e9985a560 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "b56636c33cce464eba3d4b547a7c704e" member_type: VOTER last_known_addr { host: "127.22.167.126" port: 41961 } } peers { permanent_uuid: "b421e727773044528cb946bc1c2b5b83" member_type: VOTER last_known_addr { host: "127.22.167.125" port: 46451 } attrs { promote: false } } peers { permanent_uuid: "079feed1fb674f7fae482e7e9985a560" member_type: NON_VOTER last_known_addr { host: "127.22.167.124" port: 39887 } attrs { promote: true } } }
I20260810 04:41:34.127270 2644 sys_catalog.cc:458] T 00000000000000000000000000000000 P 079feed1fb674f7fae482e7e9985a560 [sys.catalog]: This master's current role is: LEARNER
I20260810 04:41:34.128945 2663 catalog_manager.cc:1259] Loaded cluster ID: 8f7937fec90543fcb67172c95103c8d3
I20260810 04:41:34.128983 2663 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 079feed1fb674f7fae482e7e9985a560: loading cluster ID for follower catalog manager: success
I20260810 04:41:34.129819 2663 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 079feed1fb674f7fae482e7e9985a560: acquiring CA information for follower catalog manager: success
I20260810 04:41:34.130074 2663 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 079feed1fb674f7fae482e7e9985a560: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260810 04:41:34.159577 2665 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:41:34.153012 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38994 (local address 127.22.167.124:39887)
0810 04:41:34.153128 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0810 04:41:34.153130 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 04:41:34.153138 (+ 8us) server_negotiation.cc:407] Connection header received
0810 04:41:34.153158 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:41:34.153159 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:41:34.153169 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:41:34.153241 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0810 04:41:34.153818 (+ 577us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:41:34.154558 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:41:34.155277 (+ 719us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:41:34.155510 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:41:34.155578 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:41:34.156803 (+ 1225us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 04:41:34.156804 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 04:41:34.156804 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 04:41:34.156812 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0810 04:41:34.158322 (+ 1510us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 04:41:34.158658 (+ 336us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 04:41:34.158659 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 04:41:34.158659 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 04:41:34.158769 (+ 110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 04:41:34.158962 (+ 193us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 04:41:34.158963 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 04:41:34.158963 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 04:41:34.159079 (+ 116us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 04:41:34.159175 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0810 04:41:34.159289 (+ 114us) server_negotiation.cc:299] Negotiation successful
0810 04:41:34.159368 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":97,"thread_start_us":62,"threads_started":1}
I20260810 04:41:34.160115 23197 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskemv_AM/build/release/bin/kudu with pid 1411
I20260810 04:41:34.182953 23197 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskemv_AM/build/release/bin/kudu
/tmp/dist-test-taskemv_AM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786336863388938-23197-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786336863388938-23197-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786336863388938-23197-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786336863388938-23197-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.167.65:43639
--local_ip_for_outbound_sockets=127.22.167.65
--tserver_master_addrs=127.22.167.126:41961
--webserver_port=45253
--webserver_interface=127.22.167.65
--keytab_file=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786336863388938-23197-0/krb5kdc/oryx_127.22.167.65.keytab
--principal=oryx/127.22.167.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.167.84:33603
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.22.167.126:41961,127.22.167.125:46451
--tserver_master_addrs=127.22.167.126:41961,127.22.167.125:46451,127.22.167.124:39887 with env {KRB5CCNAME=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786336863388938-23197-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786336863388938-23197-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786336863388938-23197-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 04:41:34.264674 2666 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 04:41:34.264859 2666 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 04:41:34.264881 2666 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 04:41:34.265987 2666 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 04:41:34.266022 2666 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 04:41:34.266047 2666 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.167.65
I20260810 04:41:34.267465 2666 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.167.84:33603
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786336863388938-23197-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786336863388938-23197-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786336863388938-23197-0/krb5kdc/oryx_127.22.167.65.keytab
--principal=oryx/127.22.167.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.167.65:43639
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786336863388938-23197-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.22.167.65
--webserver_port=45253
--tserver_master_addrs=127.22.167.126:41961,127.22.167.125:46451,127.22.167.124:39887
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2666
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.167.65
--log_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786336863388938-23197-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 03:54:01 UTC on ccycloud.quasar-icijmt.root.comops.site
build id 4577009
I20260810 04:41:34.267649 2666 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 04:41:34.267818 2666 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 04:41:34.272464 2676 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 04:41:34.272496 2666 server_base.cc:1034] running on GCE node
W20260810 04:41:34.272679 2678 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 04:41:34.272728 2675 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 04:41:34.272989 2666 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 04:41:34.273214 2666 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 04:41:34.274370 2666 hybrid_clock.cc:648] HybridClock initialized: now 1786336894274353 us; error 29 us; skew 500 ppm
Aug 10 04:41:34 dist-test-slave-bqcl krb5kdc[1217](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786336894, etypes {rep=17 tkt=17 ses=17}, oryx/127.22.167.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 04:41:34.277541 2666 init.cc:377] Logged in from keytab as oryx/127.22.167.65@KRBTEST.COM (short username oryx)
I20260810 04:41:34.278039 2666 webserver.cc:460] Webserver started at http://127.22.167.65:45253/ using document root <none> and password file <none>
I20260810 04:41:34.278285 2666 fs_manager.cc:359] Metadata directory not provided
I20260810 04:41:34.278359 2666 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 04:41:34.280043 2666 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260810 04:41:34.280706 2691 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 04:41:34.280871 2666 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260810 04:41:34.280958 2666 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786336863388938-23197-0/minicluster-data/ts-0/data,/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786336863388938-23197-0/minicluster-data/ts-0/wal
uuid: "5e88101ae4ec49b2a0be733b6f18ea08"
format_stamp: "Formatted at 2026-08-10 04:41:33 on dist-test-slave-bqcl"
server_key: "598bfa07cd51f858e573855e59168a93"
server_key_iv: "17fc29583466f0ffe38c4a4feafabfb6"
server_key_version: "encryptionkey@0"
I20260810 04:41:34.281288 2666 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786336863388938-23197-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786336863388938-23197-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786336863388938-23197-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 04:41:34.292150 2666 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 04:41:34.294328 2666 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 04:41:34.294461 2666 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 04:41:34.294654 2666 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 04:41:34.294929 2666 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 04:41:34.294960 2666 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 04:41:34.294979 2666 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 04:41:34.294993 2666 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 04:41:34.305594 2666 rpc_server.cc:307] RPC server started. Bound to: 127.22.167.65:43639
I20260810 04:41:34.305716 2859 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.167.65:43639 every 8 connection(s)
I20260810 04:41:34.305979 2666 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786336863388938-23197-0/minicluster-data/ts-0/data/info.pb
I20260810 04:41:34.308535 23197 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskemv_AM/build/release/bin/kudu as pid 2666
I20260810 04:41:34.308714 23197 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskemv_AM/build/release/bin/kudu with pid 1636
Aug 10 04:41:34 dist-test-slave-bqcl krb5kdc[1217](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786336894, etypes {rep=17 tkt=17 ses=17}, oryx/127.22.167.65@KRBTEST.COM for oryx/127.22.167.126@KRBTEST.COM
I20260810 04:41:34.322659 1368 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:41:34.307050 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.167.65:55835 (local address 127.22.167.126:41961)
0810 04:41:34.307396 (+ 346us) server_negotiation.cc:206] Beginning negotiation
0810 04:41:34.307398 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 04:41:34.307410 (+ 12us) server_negotiation.cc:407] Connection header received
0810 04:41:34.308247 (+ 837us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:41:34.308248 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:41:34.308260 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:41:34.308291 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0810 04:41:34.309390 (+ 1099us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:41:34.309987 (+ 597us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:41:34.311155 (+ 1168us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:41:34.311352 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:41:34.311382 (+ 30us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:41:34.315430 (+ 4048us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 04:41:34.315431 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 04:41:34.315431 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 04:41:34.315439 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0810 04:41:34.320375 (+ 4936us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 04:41:34.321530 (+ 1155us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 04:41:34.321532 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 04:41:34.321532 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 04:41:34.321603 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 04:41:34.321891 (+ 288us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 04:41:34.321893 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 04:41:34.321893 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 04:41:34.322040 (+ 147us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 04:41:34.322110 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0810 04:41:34.322468 (+ 358us) server_negotiation.cc:299] Negotiation successful
0810 04:41:34.322561 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":312}
I20260810 04:41:34.323194 2869 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:41:34.306964 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.126:41961 (local address 127.22.167.65:55835)
0810 04:41:34.307122 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0810 04:41:34.307132 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0810 04:41:34.308151 (+ 1019us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 04:41:34.308453 (+ 302us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 04:41:34.308457 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 04:41:34.308882 (+ 425us) client_negotiation.cc:189] Negotiated authn=SASL
0810 04:41:34.309289 (+ 407us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:41:34.309294 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:41:34.310070 (+ 776us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:41:34.310071 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:41:34.311068 (+ 997us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:41:34.311071 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:41:34.311448 (+ 377us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:41:34.311448 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:41:34.311578 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:41:34.313778 (+ 2200us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 04:41:34.313788 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0810 04:41:34.315316 (+ 1528us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 04:41:34.320563 (+ 5247us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 04:41:34.320571 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 04:41:34.320573 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 04:41:34.320871 (+ 298us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 04:41:34.321674 (+ 803us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 04:41:34.321675 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 04:41:34.321675 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 04:41:34.321815 (+ 140us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 04:41:34.322114 (+ 299us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 04:41:34.322141 (+ 27us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 04:41:34.322308 (+ 167us) client_negotiation.cc:769] Sending connection context
0810 04:41:34.322767 (+ 459us) client_negotiation.cc:240] Negotiation successful
0810 04:41:34.322939 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":92,"threads_started":1}
Aug 10 04:41:34 dist-test-slave-bqcl krb5kdc[1217](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786336894, etypes {rep=17 tkt=17 ses=17}, oryx/127.22.167.65@KRBTEST.COM for oryx/127.22.167.124@KRBTEST.COM
I20260810 04:41:34.323674 2861 heartbeater.cc:347] Connected to a master server at 127.22.167.126:41961
I20260810 04:41:34.323772 2861 heartbeater.cc:464] Registering TS with master...
I20260810 04:41:34.324080 2861 heartbeater.cc:511] Master 127.22.167.126:41961 requested a full tablet report, sending...
I20260810 04:41:34.324704 1306 ts_manager.cc:194] Re-registered known tserver with Master: 5e88101ae4ec49b2a0be733b6f18ea08 (127.22.167.65:43639)
I20260810 04:41:34.325177 1306 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.22.167.65@KRBTEST.COM'} at 127.22.167.65:55835
Aug 10 04:41:34 dist-test-slave-bqcl krb5kdc[1217](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786336894, etypes {rep=17 tkt=17 ses=17}, oryx/127.22.167.65@KRBTEST.COM for oryx/127.22.167.125@KRBTEST.COM
I20260810 04:41:34.326926 2665 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:41:34.306685 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.167.65:47041 (local address 127.22.167.124:39887)
0810 04:41:34.306751 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0810 04:41:34.306753 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 04:41:34.307011 (+ 258us) server_negotiation.cc:407] Connection header received
0810 04:41:34.308246 (+ 1235us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:41:34.308247 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:41:34.308259 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:41:34.308287 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0810 04:41:34.319433 (+ 11146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:41:34.320059 (+ 626us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:41:34.321092 (+ 1033us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:41:34.321404 (+ 312us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:41:34.321484 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:41:34.324213 (+ 2729us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 04:41:34.324215 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 04:41:34.324215 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 04:41:34.324226 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0810 04:41:34.325685 (+ 1459us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 04:41:34.326056 (+ 371us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 04:41:34.326057 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 04:41:34.326057 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 04:41:34.326115 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 04:41:34.326332 (+ 217us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 04:41:34.326334 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 04:41:34.326334 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 04:41:34.326501 (+ 167us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 04:41:34.326575 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0810 04:41:34.326759 (+ 184us) server_negotiation.cc:299] Negotiation successful
0810 04:41:34.326846 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260810 04:41:34.327209 2867 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:41:34.306670 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.124:39887 (local address 127.22.167.65:47041)
0810 04:41:34.306879 (+ 209us) negotiation.cc:107] Waiting for socket to connect
0810 04:41:34.306925 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0810 04:41:34.308128 (+ 1203us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 04:41:34.308453 (+ 325us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 04:41:34.308456 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 04:41:34.319148 (+ 10692us) client_negotiation.cc:189] Negotiated authn=SASL
0810 04:41:34.319327 (+ 179us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:41:34.319329 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:41:34.320177 (+ 848us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:41:34.320178 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:41:34.321001 (+ 823us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:41:34.321004 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:41:34.321486 (+ 482us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:41:34.321487 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:41:34.321568 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:41:34.322198 (+ 630us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 04:41:34.322203 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0810 04:41:34.324057 (+ 1854us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 04:41:34.325772 (+ 1715us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 04:41:34.325773 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 04:41:34.325774 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 04:41:34.325981 (+ 207us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 04:41:34.326207 (+ 226us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 04:41:34.326208 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 04:41:34.326208 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 04:41:34.326262 (+ 54us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 04:41:34.326567 (+ 305us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 04:41:34.326568 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 04:41:34.326621 (+ 53us) client_negotiation.cc:769] Sending connection context
0810 04:41:34.327031 (+ 410us) client_negotiation.cc:240] Negotiation successful
0810 04:41:34.327120 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":165,"thread_start_us":123,"threads_started":1}
I20260810 04:41:34.327589 2860 heartbeater.cc:347] Connected to a master server at 127.22.167.124:39887
I20260810 04:41:34.327647 2860 heartbeater.cc:464] Registering TS with master...
I20260810 04:41:34.327709 2860 heartbeater.cc:511] Master 127.22.167.124:39887 requested a full tablet report, sending...
I20260810 04:41:34.328086 2579 ts_manager.cc:194] Registered new tserver with Master: 5e88101ae4ec49b2a0be733b6f18ea08 (127.22.167.65:43639)
I20260810 04:41:34.329849 2364 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:41:34.307463 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.167.65:36869 (local address 127.22.167.125:46451)
0810 04:41:34.307509 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0810 04:41:34.307511 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 04:41:34.307519 (+ 8us) server_negotiation.cc:407] Connection header received
0810 04:41:34.308247 (+ 728us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:41:34.308248 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:41:34.308260 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:41:34.308291 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0810 04:41:34.321549 (+ 13258us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:41:34.322441 (+ 892us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:41:34.323604 (+ 1163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:41:34.323874 (+ 270us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:41:34.323921 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:41:34.327134 (+ 3213us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 04:41:34.327135 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 04:41:34.327135 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 04:41:34.327142 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 04:41:34.328600 (+ 1458us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 04:41:34.329062 (+ 462us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 04:41:34.329063 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 04:41:34.329064 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 04:41:34.329121 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 04:41:34.329322 (+ 201us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 04:41:34.329322 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 04:41:34.329322 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 04:41:34.329457 (+ 135us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 04:41:34.329507 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0810 04:41:34.329689 (+ 182us) server_negotiation.cc:299] Negotiation successful
0810 04:41:34.329767 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260810 04:41:34.329882 23197 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskemv_AM/build/release/bin/kudu
/tmp/dist-test-taskemv_AM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786336863388938-23197-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786336863388938-23197-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786336863388938-23197-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786336863388938-23197-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.167.66:46237
--local_ip_for_outbound_sockets=127.22.167.66
--tserver_master_addrs=127.22.167.126:41961
--webserver_port=35811
--webserver_interface=127.22.167.66
--keytab_file=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786336863388938-23197-0/krb5kdc/oryx_127.22.167.66.keytab
--principal=oryx/127.22.167.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.167.84:33603
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.22.167.126:41961,127.22.167.125:46451
--tserver_master_addrs=127.22.167.126:41961,127.22.167.125:46451,127.22.167.124:39887 with env {KRB5CCNAME=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786336863388938-23197-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786336863388938-23197-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786336863388938-23197-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 04:41:34.329936 2871 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:41:34.307182 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.125:46451 (local address 127.22.167.65:36869)
0810 04:41:34.307395 (+ 213us) negotiation.cc:107] Waiting for socket to connect
0810 04:41:34.307405 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0810 04:41:34.308134 (+ 729us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 04:41:34.320954 (+ 12820us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 04:41:34.320955 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 04:41:34.321243 (+ 288us) client_negotiation.cc:189] Negotiated authn=SASL
0810 04:41:34.321431 (+ 188us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:41:34.321434 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:41:34.322524 (+ 1090us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:41:34.322525 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:41:34.323516 (+ 991us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:41:34.323518 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:41:34.323999 (+ 481us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:41:34.324000 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:41:34.324112 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:41:34.325483 (+ 1371us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 04:41:34.325488 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0810 04:41:34.327038 (+ 1550us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 04:41:34.328695 (+ 1657us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 04:41:34.328697 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 04:41:34.328697 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 04:41:34.328935 (+ 238us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 04:41:34.329184 (+ 249us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 04:41:34.329185 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 04:41:34.329185 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 04:41:34.329260 (+ 75us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 04:41:34.329520 (+ 260us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 04:41:34.329521 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 04:41:34.329566 (+ 45us) client_negotiation.cc:769] Sending connection context
0810 04:41:34.329644 (+ 78us) client_negotiation.cc:240] Negotiation successful
0810 04:41:34.329741 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":80,"threads_started":1}
I20260810 04:41:34.330348 2864 heartbeater.cc:347] Connected to a master server at 127.22.167.125:46451
I20260810 04:41:34.330397 2864 heartbeater.cc:464] Registering TS with master...
I20260810 04:41:34.330457 2864 heartbeater.cc:511] Master 127.22.167.125:46451 requested a full tablet report, sending...
I20260810 04:41:34.330839 2291 ts_manager.cc:194] Registered new tserver with Master: 5e88101ae4ec49b2a0be733b6f18ea08 (127.22.167.65:43639)
W20260810 04:41:34.418196 2872 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 04:41:34.418383 2872 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 04:41:34.418404 2872 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 04:41:34.419476 2872 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 04:41:34.419515 2872 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 04:41:34.419540 2872 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.167.66
I20260810 04:41:34.420946 2872 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.167.84:33603
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786336863388938-23197-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786336863388938-23197-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786336863388938-23197-0/krb5kdc/oryx_127.22.167.66.keytab
--principal=oryx/127.22.167.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.167.66:46237
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786336863388938-23197-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.22.167.66
--webserver_port=35811
--tserver_master_addrs=127.22.167.126:41961,127.22.167.125:46451,127.22.167.124:39887
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2872
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.167.66
--log_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786336863388938-23197-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 03:54:01 UTC on ccycloud.quasar-icijmt.root.comops.site
build id 4577009
I20260810 04:41:34.421120 2872 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 04:41:34.421298 2872 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 04:41:34.426009 2890 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 04:41:34.426047 2880 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 04:41:34.426218 2872 server_base.cc:1034] running on GCE node
W20260810 04:41:34.425992 2879 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 04:41:34.426527 2872 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 04:41:34.426750 2872 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 04:41:34.427911 2872 hybrid_clock.cc:648] HybridClock initialized: now 1786336894427894 us; error 32 us; skew 500 ppm
Aug 10 04:41:34 dist-test-slave-bqcl krb5kdc[1217](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786336894, etypes {rep=17 tkt=17 ses=17}, oryx/127.22.167.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 04:41:34.430339 2872 init.cc:377] Logged in from keytab as oryx/127.22.167.66@KRBTEST.COM (short username oryx)
I20260810 04:41:34.430769 2872 webserver.cc:460] Webserver started at http://127.22.167.66:35811/ using document root <none> and password file <none>
I20260810 04:41:34.430965 2872 fs_manager.cc:359] Metadata directory not provided
I20260810 04:41:34.431041 2872 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 04:41:34.432268 2872 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260810 04:41:34.432883 2903 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 04:41:34.433039 2872 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260810 04:41:34.433128 2872 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786336863388938-23197-0/minicluster-data/ts-1/data,/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786336863388938-23197-0/minicluster-data/ts-1/wal
uuid: "7670dc9cfd0d4c25a8294f9115c4235b"
format_stamp: "Formatted at 2026-08-10 04:41:33 on dist-test-slave-bqcl"
server_key: "db6e40629773f391648ddcd2a6a2bfda"
server_key_iv: "7dbf250e0c88d6ab6870ddbf1159e293"
server_key_version: "encryptionkey@0"
I20260810 04:41:34.433418 2872 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786336863388938-23197-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786336863388938-23197-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786336863388938-23197-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 04:41:34.455127 2872 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 04:41:34.457096 2872 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 04:41:34.457226 2872 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 04:41:34.457417 2872 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 04:41:34.457690 2872 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 04:41:34.457723 2872 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 04:41:34.457751 2872 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 04:41:34.457775 2872 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 04:41:34.463260 2872 rpc_server.cc:307] RPC server started. Bound to: 127.22.167.66:46237
I20260810 04:41:34.463292 3079 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.167.66:46237 every 8 connection(s)
I20260810 04:41:34.463629 2872 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786336863388938-23197-0/minicluster-data/ts-1/data/info.pb
I20260810 04:41:34.465039 23197 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskemv_AM/build/release/bin/kudu as pid 2872
I20260810 04:41:34.465199 23197 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskemv_AM/build/release/bin/kudu with pid 1872
Aug 10 04:41:34 dist-test-slave-bqcl krb5kdc[1217](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786336894, etypes {rep=17 tkt=17 ses=17}, oryx/127.22.167.66@KRBTEST.COM for oryx/127.22.167.126@KRBTEST.COM
Aug 10 04:41:34 dist-test-slave-bqcl krb5kdc[1217](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786336894, etypes {rep=17 tkt=17 ses=17}, oryx/127.22.167.66@KRBTEST.COM for oryx/127.22.167.125@KRBTEST.COM
I20260810 04:41:34.476826 1368 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:41:34.464349 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.167.66:40007 (local address 127.22.167.126:41961)
0810 04:41:34.464407 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0810 04:41:34.464408 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0810 04:41:34.464738 (+ 330us) server_negotiation.cc:407] Connection header received
0810 04:41:34.468688 (+ 3950us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:41:34.468690 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:41:34.468717 (+ 27us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:41:34.468752 (+ 35us) server_negotiation.cc:226] Negotiated authn=SASL
0810 04:41:34.469736 (+ 984us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:41:34.470408 (+ 672us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:41:34.471584 (+ 1176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:41:34.471822 (+ 238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:41:34.471856 (+ 34us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:41:34.474065 (+ 2209us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 04:41:34.474067 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 04:41:34.474067 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 04:41:34.474074 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 04:41:34.475424 (+ 1350us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 04:41:34.475845 (+ 421us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 04:41:34.475846 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 04:41:34.475847 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 04:41:34.475894 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 04:41:34.476092 (+ 198us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 04:41:34.476093 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 04:41:34.476093 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 04:41:34.476212 (+ 119us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 04:41:34.476253 (+ 41us) server_negotiation.cc:1035] Waiting for connection context
0810 04:41:34.476637 (+ 384us) server_negotiation.cc:299] Negotiation successful
0810 04:41:34.476733 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260810 04:41:34.477320 3091 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:41:34.464344 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.126:41961 (local address 127.22.167.66:40007)
0810 04:41:34.464618 (+ 274us) negotiation.cc:107] Waiting for socket to connect
0810 04:41:34.464649 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0810 04:41:34.468557 (+ 3908us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 04:41:34.468869 (+ 312us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 04:41:34.468870 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 04:41:34.469355 (+ 485us) client_negotiation.cc:189] Negotiated authn=SASL
0810 04:41:34.469596 (+ 241us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:41:34.469605 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:41:34.470519 (+ 914us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:41:34.470519 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:41:34.471500 (+ 981us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:41:34.471502 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:41:34.471929 (+ 427us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:41:34.471930 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:41:34.472045 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:41:34.472544 (+ 499us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 04:41:34.472553 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0810 04:41:34.473972 (+ 1419us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 04:41:34.475531 (+ 1559us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 04:41:34.475538 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 04:41:34.475540 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 04:41:34.475777 (+ 237us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 04:41:34.475947 (+ 170us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 04:41:34.475947 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 04:41:34.475948 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 04:41:34.476042 (+ 94us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 04:41:34.476321 (+ 279us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 04:41:34.476328 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 04:41:34.476479 (+ 151us) client_negotiation.cc:769] Sending connection context
0810 04:41:34.476947 (+ 468us) client_negotiation.cc:240] Negotiation successful
0810 04:41:34.477108 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":235,"mutex_wait_us":3,"thread_start_us":139,"threads_started":1}
I20260810 04:41:34.477734 3081 heartbeater.cc:347] Connected to a master server at 127.22.167.126:41961
I20260810 04:41:34.477823 3081 heartbeater.cc:464] Registering TS with master...
I20260810 04:41:34.477998 3081 heartbeater.cc:511] Master 127.22.167.126:41961 requested a full tablet report, sending...
I20260810 04:41:34.478403 1306 ts_manager.cc:194] Re-registered known tserver with Master: 7670dc9cfd0d4c25a8294f9115c4235b (127.22.167.66:46237)
I20260810 04:41:34.478868 1306 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.22.167.66@KRBTEST.COM'} at 127.22.167.66:40007
I20260810 04:41:34.479015 2364 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:41:34.464362 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.167.66:53005 (local address 127.22.167.125:46451)
0810 04:41:34.464403 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0810 04:41:34.464405 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 04:41:34.464764 (+ 359us) server_negotiation.cc:407] Connection header received
0810 04:41:34.468658 (+ 3894us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:41:34.468660 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:41:34.468675 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:41:34.468702 (+ 27us) server_negotiation.cc:226] Negotiated authn=SASL
0810 04:41:34.469696 (+ 994us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:41:34.470673 (+ 977us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:41:34.471933 (+ 1260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:41:34.472620 (+ 687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:41:34.472669 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:41:34.475636 (+ 2967us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 04:41:34.475637 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 04:41:34.475638 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 04:41:34.475647 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0810 04:41:34.477537 (+ 1890us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 04:41:34.478002 (+ 465us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 04:41:34.478003 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 04:41:34.478003 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 04:41:34.478075 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 04:41:34.478353 (+ 278us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 04:41:34.478355 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 04:41:34.478355 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 04:41:34.478512 (+ 157us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 04:41:34.478564 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0810 04:41:34.478821 (+ 257us) server_negotiation.cc:299] Negotiation successful
0810 04:41:34.478914 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260810 04:41:34.479387 3090 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:41:34.464344 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.125:46451 (local address 127.22.167.66:53005)
0810 04:41:34.464618 (+ 274us) negotiation.cc:107] Waiting for socket to connect
0810 04:41:34.464651 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0810 04:41:34.468545 (+ 3894us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 04:41:34.468854 (+ 309us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 04:41:34.468857 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 04:41:34.469344 (+ 487us) client_negotiation.cc:189] Negotiated authn=SASL
0810 04:41:34.469594 (+ 250us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:41:34.469600 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:41:34.470783 (+ 1183us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:41:34.470784 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:41:34.471822 (+ 1038us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:41:34.471826 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:41:34.472762 (+ 936us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:41:34.472763 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:41:34.472838 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:41:34.473306 (+ 468us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 04:41:34.473312 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0810 04:41:34.475514 (+ 2202us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 04:41:34.477647 (+ 2133us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 04:41:34.477648 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 04:41:34.477649 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 04:41:34.477895 (+ 246us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 04:41:34.478190 (+ 295us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 04:41:34.478190 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 04:41:34.478191 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 04:41:34.478261 (+ 70us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 04:41:34.478620 (+ 359us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 04:41:34.478621 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 04:41:34.478670 (+ 49us) client_negotiation.cc:769] Sending connection context
0810 04:41:34.479148 (+ 478us) client_negotiation.cc:240] Negotiation successful
0810 04:41:34.479271 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":204,"spinlock_wait_cycles":7424,"thread_start_us":111,"threads_started":1}
I20260810 04:41:34.479779 3082 heartbeater.cc:347] Connected to a master server at 127.22.167.125:46451
I20260810 04:41:34.479825 3082 heartbeater.cc:464] Registering TS with master...
I20260810 04:41:34.479884 3082 heartbeater.cc:511] Master 127.22.167.125:46451 requested a full tablet report, sending...
I20260810 04:41:34.480274 2291 ts_manager.cc:194] Registered new tserver with Master: 7670dc9cfd0d4c25a8294f9115c4235b (127.22.167.66:46237)
Aug 10 04:41:34 dist-test-slave-bqcl krb5kdc[1217](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786336894, etypes {rep=17 tkt=17 ses=17}, oryx/127.22.167.66@KRBTEST.COM for oryx/127.22.167.124@KRBTEST.COM
I20260810 04:41:34.484449 2665 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:41:34.464783 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.167.66:45519 (local address 127.22.167.124:39887)
0810 04:41:34.464837 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0810 04:41:34.464839 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 04:41:34.464957 (+ 118us) server_negotiation.cc:407] Connection header received
0810 04:41:34.476254 (+ 11297us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:41:34.476256 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:41:34.476270 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:41:34.476298 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0810 04:41:34.476673 (+ 375us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:41:34.477363 (+ 690us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:41:34.478830 (+ 1467us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:41:34.479082 (+ 252us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:41:34.479123 (+ 41us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:41:34.482005 (+ 2882us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 04:41:34.482006 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 04:41:34.482006 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 04:41:34.482013 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 04:41:34.483421 (+ 1408us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 04:41:34.483759 (+ 338us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 04:41:34.483760 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 04:41:34.483761 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 04:41:34.483818 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 04:41:34.483997 (+ 179us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 04:41:34.483998 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 04:41:34.483998 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 04:41:34.484105 (+ 107us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 04:41:34.484158 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0810 04:41:34.484286 (+ 128us) server_negotiation.cc:299] Negotiation successful
0810 04:41:34.484363 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260810 04:41:34.484472 3092 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:41:34.464659 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.124:39887 (local address 127.22.167.66:45519)
0810 04:41:34.464889 (+ 230us) negotiation.cc:107] Waiting for socket to connect
0810 04:41:34.464898 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0810 04:41:34.476143 (+ 11245us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 04:41:34.476367 (+ 224us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 04:41:34.476367 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 04:41:34.476530 (+ 163us) client_negotiation.cc:189] Negotiated authn=SASL
0810 04:41:34.476590 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:41:34.476592 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:41:34.477912 (+ 1320us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:41:34.477913 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:41:34.478741 (+ 828us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:41:34.478744 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:41:34.479402 (+ 658us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:41:34.479403 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:41:34.479634 (+ 231us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:41:34.480022 (+ 388us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 04:41:34.480026 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0810 04:41:34.481919 (+ 1893us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 04:41:34.483492 (+ 1573us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 04:41:34.483493 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 04:41:34.483494 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 04:41:34.483650 (+ 156us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 04:41:34.483869 (+ 219us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 04:41:34.483870 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 04:41:34.483870 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 04:41:34.483946 (+ 76us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 04:41:34.484157 (+ 211us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 04:41:34.484158 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 04:41:34.484198 (+ 40us) client_negotiation.cc:769] Sending connection context
0810 04:41:34.484259 (+ 61us) client_negotiation.cc:240] Negotiation successful
0810 04:41:34.484362 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":215,"thread_start_us":151,"threads_started":1}
I20260810 04:41:34.484926 3080 heartbeater.cc:347] Connected to a master server at 127.22.167.124:39887
I20260810 04:41:34.484993 3080 heartbeater.cc:464] Registering TS with master...
I20260810 04:41:34.485045 3080 heartbeater.cc:511] Master 127.22.167.124:39887 requested a full tablet report, sending...
I20260810 04:41:34.485359 2579 ts_manager.cc:194] Registered new tserver with Master: 7670dc9cfd0d4c25a8294f9115c4235b (127.22.167.66:46237)
I20260810 04:41:34.490656 23197 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskemv_AM/build/release/bin/kudu
/tmp/dist-test-taskemv_AM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786336863388938-23197-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786336863388938-23197-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786336863388938-23197-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786336863388938-23197-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.167.67:41131
--local_ip_for_outbound_sockets=127.22.167.67
--tserver_master_addrs=127.22.167.126:41961
--webserver_port=44653
--webserver_interface=127.22.167.67
--keytab_file=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786336863388938-23197-0/krb5kdc/oryx_127.22.167.67.keytab
--principal=oryx/127.22.167.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.167.84:33603
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.22.167.126:41961,127.22.167.125:46451
--tserver_master_addrs=127.22.167.126:41961,127.22.167.125:46451,127.22.167.124:39887 with env {KRB5CCNAME=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786336863388938-23197-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786336863388938-23197-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786336863388938-23197-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 04:41:34.579007 3093 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 04:41:34.579216 3093 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 04:41:34.579250 3093 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 04:41:34.580399 3093 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 04:41:34.580436 3093 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 04:41:34.580470 3093 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.167.67
I20260810 04:41:34.582044 3093 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.167.84:33603
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786336863388938-23197-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786336863388938-23197-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786336863388938-23197-0/krb5kdc/oryx_127.22.167.67.keytab
--principal=oryx/127.22.167.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.167.67:41131
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786336863388938-23197-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.22.167.67
--webserver_port=44653
--tserver_master_addrs=127.22.167.126:41961,127.22.167.125:46451,127.22.167.124:39887
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3093
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.167.67
--log_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786336863388938-23197-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 03:54:01 UTC on ccycloud.quasar-icijmt.root.comops.site
build id 4577009
I20260810 04:41:34.582319 3093 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 04:41:34.582545 3093 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 04:41:34.588428 3106 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 04:41:34.588442 3102 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 04:41:34.588497 3103 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 04:41:34.588852 3093 server_base.cc:1034] running on GCE node
I20260810 04:41:34.589041 3093 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 04:41:34.589249 3093 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 04:41:34.590385 3093 hybrid_clock.cc:648] HybridClock initialized: now 1786336894590367 us; error 26 us; skew 500 ppm
Aug 10 04:41:34 dist-test-slave-bqcl krb5kdc[1217](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786336894, etypes {rep=17 tkt=17 ses=17}, oryx/127.22.167.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 04:41:34.593031 3093 init.cc:377] Logged in from keytab as oryx/127.22.167.67@KRBTEST.COM (short username oryx)
I20260810 04:41:34.593442 3093 webserver.cc:460] Webserver started at http://127.22.167.67:44653/ using document root <none> and password file <none>
I20260810 04:41:34.593647 3093 fs_manager.cc:359] Metadata directory not provided
I20260810 04:41:34.593710 3093 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 04:41:34.595227 3093 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260810 04:41:34.595808 3117 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 04:41:34.595961 3093 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260810 04:41:34.596053 3093 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786336863388938-23197-0/minicluster-data/ts-2/data,/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786336863388938-23197-0/minicluster-data/ts-2/wal
uuid: "1f3cb452053d447290369216f3e9ed93"
format_stamp: "Formatted at 2026-08-10 04:41:33 on dist-test-slave-bqcl"
server_key: "819e4ec0ea94ad7ab88abc284ec7d62d"
server_key_iv: "2703982d6d1d801dc367043ed5ff1a30"
server_key_version: "encryptionkey@0"
I20260810 04:41:34.596372 3093 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786336863388938-23197-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786336863388938-23197-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786336863388938-23197-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 04:41:34.605708 3093 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 04:41:34.607823 3093 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 04:41:34.607959 3093 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 04:41:34.608165 3093 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 04:41:34.608431 3093 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 04:41:34.608465 3093 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 04:41:34.608491 3093 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 04:41:34.608515 3093 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 04:41:34.615585 3093 rpc_server.cc:307] RPC server started. Bound to: 127.22.167.67:41131
I20260810 04:41:34.615676 3312 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.167.67:41131 every 8 connection(s)
I20260810 04:41:34.615991 3093 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786336863388938-23197-0/minicluster-data/ts-2/data/info.pb
Aug 10 04:41:34 dist-test-slave-bqcl krb5kdc[1217](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786336894, etypes {rep=17 tkt=17 ses=17}, oryx/127.22.167.67@KRBTEST.COM for oryx/127.22.167.124@KRBTEST.COM
Aug 10 04:41:34 dist-test-slave-bqcl krb5kdc[1217](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786336894, etypes {rep=17 tkt=17 ses=17}, oryx/127.22.167.67@KRBTEST.COM for oryx/127.22.167.126@KRBTEST.COM
Aug 10 04:41:34 dist-test-slave-bqcl krb5kdc[1217](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786336894, etypes {rep=17 tkt=17 ses=17}, oryx/127.22.167.67@KRBTEST.COM for oryx/127.22.167.125@KRBTEST.COM
I20260810 04:41:34.625612 23197 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskemv_AM/build/release/bin/kudu as pid 3093
I20260810 04:41:34.626462 2665 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:41:34.616976 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.167.67:53311 (local address 127.22.167.124:39887)
0810 04:41:34.617017 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0810 04:41:34.617019 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 04:41:34.617321 (+ 302us) server_negotiation.cc:407] Connection header received
0810 04:41:34.618202 (+ 881us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:41:34.618205 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:41:34.618218 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:41:34.618265 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0810 04:41:34.619108 (+ 843us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:41:34.619612 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:41:34.620898 (+ 1286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:41:34.621144 (+ 246us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:41:34.621191 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:41:34.623319 (+ 2128us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 04:41:34.623320 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 04:41:34.623321 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 04:41:34.623326 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0810 04:41:34.624767 (+ 1441us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 04:41:34.625275 (+ 508us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 04:41:34.625276 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 04:41:34.625276 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 04:41:34.625340 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 04:41:34.625643 (+ 303us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 04:41:34.625644 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 04:41:34.625644 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 04:41:34.625804 (+ 160us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 04:41:34.625849 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0810 04:41:34.626272 (+ 423us) server_negotiation.cc:299] Negotiation successful
0810 04:41:34.626372 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260810 04:41:34.626875 2364 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:41:34.617152 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.167.67:60207 (local address 127.22.167.125:46451)
0810 04:41:34.617190 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0810 04:41:34.617192 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 04:41:34.617316 (+ 124us) server_negotiation.cc:407] Connection header received
0810 04:41:34.618478 (+ 1162us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:41:34.618479 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:41:34.618491 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:41:34.618574 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0810 04:41:34.619115 (+ 541us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:41:34.619846 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:41:34.621235 (+ 1389us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:41:34.621409 (+ 174us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:41:34.621451 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:41:34.624232 (+ 2781us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 04:41:34.624233 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 04:41:34.624234 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 04:41:34.624240 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0810 04:41:34.625606 (+ 1366us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 04:41:34.626027 (+ 421us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 04:41:34.626029 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 04:41:34.626029 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 04:41:34.626085 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 04:41:34.626321 (+ 236us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 04:41:34.626322 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 04:41:34.626322 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 04:41:34.626444 (+ 122us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 04:41:34.626486 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0810 04:41:34.626688 (+ 202us) server_negotiation.cc:299] Negotiation successful
0810 04:41:34.626763 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260810 04:41:34.627038 3324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:41:34.616915 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.124:39887 (local address 127.22.167.67:53311)
0810 04:41:34.617252 (+ 337us) negotiation.cc:107] Waiting for socket to connect
0810 04:41:34.617262 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0810 04:41:34.618084 (+ 822us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 04:41:34.618351 (+ 267us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 04:41:34.618354 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 04:41:34.618801 (+ 447us) client_negotiation.cc:189] Negotiated authn=SASL
0810 04:41:34.619008 (+ 207us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:41:34.619010 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:41:34.619737 (+ 727us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:41:34.619738 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:41:34.620813 (+ 1075us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:41:34.620816 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:41:34.621210 (+ 394us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:41:34.621211 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:41:34.621266 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:41:34.621663 (+ 397us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 04:41:34.621673 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0810 04:41:34.623232 (+ 1559us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 04:41:34.624870 (+ 1638us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 04:41:34.624877 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 04:41:34.624879 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 04:41:34.625165 (+ 286us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 04:41:34.625410 (+ 245us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 04:41:34.625411 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 04:41:34.625411 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 04:41:34.625552 (+ 141us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 04:41:34.625897 (+ 345us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 04:41:34.625904 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 04:41:34.626056 (+ 152us) client_negotiation.cc:769] Sending connection context
0810 04:41:34.626587 (+ 531us) client_negotiation.cc:240] Negotiation successful
0810 04:41:34.626798 (+ 211us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":312,"mutex_wait_us":24,"thread_start_us":67,"threads_started":1}
I20260810 04:41:34.627184 3323 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:41:34.617108 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.125:46451 (local address 127.22.167.67:60207)
0810 04:41:34.617249 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0810 04:41:34.617258 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0810 04:41:34.618078 (+ 820us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 04:41:34.618599 (+ 521us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 04:41:34.618600 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 04:41:34.618838 (+ 238us) client_negotiation.cc:189] Negotiated authn=SASL
0810 04:41:34.618988 (+ 150us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:41:34.618996 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:41:34.619927 (+ 931us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:41:34.619928 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:41:34.620817 (+ 889us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:41:34.620819 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:41:34.621467 (+ 648us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:41:34.621468 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:41:34.621529 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:41:34.621993 (+ 464us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 04:41:34.621997 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0810 04:41:34.624121 (+ 2124us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 04:41:34.625692 (+ 1571us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 04:41:34.625693 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 04:41:34.625694 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 04:41:34.625957 (+ 263us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 04:41:34.626185 (+ 228us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 04:41:34.626185 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 04:41:34.626186 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 04:41:34.626252 (+ 66us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 04:41:34.626534 (+ 282us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 04:41:34.626535 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 04:41:34.626576 (+ 41us) client_negotiation.cc:769] Sending connection context
0810 04:41:34.627001 (+ 425us) client_negotiation.cc:240] Negotiation successful
0810 04:41:34.627091 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":127,"mutex_wait_us":34,"thread_start_us":79,"threads_started":1}
I20260810 04:41:34.627453 3313 heartbeater.cc:347] Connected to a master server at 127.22.167.124:39887
I20260810 04:41:34.627579 3313 heartbeater.cc:464] Registering TS with master...
I20260810 04:41:34.627641 3319 heartbeater.cc:347] Connected to a master server at 127.22.167.125:46451
I20260810 04:41:34.627708 3319 heartbeater.cc:464] Registering TS with master...
I20260810 04:41:34.627761 3313 heartbeater.cc:511] Master 127.22.167.124:39887 requested a full tablet report, sending...
I20260810 04:41:34.627774 3319 heartbeater.cc:511] Master 127.22.167.125:46451 requested a full tablet report, sending...
I20260810 04:41:34.628185 2291 ts_manager.cc:194] Registered new tserver with Master: 1f3cb452053d447290369216f3e9ed93 (127.22.167.67:41131)
I20260810 04:41:34.628194 2579 ts_manager.cc:194] Registered new tserver with Master: 1f3cb452053d447290369216f3e9ed93 (127.22.167.67:41131)
I20260810 04:41:34.628935 1368 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:41:34.616645 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.167.67:55941 (local address 127.22.167.126:41961)
0810 04:41:34.616703 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0810 04:41:34.616705 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 04:41:34.616975 (+ 270us) server_negotiation.cc:407] Connection header received
0810 04:41:34.618275 (+ 1300us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:41:34.618277 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:41:34.618328 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:41:34.618408 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0810 04:41:34.619093 (+ 685us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:41:34.619850 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:41:34.620721 (+ 871us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:41:34.620929 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:41:34.620987 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:41:34.624235 (+ 3248us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 04:41:34.624237 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 04:41:34.624237 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 04:41:34.624246 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0810 04:41:34.626071 (+ 1825us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 04:41:34.628141 (+ 2070us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 04:41:34.628142 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 04:41:34.628142 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 04:41:34.628204 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 04:41:34.628412 (+ 208us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 04:41:34.628414 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 04:41:34.628414 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 04:41:34.628546 (+ 132us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 04:41:34.628585 (+ 39us) server_negotiation.cc:1035] Waiting for connection context
0810 04:41:34.628794 (+ 209us) server_negotiation.cc:299] Negotiation successful
0810 04:41:34.628865 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260810 04:41:34.629365 3321 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:41:34.616640 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.126:41961 (local address 127.22.167.67:55941)
0810 04:41:34.616853 (+ 213us) negotiation.cc:107] Waiting for socket to connect
0810 04:41:34.616893 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0810 04:41:34.618068 (+ 1175us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 04:41:34.618438 (+ 370us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 04:41:34.618439 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 04:41:34.618800 (+ 361us) client_negotiation.cc:189] Negotiated authn=SASL
0810 04:41:34.618986 (+ 186us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:41:34.618991 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:41:34.619955 (+ 964us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:41:34.619956 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:41:34.620643 (+ 687us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:41:34.620645 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:41:34.621003 (+ 358us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:41:34.621004 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:41:34.621117 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:41:34.621824 (+ 707us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 04:41:34.621829 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0810 04:41:34.624121 (+ 2292us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 04:41:34.626196 (+ 2075us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 04:41:34.626196 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 04:41:34.626197 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 04:41:34.628062 (+ 1865us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 04:41:34.628277 (+ 215us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 04:41:34.628277 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 04:41:34.628278 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 04:41:34.628337 (+ 59us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 04:41:34.628639 (+ 302us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 04:41:34.628640 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 04:41:34.628679 (+ 39us) client_negotiation.cc:769] Sending connection context
0810 04:41:34.629060 (+ 381us) client_negotiation.cc:240] Negotiation successful
0810 04:41:34.629152 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":165,"thread_start_us":75,"threads_started":1}
I20260810 04:41:34.630298 3314 heartbeater.cc:347] Connected to a master server at 127.22.167.126:41961
I20260810 04:41:34.630347 3314 heartbeater.cc:464] Registering TS with master...
I20260810 04:41:34.630409 3314 heartbeater.cc:511] Master 127.22.167.126:41961 requested a full tablet report, sending...
I20260810 04:41:34.630626 1306 ts_manager.cc:194] Re-registered known tserver with Master: 1f3cb452053d447290369216f3e9ed93 (127.22.167.67:41131)
I20260810 04:41:34.631114 1306 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.22.167.67@KRBTEST.COM'} at 127.22.167.67:55941
I20260810 04:41:34.634948 2364 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:41:34.627025 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39352 (local address 127.22.167.125:46451)
0810 04:41:34.627059 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0810 04:41:34.627061 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 04:41:34.627653 (+ 592us) server_negotiation.cc:407] Connection header received
0810 04:41:34.627788 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:41:34.627789 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:41:34.627798 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:41:34.627826 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0810 04:41:34.628739 (+ 913us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:41:34.629414 (+ 675us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:41:34.630628 (+ 1214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:41:34.630896 (+ 268us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:41:34.630934 (+ 38us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:41:34.632466 (+ 1532us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 04:41:34.632467 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 04:41:34.632468 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 04:41:34.632474 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0810 04:41:34.633815 (+ 1341us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 04:41:34.634178 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 04:41:34.634179 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 04:41:34.634180 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 04:41:34.634244 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 04:41:34.634438 (+ 194us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 04:41:34.634439 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 04:41:34.634439 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 04:41:34.634560 (+ 121us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 04:41:34.634652 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0810 04:41:34.634795 (+ 143us) server_negotiation.cc:299] Negotiation successful
0810 04:41:34.634865 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260810 04:41:34.636714 3330 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:41:34.627292 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48392 (local address 127.22.167.126:41961)
0810 04:41:34.628978 (+ 1686us) server_negotiation.cc:206] Beginning negotiation
0810 04:41:34.628980 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 04:41:34.628988 (+ 8us) server_negotiation.cc:407] Connection header received
0810 04:41:34.629007 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:41:34.629008 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:41:34.629019 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:41:34.629727 (+ 708us) server_negotiation.cc:226] Negotiated authn=SASL
0810 04:41:34.629761 (+ 34us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:41:34.630532 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:41:34.631528 (+ 996us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:41:34.631822 (+ 294us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:41:34.631881 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:41:34.633119 (+ 1238us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 04:41:34.633121 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 04:41:34.633121 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 04:41:34.633128 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 04:41:34.635603 (+ 2475us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 04:41:34.636005 (+ 402us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 04:41:34.636006 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 04:41:34.636006 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 04:41:34.636069 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 04:41:34.636231 (+ 162us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 04:41:34.636232 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 04:41:34.636232 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 04:41:34.636342 (+ 110us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 04:41:34.636410 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0810 04:41:34.636548 (+ 138us) server_negotiation.cc:299] Negotiation successful
0810 04:41:34.636629 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1665,"thread_start_us":76,"threads_started":1}
I20260810 04:41:34.637269 2665 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:41:34.628768 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39002 (local address 127.22.167.124:39887)
0810 04:41:34.628801 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0810 04:41:34.628802 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0810 04:41:34.630334 (+ 1532us) server_negotiation.cc:407] Connection header received
0810 04:41:34.630535 (+ 201us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:41:34.630536 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:41:34.630546 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:41:34.630573 (+ 27us) server_negotiation.cc:226] Negotiated authn=SASL
0810 04:41:34.631590 (+ 1017us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:41:34.632432 (+ 842us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:41:34.633350 (+ 918us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:41:34.633632 (+ 282us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:41:34.633697 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:41:34.634850 (+ 1153us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 04:41:34.634851 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 04:41:34.634851 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 04:41:34.634858 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 04:41:34.636195 (+ 1337us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 04:41:34.636482 (+ 287us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 04:41:34.636483 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 04:41:34.636483 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 04:41:34.636541 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 04:41:34.636749 (+ 208us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 04:41:34.636750 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 04:41:34.636750 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 04:41:34.636863 (+ 113us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 04:41:34.636930 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0810 04:41:34.637126 (+ 196us) server_negotiation.cc:299] Negotiation successful
0810 04:41:34.637188 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260810 04:41:34.638247 1306 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48392:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260810 04:41:34.640136 3337 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:41:34.636707 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.124:39887 (local address 127.0.0.1:39010)
0810 04:41:34.637068 (+ 361us) negotiation.cc:107] Waiting for socket to connect
0810 04:41:34.637076 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0810 04:41:34.637130 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 04:41:34.637305 (+ 175us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 04:41:34.637307 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 04:41:34.637309 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 04:41:34.637346 (+ 37us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:41:34.637347 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:41:34.638307 (+ 960us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:41:34.638308 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:41:34.639354 (+ 1046us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:41:34.639357 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:41:34.639894 (+ 537us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:41:34.639895 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:41:34.639954 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:41:34.639958 (+ 4us) client_negotiation.cc:769] Sending connection context
0810 04:41:34.640036 (+ 78us) client_negotiation.cc:240] Negotiation successful
0810 04:41:34.640075 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":349,"thread_start_us":52,"threads_started":1}
W20260810 04:41:34.640203 1306 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 04:41:34.640239 3339 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:41:34.636845 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39010 (local address 127.22.167.124:39887)
0810 04:41:34.636994 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0810 04:41:34.636995 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0810 04:41:34.637181 (+ 186us) server_negotiation.cc:407] Connection header received
0810 04:41:34.637206 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:41:34.637207 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:41:34.637230 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:41:34.637296 (+ 66us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 04:41:34.637583 (+ 287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:41:34.638244 (+ 661us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:41:34.639436 (+ 1192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:41:34.639820 (+ 384us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:41:34.639875 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:41:34.639920 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0810 04:41:34.640138 (+ 218us) server_negotiation.cc:299] Negotiation successful
0810 04:41:34.640176 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":134,"thread_start_us":60,"threads_started":1}
I20260810 04:41:34.640439 2593 raft_consensus.cc:1203] T 00000000000000000000000000000000 P 079feed1fb674f7fae482e7e9985a560 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.6->[1.7-1.7] Dedup: 1.7->[]
I20260810 04:41:34.640669 3342 sys_catalog.cc:455] T 00000000000000000000000000000000 P 079feed1fb674f7fae482e7e9985a560 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b56636c33cce464eba3d4b547a7c704e. Latest consensus state: current_term: 1 leader_uuid: "b56636c33cce464eba3d4b547a7c704e" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "b56636c33cce464eba3d4b547a7c704e" member_type: VOTER last_known_addr { host: "127.22.167.126" port: 41961 } } peers { permanent_uuid: "b421e727773044528cb946bc1c2b5b83" member_type: VOTER last_known_addr { host: "127.22.167.125" port: 46451 } attrs { promote: false } } peers { permanent_uuid: "079feed1fb674f7fae482e7e9985a560" member_type: NON_VOTER last_known_addr { host: "127.22.167.124" port: 39887 } attrs { promote: true } } }
I20260810 04:41:34.640854 3342 sys_catalog.cc:458] T 00000000000000000000000000000000 P 079feed1fb674f7fae482e7e9985a560 [sys.catalog]: This master's current role is: LEARNER
I20260810 04:41:34.640992 3123 sys_catalog.cc:455] T 00000000000000000000000000000000 P b56636c33cce464eba3d4b547a7c704e [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "b56636c33cce464eba3d4b547a7c704e" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "b56636c33cce464eba3d4b547a7c704e" member_type: VOTER last_known_addr { host: "127.22.167.126" port: 41961 } } peers { permanent_uuid: "b421e727773044528cb946bc1c2b5b83" member_type: VOTER last_known_addr { host: "127.22.167.125" port: 46451 } attrs { promote: false } } peers { permanent_uuid: "079feed1fb674f7fae482e7e9985a560" member_type: NON_VOTER last_known_addr { host: "127.22.167.124" port: 39887 } attrs { promote: true } } }
I20260810 04:41:34.641088 3123 sys_catalog.cc:458] T 00000000000000000000000000000000 P b56636c33cce464eba3d4b547a7c704e [sys.catalog]: This master's current role is: LEADER
I20260810 04:41:34.642036 3345 raft_consensus.cc:1050] T 00000000000000000000000000000000 P b56636c33cce464eba3d4b547a7c704e: attempting to promote NON_VOTER 079feed1fb674f7fae482e7e9985a560 to VOTER
I20260810 04:41:34.642190 3345 consensus_queue.cc:237] T 00000000000000000000000000000000 P b56636c33cce464eba3d4b547a7c704e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 1.8, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "b56636c33cce464eba3d4b547a7c704e" member_type: VOTER last_known_addr { host: "127.22.167.126" port: 41961 } } peers { permanent_uuid: "b421e727773044528cb946bc1c2b5b83" member_type: VOTER last_known_addr { host: "127.22.167.125" port: 46451 } attrs { promote: false } } peers { permanent_uuid: "079feed1fb674f7fae482e7e9985a560" member_type: VOTER last_known_addr { host: "127.22.167.124" port: 39887 } attrs { promote: false } }
I20260810 04:41:34.642606 2593 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 079feed1fb674f7fae482e7e9985a560 [term 1 LEARNER]: Refusing update from remote peer b56636c33cce464eba3d4b547a7c704e: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20260810 04:41:34.642637 2305 raft_consensus.cc:1261] T 00000000000000000000000000000000 P b421e727773044528cb946bc1c2b5b83 [term 1 FOLLOWER]: Refusing update from remote peer b56636c33cce464eba3d4b547a7c704e: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20260810 04:41:34.642843 3351 consensus_queue.cc:1035] T 00000000000000000000000000000000 P b56636c33cce464eba3d4b547a7c704e [LEADER]: Connected to new peer: Peer: permanent_uuid: "079feed1fb674f7fae482e7e9985a560" member_type: VOTER last_known_addr { host: "127.22.167.124" port: 39887 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.000s
I20260810 04:41:34.642936 3123 consensus_queue.cc:1035] T 00000000000000000000000000000000 P b56636c33cce464eba3d4b547a7c704e [LEADER]: Connected to new peer: Peer: permanent_uuid: "b421e727773044528cb946bc1c2b5b83" member_type: VOTER last_known_addr { host: "127.22.167.125" port: 46451 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.000s
I20260810 04:41:34.643514 3123 raft_consensus.cc:2932] T 00000000000000000000000000000000 P b56636c33cce464eba3d4b547a7c704e [term 1 LEADER]: Committing config change with OpId 1.9: config changed from index 7 to 9, 079feed1fb674f7fae482e7e9985a560 (127.22.167.124) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "b56636c33cce464eba3d4b547a7c704e" member_type: VOTER last_known_addr { host: "127.22.167.126" port: 41961 } } peers { permanent_uuid: "b421e727773044528cb946bc1c2b5b83" member_type: VOTER last_known_addr { host: "127.22.167.125" port: 46451 } attrs { promote: false } } peers { permanent_uuid: "079feed1fb674f7fae482e7e9985a560" member_type: VOTER last_known_addr { host: "127.22.167.124" port: 39887 } attrs { promote: false } } }
I20260810 04:41:34.643661 2308 raft_consensus.cc:2932] T 00000000000000000000000000000000 P b421e727773044528cb946bc1c2b5b83 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 7 to 9, 079feed1fb674f7fae482e7e9985a560 (127.22.167.124) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "b56636c33cce464eba3d4b547a7c704e" member_type: VOTER last_known_addr { host: "127.22.167.126" port: 41961 } } peers { permanent_uuid: "b421e727773044528cb946bc1c2b5b83" member_type: VOTER last_known_addr { host: "127.22.167.125" port: 46451 } attrs { promote: false } } peers { permanent_uuid: "079feed1fb674f7fae482e7e9985a560" member_type: VOTER last_known_addr { host: "127.22.167.124" port: 39887 } attrs { promote: false } } }
I20260810 04:41:34.643913 2593 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 079feed1fb674f7fae482e7e9985a560 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 7 to 9, 079feed1fb674f7fae482e7e9985a560 (127.22.167.124) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "b56636c33cce464eba3d4b547a7c704e" member_type: VOTER last_known_addr { host: "127.22.167.126" port: 41961 } } peers { permanent_uuid: "b421e727773044528cb946bc1c2b5b83" member_type: VOTER last_known_addr { host: "127.22.167.125" port: 46451 } attrs { promote: false } } peers { permanent_uuid: "079feed1fb674f7fae482e7e9985a560" member_type: VOTER last_known_addr { host: "127.22.167.124" port: 39887 } attrs { promote: false } } }
I20260810 04:41:34.644451 3123 sys_catalog.cc:455] T 00000000000000000000000000000000 P b56636c33cce464eba3d4b547a7c704e [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "b56636c33cce464eba3d4b547a7c704e" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "b56636c33cce464eba3d4b547a7c704e" member_type: VOTER last_known_addr { host: "127.22.167.126" port: 41961 } } peers { permanent_uuid: "b421e727773044528cb946bc1c2b5b83" member_type: VOTER last_known_addr { host: "127.22.167.125" port: 46451 } attrs { promote: false } } peers { permanent_uuid: "079feed1fb674f7fae482e7e9985a560" member_type: VOTER last_known_addr { host: "127.22.167.124" port: 39887 } attrs { promote: false } } }
I20260810 04:41:34.644518 3123 sys_catalog.cc:458] T 00000000000000000000000000000000 P b56636c33cce464eba3d4b547a7c704e [sys.catalog]: This master's current role is: LEADER
I20260810 04:41:34.644564 3123 sys_catalog.cc:455] T 00000000000000000000000000000000 P b56636c33cce464eba3d4b547a7c704e [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "b56636c33cce464eba3d4b547a7c704e" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "b56636c33cce464eba3d4b547a7c704e" member_type: VOTER last_known_addr { host: "127.22.167.126" port: 41961 } } peers { permanent_uuid: "b421e727773044528cb946bc1c2b5b83" member_type: VOTER last_known_addr { host: "127.22.167.125" port: 46451 } attrs { promote: false } } peers { permanent_uuid: "079feed1fb674f7fae482e7e9985a560" member_type: VOTER last_known_addr { host: "127.22.167.124" port: 39887 } attrs { promote: false } } }
I20260810 04:41:34.644603 3123 sys_catalog.cc:458] T 00000000000000000000000000000000 P b56636c33cce464eba3d4b547a7c704e [sys.catalog]: This master's current role is: LEADER
I20260810 04:41:34.644515 3345 sys_catalog.cc:455] T 00000000000000000000000000000000 P b56636c33cce464eba3d4b547a7c704e [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "b56636c33cce464eba3d4b547a7c704e" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "b56636c33cce464eba3d4b547a7c704e" member_type: VOTER last_known_addr { host: "127.22.167.126" port: 41961 } } peers { permanent_uuid: "b421e727773044528cb946bc1c2b5b83" member_type: VOTER last_known_addr { host: "127.22.167.125" port: 46451 } attrs { promote: false } } peers { permanent_uuid: "079feed1fb674f7fae482e7e9985a560" member_type: VOTER last_known_addr { host: "127.22.167.124" port: 39887 } attrs { promote: false } } }
I20260810 04:41:34.644698 3345 sys_catalog.cc:458] T 00000000000000000000000000000000 P b56636c33cce464eba3d4b547a7c704e [sys.catalog]: This master's current role is: LEADER
I20260810 04:41:34.644750 3357 sys_catalog.cc:455] T 00000000000000000000000000000000 P b421e727773044528cb946bc1c2b5b83 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "b56636c33cce464eba3d4b547a7c704e" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "b56636c33cce464eba3d4b547a7c704e" member_type: VOTER last_known_addr { host: "127.22.167.126" port: 41961 } } peers { permanent_uuid: "b421e727773044528cb946bc1c2b5b83" member_type: VOTER last_known_addr { host: "127.22.167.125" port: 46451 } attrs { promote: false } } peers { permanent_uuid: "079feed1fb674f7fae482e7e9985a560" member_type: VOTER last_known_addr { host: "127.22.167.124" port: 39887 } attrs { promote: false } } }
I20260810 04:41:34.644814 3357 sys_catalog.cc:458] T 00000000000000000000000000000000 P b421e727773044528cb946bc1c2b5b83 [sys.catalog]: This master's current role is: FOLLOWER
I20260810 04:41:34.644973 3342 sys_catalog.cc:455] T 00000000000000000000000000000000 P 079feed1fb674f7fae482e7e9985a560 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "b56636c33cce464eba3d4b547a7c704e" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "b56636c33cce464eba3d4b547a7c704e" member_type: VOTER last_known_addr { host: "127.22.167.126" port: 41961 } } peers { permanent_uuid: "b421e727773044528cb946bc1c2b5b83" member_type: VOTER last_known_addr { host: "127.22.167.125" port: 46451 } attrs { promote: false } } peers { permanent_uuid: "079feed1fb674f7fae482e7e9985a560" member_type: VOTER last_known_addr { host: "127.22.167.124" port: 39887 } attrs { promote: false } } }
I20260810 04:41:34.645047 3342 sys_catalog.cc:458] T 00000000000000000000000000000000 P 079feed1fb674f7fae482e7e9985a560 [sys.catalog]: This master's current role is: FOLLOWER
I20260810 04:41:34.649104 3337 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:41:34.645648 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.65:43639 (local address 127.0.0.1:34598)
0810 04:41:34.645692 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0810 04:41:34.645702 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0810 04:41:34.645739 (+ 37us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 04:41:34.646051 (+ 312us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 04:41:34.646052 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 04:41:34.646054 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 04:41:34.646085 (+ 31us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:41:34.646087 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:41:34.647145 (+ 1058us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:41:34.647146 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:41:34.647993 (+ 847us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:41:34.647995 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:41:34.648881 (+ 886us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:41:34.648882 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:41:34.648942 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:41:34.648945 (+ 3us) client_negotiation.cc:769] Sending connection context
0810 04:41:34.649002 (+ 57us) client_negotiation.cc:240] Negotiation successful
0810 04:41:34.649040 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":28}
I20260810 04:41:34.649292 3359 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:41:34.645731 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34598 (local address 127.22.167.65:43639)
0810 04:41:34.645886 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0810 04:41:34.645892 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 04:41:34.645902 (+ 10us) server_negotiation.cc:407] Connection header received
0810 04:41:34.645940 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:41:34.645942 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:41:34.645958 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:41:34.646052 (+ 94us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 04:41:34.646213 (+ 161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:41:34.647052 (+ 839us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:41:34.648101 (+ 1049us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:41:34.648809 (+ 708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:41:34.648879 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:41:34.648950 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0810 04:41:34.649026 (+ 76us) server_negotiation.cc:299] Negotiation successful
0810 04:41:34.649064 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":125,"thread_start_us":94,"threads_started":1}
I20260810 04:41:34.649868 3360 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:41:34.645951 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.66:46237 (local address 127.0.0.1:44336)
0810 04:41:34.646382 (+ 431us) negotiation.cc:107] Waiting for socket to connect
0810 04:41:34.646392 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0810 04:41:34.646432 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 04:41:34.646741 (+ 309us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 04:41:34.646742 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 04:41:34.646744 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 04:41:34.646782 (+ 38us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:41:34.646783 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:41:34.647911 (+ 1128us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:41:34.647912 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:41:34.648641 (+ 729us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:41:34.648643 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:41:34.649646 (+ 1003us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:41:34.649647 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:41:34.649715 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:41:34.649717 (+ 2us) client_negotiation.cc:769] Sending connection context
0810 04:41:34.649764 (+ 47us) client_negotiation.cc:240] Negotiation successful
0810 04:41:34.649801 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":416,"mutex_wait_us":15,"thread_start_us":75,"threads_started":1}
I20260810 04:41:34.649897 3365 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:41:34.646191 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44336 (local address 127.22.167.66:46237)
0810 04:41:34.646536 (+ 345us) server_negotiation.cc:206] Beginning negotiation
0810 04:41:34.646540 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 04:41:34.646549 (+ 9us) server_negotiation.cc:407] Connection header received
0810 04:41:34.646582 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:41:34.646584 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:41:34.646599 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:41:34.646679 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 04:41:34.646855 (+ 176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:41:34.647846 (+ 991us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:41:34.648718 (+ 872us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:41:34.649478 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:41:34.649541 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:41:34.649614 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0810 04:41:34.649791 (+ 177us) server_negotiation.cc:299] Negotiation successful
0810 04:41:34.649826 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":312,"thread_start_us":77,"threads_started":1}
I20260810 04:41:34.649925 2765 tablet_service.cc:1467] Processing CreateTablet for tablet 5153ad6a0c0f4d24b8ec35dd69b661df (DEFAULT_TABLE table=test-table [id=2411b72687b44b08b3dab8c924c24f5e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 04:41:34.650236 3364 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:41:34.645765 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.67:41131 (local address 127.0.0.1:36818)
0810 04:41:34.646209 (+ 444us) negotiation.cc:107] Waiting for socket to connect
0810 04:41:34.646219 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0810 04:41:34.646275 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 04:41:34.646440 (+ 165us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 04:41:34.646441 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 04:41:34.646443 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 04:41:34.646488 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:41:34.646490 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:41:34.647776 (+ 1286us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:41:34.647777 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:41:34.648872 (+ 1095us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:41:34.648875 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:41:34.649999 (+ 1124us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:41:34.650000 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:41:34.650064 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:41:34.650067 (+ 3us) client_negotiation.cc:769] Sending connection context
0810 04:41:34.650111 (+ 44us) client_negotiation.cc:240] Negotiation successful
0810 04:41:34.650172 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":428,"mutex_wait_us":22,"thread_start_us":88,"threads_started":1}
I20260810 04:41:34.650339 2765 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5153ad6a0c0f4d24b8ec35dd69b661df. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 04:41:34.650442 3361 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:41:34.645837 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36818 (local address 127.22.167.67:41131)
0810 04:41:34.645969 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0810 04:41:34.645972 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 04:41:34.646273 (+ 301us) server_negotiation.cc:407] Connection header received
0810 04:41:34.646342 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:41:34.646344 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:41:34.646360 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:41:34.646438 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 04:41:34.646581 (+ 143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:41:34.647681 (+ 1100us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:41:34.648963 (+ 1282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:41:34.649828 (+ 865us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:41:34.649872 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:41:34.649936 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0810 04:41:34.650337 (+ 401us) server_negotiation.cc:299] Negotiation successful
0810 04:41:34.650375 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":104,"thread_start_us":71,"threads_started":1}
I20260810 04:41:34.650640 2976 tablet_service.cc:1467] Processing CreateTablet for tablet 5153ad6a0c0f4d24b8ec35dd69b661df (DEFAULT_TABLE table=test-table [id=2411b72687b44b08b3dab8c924c24f5e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 04:41:34.650957 2976 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5153ad6a0c0f4d24b8ec35dd69b661df. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 04:41:34.653070 3366 tablet_bootstrap.cc:492] T 5153ad6a0c0f4d24b8ec35dd69b661df P 5e88101ae4ec49b2a0be733b6f18ea08: Bootstrap starting.
I20260810 04:41:34.653615 3367 tablet_bootstrap.cc:492] T 5153ad6a0c0f4d24b8ec35dd69b661df P 7670dc9cfd0d4c25a8294f9115c4235b: Bootstrap starting.
I20260810 04:41:34.653748 3366 tablet_bootstrap.cc:654] T 5153ad6a0c0f4d24b8ec35dd69b661df P 5e88101ae4ec49b2a0be733b6f18ea08: Neither blocks nor log segments found. Creating new log.
I20260810 04:41:34.654060 3366 log.cc:824] T 5153ad6a0c0f4d24b8ec35dd69b661df P 5e88101ae4ec49b2a0be733b6f18ea08: Log is configured to *not* fsync() on all Append() calls
I20260810 04:41:34.654229 3367 tablet_bootstrap.cc:654] T 5153ad6a0c0f4d24b8ec35dd69b661df P 7670dc9cfd0d4c25a8294f9115c4235b: Neither blocks nor log segments found. Creating new log.
I20260810 04:41:34.654307 3198 tablet_service.cc:1467] Processing CreateTablet for tablet 5153ad6a0c0f4d24b8ec35dd69b661df (DEFAULT_TABLE table=test-table [id=2411b72687b44b08b3dab8c924c24f5e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 04:41:34.654518 3367 log.cc:824] T 5153ad6a0c0f4d24b8ec35dd69b661df P 7670dc9cfd0d4c25a8294f9115c4235b: Log is configured to *not* fsync() on all Append() calls
I20260810 04:41:34.654572 3198 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5153ad6a0c0f4d24b8ec35dd69b661df. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 04:41:34.655064 3366 tablet_bootstrap.cc:492] T 5153ad6a0c0f4d24b8ec35dd69b661df P 5e88101ae4ec49b2a0be733b6f18ea08: No bootstrap required, opened a new log
I20260810 04:41:34.655138 3366 ts_tablet_manager.cc:1397] T 5153ad6a0c0f4d24b8ec35dd69b661df P 5e88101ae4ec49b2a0be733b6f18ea08: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260810 04:41:34.655196 3367 tablet_bootstrap.cc:492] T 5153ad6a0c0f4d24b8ec35dd69b661df P 7670dc9cfd0d4c25a8294f9115c4235b: No bootstrap required, opened a new log
I20260810 04:41:34.655260 3367 ts_tablet_manager.cc:1397] T 5153ad6a0c0f4d24b8ec35dd69b661df P 7670dc9cfd0d4c25a8294f9115c4235b: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260810 04:41:34.656667 3367 raft_consensus.cc:348] T 5153ad6a0c0f4d24b8ec35dd69b661df P 7670dc9cfd0d4c25a8294f9115c4235b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e88101ae4ec49b2a0be733b6f18ea08" member_type: VOTER last_known_addr { host: "127.22.167.65" port: 43639 } } peers { permanent_uuid: "1f3cb452053d447290369216f3e9ed93" member_type: VOTER last_known_addr { host: "127.22.167.67" port: 41131 } } peers { permanent_uuid: "7670dc9cfd0d4c25a8294f9115c4235b" member_type: VOTER last_known_addr { host: "127.22.167.66" port: 46237 } }
I20260810 04:41:34.656818 3367 raft_consensus.cc:374] T 5153ad6a0c0f4d24b8ec35dd69b661df P 7670dc9cfd0d4c25a8294f9115c4235b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 04:41:34.656855 3367 raft_consensus.cc:729] T 5153ad6a0c0f4d24b8ec35dd69b661df P 7670dc9cfd0d4c25a8294f9115c4235b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7670dc9cfd0d4c25a8294f9115c4235b, State: Initialized, Role: FOLLOWER
I20260810 04:41:34.656982 3367 consensus_queue.cc:260] T 5153ad6a0c0f4d24b8ec35dd69b661df P 7670dc9cfd0d4c25a8294f9115c4235b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e88101ae4ec49b2a0be733b6f18ea08" member_type: VOTER last_known_addr { host: "127.22.167.65" port: 43639 } } peers { permanent_uuid: "1f3cb452053d447290369216f3e9ed93" member_type: VOTER last_known_addr { host: "127.22.167.67" port: 41131 } } peers { permanent_uuid: "7670dc9cfd0d4c25a8294f9115c4235b" member_type: VOTER last_known_addr { host: "127.22.167.66" port: 46237 } }
I20260810 04:41:34.657034 3366 raft_consensus.cc:348] T 5153ad6a0c0f4d24b8ec35dd69b661df P 5e88101ae4ec49b2a0be733b6f18ea08 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e88101ae4ec49b2a0be733b6f18ea08" member_type: VOTER last_known_addr { host: "127.22.167.65" port: 43639 } } peers { permanent_uuid: "1f3cb452053d447290369216f3e9ed93" member_type: VOTER last_known_addr { host: "127.22.167.67" port: 41131 } } peers { permanent_uuid: "7670dc9cfd0d4c25a8294f9115c4235b" member_type: VOTER last_known_addr { host: "127.22.167.66" port: 46237 } }
I20260810 04:41:34.657207 3366 raft_consensus.cc:374] T 5153ad6a0c0f4d24b8ec35dd69b661df P 5e88101ae4ec49b2a0be733b6f18ea08 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 04:41:34.657217 3367 ts_tablet_manager.cc:1428] T 5153ad6a0c0f4d24b8ec35dd69b661df P 7670dc9cfd0d4c25a8294f9115c4235b: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260810 04:41:34.657254 3366 raft_consensus.cc:729] T 5153ad6a0c0f4d24b8ec35dd69b661df P 5e88101ae4ec49b2a0be733b6f18ea08 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5e88101ae4ec49b2a0be733b6f18ea08, State: Initialized, Role: FOLLOWER
I20260810 04:41:34.657322 3081 heartbeater.cc:503] Master 127.22.167.126:41961 was elected leader, sending a full tablet report...
I20260810 04:41:34.657392 3366 consensus_queue.cc:260] T 5153ad6a0c0f4d24b8ec35dd69b661df P 5e88101ae4ec49b2a0be733b6f18ea08 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e88101ae4ec49b2a0be733b6f18ea08" member_type: VOTER last_known_addr { host: "127.22.167.65" port: 43639 } } peers { permanent_uuid: "1f3cb452053d447290369216f3e9ed93" member_type: VOTER last_known_addr { host: "127.22.167.67" port: 41131 } } peers { permanent_uuid: "7670dc9cfd0d4c25a8294f9115c4235b" member_type: VOTER last_known_addr { host: "127.22.167.66" port: 46237 } }
I20260810 04:41:34.657635 3366 ts_tablet_manager.cc:1428] T 5153ad6a0c0f4d24b8ec35dd69b661df P 5e88101ae4ec49b2a0be733b6f18ea08: Time spent starting tablet: real 0.002s user 0.000s sys 0.003s
I20260810 04:41:34.657934 2861 heartbeater.cc:503] Master 127.22.167.126:41961 was elected leader, sending a full tablet report...
I20260810 04:41:34.657956 3375 tablet_bootstrap.cc:492] T 5153ad6a0c0f4d24b8ec35dd69b661df P 1f3cb452053d447290369216f3e9ed93: Bootstrap starting.
I20260810 04:41:34.659128 3375 tablet_bootstrap.cc:654] T 5153ad6a0c0f4d24b8ec35dd69b661df P 1f3cb452053d447290369216f3e9ed93: Neither blocks nor log segments found. Creating new log.
I20260810 04:41:34.659489 3375 log.cc:824] T 5153ad6a0c0f4d24b8ec35dd69b661df P 1f3cb452053d447290369216f3e9ed93: Log is configured to *not* fsync() on all Append() calls
I20260810 04:41:34.660166 3374 raft_consensus.cc:482] T 5153ad6a0c0f4d24b8ec35dd69b661df P 5e88101ae4ec49b2a0be733b6f18ea08 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 04:41:34.660190 3375 tablet_bootstrap.cc:492] T 5153ad6a0c0f4d24b8ec35dd69b661df P 1f3cb452053d447290369216f3e9ed93: No bootstrap required, opened a new log
I20260810 04:41:34.660241 3375 ts_tablet_manager.cc:1397] T 5153ad6a0c0f4d24b8ec35dd69b661df P 1f3cb452053d447290369216f3e9ed93: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260810 04:41:34.660234 3374 raft_consensus.cc:504] T 5153ad6a0c0f4d24b8ec35dd69b661df P 5e88101ae4ec49b2a0be733b6f18ea08 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e88101ae4ec49b2a0be733b6f18ea08" member_type: VOTER last_known_addr { host: "127.22.167.65" port: 43639 } } peers { permanent_uuid: "1f3cb452053d447290369216f3e9ed93" member_type: VOTER last_known_addr { host: "127.22.167.67" port: 41131 } } peers { permanent_uuid: "7670dc9cfd0d4c25a8294f9115c4235b" member_type: VOTER last_known_addr { host: "127.22.167.66" port: 46237 } }
I20260810 04:41:34.660835 3374 leader_election.cc:288] T 5153ad6a0c0f4d24b8ec35dd69b661df P 5e88101ae4ec49b2a0be733b6f18ea08 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1f3cb452053d447290369216f3e9ed93 (127.22.167.67:41131), 7670dc9cfd0d4c25a8294f9115c4235b (127.22.167.66:46237)
I20260810 04:41:34.662369 3375 raft_consensus.cc:348] T 5153ad6a0c0f4d24b8ec35dd69b661df P 1f3cb452053d447290369216f3e9ed93 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e88101ae4ec49b2a0be733b6f18ea08" member_type: VOTER last_known_addr { host: "127.22.167.65" port: 43639 } } peers { permanent_uuid: "1f3cb452053d447290369216f3e9ed93" member_type: VOTER last_known_addr { host: "127.22.167.67" port: 41131 } } peers { permanent_uuid: "7670dc9cfd0d4c25a8294f9115c4235b" member_type: VOTER last_known_addr { host: "127.22.167.66" port: 46237 } }
I20260810 04:41:34.662482 3375 raft_consensus.cc:374] T 5153ad6a0c0f4d24b8ec35dd69b661df P 1f3cb452053d447290369216f3e9ed93 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 04:41:34.662499 3375 raft_consensus.cc:729] T 5153ad6a0c0f4d24b8ec35dd69b661df P 1f3cb452053d447290369216f3e9ed93 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1f3cb452053d447290369216f3e9ed93, State: Initialized, Role: FOLLOWER
I20260810 04:41:34.662585 3375 consensus_queue.cc:260] T 5153ad6a0c0f4d24b8ec35dd69b661df P 1f3cb452053d447290369216f3e9ed93 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e88101ae4ec49b2a0be733b6f18ea08" member_type: VOTER last_known_addr { host: "127.22.167.65" port: 43639 } } peers { permanent_uuid: "1f3cb452053d447290369216f3e9ed93" member_type: VOTER last_known_addr { host: "127.22.167.67" port: 41131 } } peers { permanent_uuid: "7670dc9cfd0d4c25a8294f9115c4235b" member_type: VOTER last_known_addr { host: "127.22.167.66" port: 46237 } }
I20260810 04:41:34.662803 3375 ts_tablet_manager.cc:1428] T 5153ad6a0c0f4d24b8ec35dd69b661df P 1f3cb452053d447290369216f3e9ed93: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260810 04:41:34.662873 3314 heartbeater.cc:503] Master 127.22.167.126:41961 was elected leader, sending a full tablet report...
I20260810 04:41:34.663856 3365 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:41:34.661053 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.167.65:41931 (local address 127.22.167.66:46237)
0810 04:41:34.661093 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0810 04:41:34.661095 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 04:41:34.661104 (+ 9us) server_negotiation.cc:407] Connection header received
0810 04:41:34.661144 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:41:34.661145 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:41:34.661151 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:41:34.661208 (+ 57us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 04:41:34.661336 (+ 128us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:41:34.661765 (+ 429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:41:34.662971 (+ 1206us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:41:34.663500 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:41:34.663557 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:41:34.663564 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0810 04:41:34.663768 (+ 204us) server_negotiation.cc:299] Negotiation successful
0810 04:41:34.663801 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260810 04:41:34.663882 2867 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:41:34.660964 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.66:46237 (local address 127.22.167.65:41931)
0810 04:41:34.661059 (+ 95us) negotiation.cc:107] Waiting for socket to connect
0810 04:41:34.661068 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0810 04:41:34.661097 (+ 29us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 04:41:34.661222 (+ 125us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 04:41:34.661223 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 04:41:34.661225 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 04:41:34.661282 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:41:34.661283 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:41:34.661829 (+ 546us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:41:34.661830 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:41:34.662877 (+ 1047us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:41:34.662880 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:41:34.663607 (+ 727us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:41:34.663608 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:41:34.663693 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:41:34.663697 (+ 4us) client_negotiation.cc:769] Sending connection context
0810 04:41:34.663763 (+ 66us) client_negotiation.cc:240] Negotiation successful
0810 04:41:34.663799 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":83,"mutex_wait_us":11}
I20260810 04:41:34.664088 3361 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:41:34.660993 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.167.65:32775 (local address 127.22.167.67:41131)
0810 04:41:34.661165 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0810 04:41:34.661167 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 04:41:34.661172 (+ 5us) server_negotiation.cc:407] Connection header received
0810 04:41:34.661186 (+ 14us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:41:34.661187 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:41:34.661193 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:41:34.661232 (+ 39us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 04:41:34.661390 (+ 158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:41:34.662034 (+ 644us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:41:34.663300 (+ 1266us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:41:34.663786 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:41:34.663847 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:41:34.663854 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0810 04:41:34.664004 (+ 150us) server_negotiation.cc:299] Negotiation successful
0810 04:41:34.664038 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":153}
I20260810 04:41:34.664106 2871 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:41:34.660956 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.67:41131 (local address 127.22.167.65:32775)
0810 04:41:34.661001 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0810 04:41:34.661019 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0810 04:41:34.661055 (+ 36us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 04:41:34.661243 (+ 188us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 04:41:34.661244 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 04:41:34.661245 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 04:41:34.661315 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:41:34.661317 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:41:34.662116 (+ 799us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:41:34.662117 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:41:34.663198 (+ 1081us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:41:34.663200 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:41:34.663856 (+ 656us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:41:34.663857 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:41:34.663940 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:41:34.663944 (+ 4us) client_negotiation.cc:769] Sending connection context
0810 04:41:34.663998 (+ 54us) client_negotiation.cc:240] Negotiation successful
0810 04:41:34.664033 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":31}
I20260810 04:41:34.664641 3007 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5153ad6a0c0f4d24b8ec35dd69b661df" candidate_uuid: "5e88101ae4ec49b2a0be733b6f18ea08" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7670dc9cfd0d4c25a8294f9115c4235b" is_pre_election: true
I20260810 04:41:34.664745 3007 raft_consensus.cc:2454] T 5153ad6a0c0f4d24b8ec35dd69b661df P 7670dc9cfd0d4c25a8294f9115c4235b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e88101ae4ec49b2a0be733b6f18ea08 in term 0.
I20260810 04:41:34.664943 2695 leader_election.cc:302] T 5153ad6a0c0f4d24b8ec35dd69b661df P 5e88101ae4ec49b2a0be733b6f18ea08 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e88101ae4ec49b2a0be733b6f18ea08, 7670dc9cfd0d4c25a8294f9115c4235b; no voters:
I20260810 04:41:34.665055 3374 raft_consensus.cc:2781] T 5153ad6a0c0f4d24b8ec35dd69b661df P 5e88101ae4ec49b2a0be733b6f18ea08 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 04:41:34.665107 3374 raft_consensus.cc:482] T 5153ad6a0c0f4d24b8ec35dd69b661df P 5e88101ae4ec49b2a0be733b6f18ea08 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 04:41:34.665127 3374 raft_consensus.cc:3037] T 5153ad6a0c0f4d24b8ec35dd69b661df P 5e88101ae4ec49b2a0be733b6f18ea08 [term 0 FOLLOWER]: Advancing to term 1
I20260810 04:41:34.665164 3227 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5153ad6a0c0f4d24b8ec35dd69b661df" candidate_uuid: "5e88101ae4ec49b2a0be733b6f18ea08" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f3cb452053d447290369216f3e9ed93" is_pre_election: true
I20260810 04:41:34.665290 3227 raft_consensus.cc:2454] T 5153ad6a0c0f4d24b8ec35dd69b661df P 1f3cb452053d447290369216f3e9ed93 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e88101ae4ec49b2a0be733b6f18ea08 in term 0.
I20260810 04:41:34.665870 3374 raft_consensus.cc:504] T 5153ad6a0c0f4d24b8ec35dd69b661df P 5e88101ae4ec49b2a0be733b6f18ea08 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e88101ae4ec49b2a0be733b6f18ea08" member_type: VOTER last_known_addr { host: "127.22.167.65" port: 43639 } } peers { permanent_uuid: "1f3cb452053d447290369216f3e9ed93" member_type: VOTER last_known_addr { host: "127.22.167.67" port: 41131 } } peers { permanent_uuid: "7670dc9cfd0d4c25a8294f9115c4235b" member_type: VOTER last_known_addr { host: "127.22.167.66" port: 46237 } }
I20260810 04:41:34.665987 3374 leader_election.cc:288] T 5153ad6a0c0f4d24b8ec35dd69b661df P 5e88101ae4ec49b2a0be733b6f18ea08 [CANDIDATE]: Term 1 election: Requested vote from peers 1f3cb452053d447290369216f3e9ed93 (127.22.167.67:41131), 7670dc9cfd0d4c25a8294f9115c4235b (127.22.167.66:46237)
I20260810 04:41:34.666101 3227 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5153ad6a0c0f4d24b8ec35dd69b661df" candidate_uuid: "5e88101ae4ec49b2a0be733b6f18ea08" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f3cb452053d447290369216f3e9ed93"
I20260810 04:41:34.666198 3227 raft_consensus.cc:3037] T 5153ad6a0c0f4d24b8ec35dd69b661df P 1f3cb452053d447290369216f3e9ed93 [term 0 FOLLOWER]: Advancing to term 1
I20260810 04:41:34.666172 3007 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5153ad6a0c0f4d24b8ec35dd69b661df" candidate_uuid: "5e88101ae4ec49b2a0be733b6f18ea08" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7670dc9cfd0d4c25a8294f9115c4235b"
I20260810 04:41:34.666239 3007 raft_consensus.cc:3037] T 5153ad6a0c0f4d24b8ec35dd69b661df P 7670dc9cfd0d4c25a8294f9115c4235b [term 0 FOLLOWER]: Advancing to term 1
I20260810 04:41:34.666749 3227 raft_consensus.cc:2454] T 5153ad6a0c0f4d24b8ec35dd69b661df P 1f3cb452053d447290369216f3e9ed93 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e88101ae4ec49b2a0be733b6f18ea08 in term 1.
I20260810 04:41:34.666824 3007 raft_consensus.cc:2454] T 5153ad6a0c0f4d24b8ec35dd69b661df P 7670dc9cfd0d4c25a8294f9115c4235b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e88101ae4ec49b2a0be733b6f18ea08 in term 1.
I20260810 04:41:34.666910 2696 leader_election.cc:302] T 5153ad6a0c0f4d24b8ec35dd69b661df P 5e88101ae4ec49b2a0be733b6f18ea08 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f3cb452053d447290369216f3e9ed93, 5e88101ae4ec49b2a0be733b6f18ea08; no voters:
I20260810 04:41:34.666985 3374 raft_consensus.cc:2781] T 5153ad6a0c0f4d24b8ec35dd69b661df P 5e88101ae4ec49b2a0be733b6f18ea08 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 04:41:34.667109 3374 raft_consensus.cc:686] T 5153ad6a0c0f4d24b8ec35dd69b661df P 5e88101ae4ec49b2a0be733b6f18ea08 [term 1 LEADER]: Becoming Leader. State: Replica: 5e88101ae4ec49b2a0be733b6f18ea08, State: Running, Role: LEADER
I20260810 04:41:34.667212 3374 consensus_queue.cc:237] T 5153ad6a0c0f4d24b8ec35dd69b661df P 5e88101ae4ec49b2a0be733b6f18ea08 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e88101ae4ec49b2a0be733b6f18ea08" member_type: VOTER last_known_addr { host: "127.22.167.65" port: 43639 } } peers { permanent_uuid: "1f3cb452053d447290369216f3e9ed93" member_type: VOTER last_known_addr { host: "127.22.167.67" port: 41131 } } peers { permanent_uuid: "7670dc9cfd0d4c25a8294f9115c4235b" member_type: VOTER last_known_addr { host: "127.22.167.66" port: 46237 } }
I20260810 04:41:34.667877 1306 catalog_manager.cc:5495] T 5153ad6a0c0f4d24b8ec35dd69b661df P 5e88101ae4ec49b2a0be733b6f18ea08 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5e88101ae4ec49b2a0be733b6f18ea08 (127.22.167.65). New cstate: current_term: 1 leader_uuid: "5e88101ae4ec49b2a0be733b6f18ea08" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e88101ae4ec49b2a0be733b6f18ea08" member_type: VOTER last_known_addr { host: "127.22.167.65" port: 43639 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1f3cb452053d447290369216f3e9ed93" member_type: VOTER last_known_addr { host: "127.22.167.67" port: 41131 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7670dc9cfd0d4c25a8294f9115c4235b" member_type: VOTER last_known_addr { host: "127.22.167.66" port: 46237 } health_report { overall_health: UNKNOWN } } }
I20260810 04:41:34.675191 3359 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:41:34.673251 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34600 (local address 127.22.167.65:43639)
0810 04:41:34.673286 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0810 04:41:34.673288 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 04:41:34.673332 (+ 44us) server_negotiation.cc:407] Connection header received
0810 04:41:34.673390 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:41:34.673391 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:41:34.673404 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:41:34.673455 (+ 51us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 04:41:34.673537 (+ 82us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:41:34.673955 (+ 418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:41:34.674655 (+ 700us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:41:34.674829 (+ 174us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:41:34.674872 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:41:34.674963 (+ 91us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 04:41:34.675049 (+ 86us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 04:41:34.675098 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0810 04:41:34.675125 (+ 27us) server_negotiation.cc:299] Negotiation successful
0810 04:41:34.675141 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260810 04:41:34.676662 3227 raft_consensus.cc:1261] T 5153ad6a0c0f4d24b8ec35dd69b661df P 1f3cb452053d447290369216f3e9ed93 [term 1 FOLLOWER]: Refusing update from remote peer 5e88101ae4ec49b2a0be733b6f18ea08: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 04:41:34.676677 3007 raft_consensus.cc:1261] T 5153ad6a0c0f4d24b8ec35dd69b661df P 7670dc9cfd0d4c25a8294f9115c4235b [term 1 FOLLOWER]: Refusing update from remote peer 5e88101ae4ec49b2a0be733b6f18ea08: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 04:41:34.676977 3381 consensus_queue.cc:1035] T 5153ad6a0c0f4d24b8ec35dd69b661df P 5e88101ae4ec49b2a0be733b6f18ea08 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7670dc9cfd0d4c25a8294f9115c4235b" member_type: VOTER last_known_addr { host: "127.22.167.66" port: 46237 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 04:41:34.677095 3374 consensus_queue.cc:1035] T 5153ad6a0c0f4d24b8ec35dd69b661df P 5e88101ae4ec49b2a0be733b6f18ea08 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1f3cb452053d447290369216f3e9ed93" member_type: VOTER last_known_addr { host: "127.22.167.67" port: 41131 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 04:41:34.679761 3393 mvcc.cc:205] Tried to move back new op lower bound from 7316835920593297408 to 7316835920557400064. Current Snapshot: MvccSnapshot[applied={T|T < 7316835920593297408}]
I20260810 04:41:34.682458 1303 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48392:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 04:41:34.682554 1303 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48392:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 04:41:34.684062 1303 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b56636c33cce464eba3d4b547a7c704e: Sending DeleteTablet for 3 replicas of tablet 5153ad6a0c0f4d24b8ec35dd69b661df
I20260810 04:41:34.684346 2765 tablet_service.cc:1514] Processing DeleteTablet for tablet 5153ad6a0c0f4d24b8ec35dd69b661df with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 04:41:34 UTC) from {username='oryx', principal='oryx/127.22.167.126@KRBTEST.COM'} at 127.0.0.1:34598
I20260810 04:41:34.684365 2976 tablet_service.cc:1514] Processing DeleteTablet for tablet 5153ad6a0c0f4d24b8ec35dd69b661df with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 04:41:34 UTC) from {username='oryx', principal='oryx/127.22.167.126@KRBTEST.COM'} at 127.0.0.1:44336
I20260810 04:41:34.684669 3405 tablet_replica.cc:330] T 5153ad6a0c0f4d24b8ec35dd69b661df P 5e88101ae4ec49b2a0be733b6f18ea08: stopping tablet replica
I20260810 04:41:34.684705 3198 tablet_service.cc:1514] Processing DeleteTablet for tablet 5153ad6a0c0f4d24b8ec35dd69b661df with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 04:41:34 UTC) from {username='oryx', principal='oryx/127.22.167.126@KRBTEST.COM'} at 127.0.0.1:36818
I20260810 04:41:34.684767 3405 raft_consensus.cc:2229] T 5153ad6a0c0f4d24b8ec35dd69b661df P 5e88101ae4ec49b2a0be733b6f18ea08 [term 1 LEADER]: Raft consensus shutting down.
I20260810 04:41:34.684868 3405 raft_consensus.cc:2258] T 5153ad6a0c0f4d24b8ec35dd69b661df P 5e88101ae4ec49b2a0be733b6f18ea08 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 04:41:34.685189 3407 tablet_replica.cc:330] T 5153ad6a0c0f4d24b8ec35dd69b661df P 7670dc9cfd0d4c25a8294f9115c4235b: stopping tablet replica
I20260810 04:41:34.685302 3407 raft_consensus.cc:2229] T 5153ad6a0c0f4d24b8ec35dd69b661df P 7670dc9cfd0d4c25a8294f9115c4235b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 04:41:34.685397 3407 raft_consensus.cc:2258] T 5153ad6a0c0f4d24b8ec35dd69b661df P 7670dc9cfd0d4c25a8294f9115c4235b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 04:41:34.685544 3412 tablet_replica.cc:330] T 5153ad6a0c0f4d24b8ec35dd69b661df P 1f3cb452053d447290369216f3e9ed93: stopping tablet replica
I20260810 04:41:34.685676 23197 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskemv_AM/build/release/bin/kudu with pid 1252
I20260810 04:41:34.685801 3405 ts_tablet_manager.cc:1905] T 5153ad6a0c0f4d24b8ec35dd69b661df P 5e88101ae4ec49b2a0be733b6f18ea08: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 04:41:34.698035 3405 ts_tablet_manager.cc:1918] T 5153ad6a0c0f4d24b8ec35dd69b661df P 5e88101ae4ec49b2a0be733b6f18ea08: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 04:41:34.698153 3405 log.cc:1196] T 5153ad6a0c0f4d24b8ec35dd69b661df P 5e88101ae4ec49b2a0be733b6f18ea08: Deleting WAL directory at /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786336863388938-23197-0/minicluster-data/ts-0/wal/wals/5153ad6a0c0f4d24b8ec35dd69b661df
I20260810 04:41:34.698428 3405 ts_tablet_manager.cc:1939] T 5153ad6a0c0f4d24b8ec35dd69b661df P 5e88101ae4ec49b2a0be733b6f18ea08: Deleting consensus metadata
I20260810 04:41:34.710354 3412 raft_consensus.cc:2229] T 5153ad6a0c0f4d24b8ec35dd69b661df P 1f3cb452053d447290369216f3e9ed93 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 04:41:34.710515 3412 raft_consensus.cc:2258] T 5153ad6a0c0f4d24b8ec35dd69b661df P 1f3cb452053d447290369216f3e9ed93 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 04:41:34.711072 3412 ts_tablet_manager.cc:1905] T 5153ad6a0c0f4d24b8ec35dd69b661df P 1f3cb452053d447290369216f3e9ed93: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 04:41:34.711074 3407 ts_tablet_manager.cc:1905] T 5153ad6a0c0f4d24b8ec35dd69b661df P 7670dc9cfd0d4c25a8294f9115c4235b: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260810 04:41:34.712203 2696 connection.cc:535] server connection from 127.0.0.1:34598 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260810 04:41:34.712992 3407 ts_tablet_manager.cc:1918] T 5153ad6a0c0f4d24b8ec35dd69b661df P 7670dc9cfd0d4c25a8294f9115c4235b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 04:41:34.713150 3407 log.cc:1196] T 5153ad6a0c0f4d24b8ec35dd69b661df P 7670dc9cfd0d4c25a8294f9115c4235b: Deleting WAL directory at /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786336863388938-23197-0/minicluster-data/ts-1/wal/wals/5153ad6a0c0f4d24b8ec35dd69b661df
I20260810 04:41:34.713501 3407 ts_tablet_manager.cc:1939] T 5153ad6a0c0f4d24b8ec35dd69b661df P 7670dc9cfd0d4c25a8294f9115c4235b: Deleting consensus metadata
W20260810 04:41:34.713907 2908 connection.cc:441] server connection from 127.0.0.1:44336 torn down before Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:44336 (request call id 1) could send its response
I20260810 04:41:34.714361 3412 ts_tablet_manager.cc:1918] T 5153ad6a0c0f4d24b8ec35dd69b661df P 1f3cb452053d447290369216f3e9ed93: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 04:41:34.714440 3412 log.cc:1196] T 5153ad6a0c0f4d24b8ec35dd69b661df P 1f3cb452053d447290369216f3e9ed93: Deleting WAL directory at /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786336863388938-23197-0/minicluster-data/ts-2/wal/wals/5153ad6a0c0f4d24b8ec35dd69b661df
I20260810 04:41:34.714725 3412 ts_tablet_manager.cc:1939] T 5153ad6a0c0f4d24b8ec35dd69b661df P 1f3cb452053d447290369216f3e9ed93: Deleting consensus metadata
W20260810 04:41:34.715075 3122 connection.cc:441] server connection from 127.0.0.1:36818 torn down before Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:36818 (request call id 1) could send its response
I20260810 04:41:34.721150 2364 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:41:34.713942 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39354 (local address 127.22.167.125:46451)
0810 04:41:34.713988 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0810 04:41:34.713990 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 04:41:34.714227 (+ 237us) server_negotiation.cc:407] Connection header received
0810 04:41:34.714333 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:41:34.714334 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:41:34.714346 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:41:34.714376 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0810 04:41:34.715133 (+ 757us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:41:34.715885 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:41:34.716942 (+ 1057us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:41:34.717244 (+ 302us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:41:34.717302 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:41:34.718715 (+ 1413us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 04:41:34.718716 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 04:41:34.718717 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 04:41:34.718724 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 04:41:34.720150 (+ 1426us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 04:41:34.720451 (+ 301us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 04:41:34.720452 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 04:41:34.720452 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 04:41:34.720509 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 04:41:34.720677 (+ 168us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 04:41:34.720678 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 04:41:34.720678 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 04:41:34.720794 (+ 116us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 04:41:34.720879 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0810 04:41:34.720977 (+ 98us) server_negotiation.cc:299] Negotiation successful
0810 04:41:34.721052 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260810 04:41:34.721906 3339 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:41:34.714103 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39026 (local address 127.22.167.124:39887)
0810 04:41:34.714196 (+ 93us) server_negotiation.cc:206] Beginning negotiation
0810 04:41:34.714198 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 04:41:34.715216 (+ 1018us) server_negotiation.cc:407] Connection header received
0810 04:41:34.715301 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:41:34.715302 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:41:34.715311 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:41:34.715339 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0810 04:41:34.716028 (+ 689us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:41:34.716499 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:41:34.717293 (+ 794us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:41:34.717605 (+ 312us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:41:34.717670 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:41:34.718956 (+ 1286us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 04:41:34.718957 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 04:41:34.718958 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 04:41:34.718964 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0810 04:41:34.720476 (+ 1512us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 04:41:34.720877 (+ 401us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 04:41:34.720878 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 04:41:34.720878 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 04:41:34.720938 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 04:41:34.721170 (+ 232us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 04:41:34.721170 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 04:41:34.721171 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 04:41:34.721298 (+ 127us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 04:41:34.721378 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0810 04:41:34.721778 (+ 400us) server_negotiation.cc:299] Negotiation successful
0810 04:41:34.721845 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260810 04:41:35.680908 3422 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:41:35.680558 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.126:41961 (local address 127.22.167.66:39263)
0810 04:41:35.680713 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0810 04:41:35.680862 (+ 149us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.167.126:41961: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":54,"threads_started":1}
I20260810 04:41:35.681139 3423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:41:35.680888 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.126:41961 (local address 127.22.167.67:58927)
0810 04:41:35.681023 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0810 04:41:35.681091 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.167.126:41961: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":49,"threads_started":1}
I20260810 04:41:35.680615 3421 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:41:35.680327 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.126:41961 (local address 127.22.167.65:44229)
0810 04:41:35.680470 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0810 04:41:35.680556 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.167.126:41961: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":69,"threads_started":1}
W20260810 04:41:35.681360 3314 heartbeater.cc:644] Failed to heartbeat to 127.22.167.126:41961 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.22.167.126:41961: connect: Connection refused (error 111)
W20260810 04:41:35.681401 2861 heartbeater.cc:644] Failed to heartbeat to 127.22.167.126:41961 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.22.167.126:41961: connect: Connection refused (error 111)
W20260810 04:41:35.681558 3081 heartbeater.cc:644] Failed to heartbeat to 127.22.167.126:41961 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.22.167.126:41961: connect: Connection refused (error 111)
I20260810 04:41:36.184639 3426 raft_consensus.cc:482] T 00000000000000000000000000000000 P 079feed1fb674f7fae482e7e9985a560 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader b56636c33cce464eba3d4b547a7c704e)
I20260810 04:41:36.184777 3426 raft_consensus.cc:504] T 00000000000000000000000000000000 P 079feed1fb674f7fae482e7e9985a560 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "b56636c33cce464eba3d4b547a7c704e" member_type: VOTER last_known_addr { host: "127.22.167.126" port: 41961 } } peers { permanent_uuid: "b421e727773044528cb946bc1c2b5b83" member_type: VOTER last_known_addr { host: "127.22.167.125" port: 46451 } attrs { promote: false } } peers { permanent_uuid: "079feed1fb674f7fae482e7e9985a560" member_type: VOTER last_known_addr { host: "127.22.167.124" port: 39887 } attrs { promote: false } }
I20260810 04:41:36.184799 3427 raft_consensus.cc:482] T 00000000000000000000000000000000 P b421e727773044528cb946bc1c2b5b83 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader b56636c33cce464eba3d4b547a7c704e)
I20260810 04:41:36.184926 3427 raft_consensus.cc:504] T 00000000000000000000000000000000 P b421e727773044528cb946bc1c2b5b83 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "b56636c33cce464eba3d4b547a7c704e" member_type: VOTER last_known_addr { host: "127.22.167.126" port: 41961 } } peers { permanent_uuid: "b421e727773044528cb946bc1c2b5b83" member_type: VOTER last_known_addr { host: "127.22.167.125" port: 46451 } attrs { promote: false } } peers { permanent_uuid: "079feed1fb674f7fae482e7e9985a560" member_type: VOTER last_known_addr { host: "127.22.167.124" port: 39887 } attrs { promote: false } }
I20260810 04:41:36.185058 3426 leader_election.cc:288] T 00000000000000000000000000000000 P 079feed1fb674f7fae482e7e9985a560 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b56636c33cce464eba3d4b547a7c704e (127.22.167.126:41961), b421e727773044528cb946bc1c2b5b83 (127.22.167.125:46451)
I20260810 04:41:36.185421 3428 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:41:36.185187 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.126:41961 (local address 127.0.0.1:40074)
0810 04:41:36.185318 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0810 04:41:36.185376 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.167.126:41961: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":59,"threads_started":1}
W20260810 04:41:36.185652 2558 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.22.167.126:41961: connect: Connection refused (error 111)
I20260810 04:41:36.185796 3434 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:41:36.185538 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.126:41961 (local address 127.0.0.1:40082)
0810 04:41:36.185712 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0810 04:41:36.185756 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.167.126:41961: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":163,"thread_start_us":58,"threads_started":1}
I20260810 04:41:36.186008 3428 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:41:36.185927 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.126:41961 (local address 127.0.0.1:40090)
0810 04:41:36.185961 (+ 34us) negotiation.cc:107] Waiting for socket to connect
0810 04:41:36.185983 (+ 22us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.167.126:41961: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":21}
W20260810 04:41:36.186081 2558 leader_election.cc:334] T 00000000000000000000000000000000 P 079feed1fb674f7fae482e7e9985a560 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer b56636c33cce464eba3d4b547a7c704e (127.22.167.126:41961): Network error: Client connection negotiation failed: client connection to 127.22.167.126:41961: connect: Connection refused (error 111)
W20260810 04:41:36.186079 2273 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.22.167.126:41961: connect: Connection refused (error 111)
I20260810 04:41:36.186245 3427 leader_election.cc:288] T 00000000000000000000000000000000 P b421e727773044528cb946bc1c2b5b83 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b56636c33cce464eba3d4b547a7c704e (127.22.167.126:41961), 079feed1fb674f7fae482e7e9985a560 (127.22.167.124:39887)
I20260810 04:41:36.186532 3435 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:41:36.186415 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.126:41961 (local address 127.0.0.1:40092)
0810 04:41:36.186463 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0810 04:41:36.186502 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.167.126:41961: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260810 04:41:36.186609 2273 leader_election.cc:334] T 00000000000000000000000000000000 P b421e727773044528cb946bc1c2b5b83 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer b56636c33cce464eba3d4b547a7c704e (127.22.167.126:41961): Network error: Client connection negotiation failed: client connection to 127.22.167.126:41961: connect: Connection refused (error 111)
I20260810 04:41:36.188751 3432 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:41:36.185217 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.125:46451 (local address 127.0.0.1:59080)
0810 04:41:36.185398 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0810 04:41:36.185409 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0810 04:41:36.185478 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 04:41:36.185645 (+ 167us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 04:41:36.185646 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 04:41:36.185648 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 04:41:36.185708 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:41:36.185754 (+ 46us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:41:36.186937 (+ 1183us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:41:36.186938 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:41:36.187791 (+ 853us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:41:36.187794 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:41:36.188501 (+ 707us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:41:36.188501 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:41:36.188571 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:41:36.188575 (+ 4us) client_negotiation.cc:769] Sending connection context
0810 04:41:36.188633 (+ 58us) client_negotiation.cc:240] Negotiation successful
0810 04:41:36.188677 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":166,"thread_start_us":66,"threads_started":1}
I20260810 04:41:36.188766 3442 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:41:36.185921 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38528 (local address 127.22.167.124:39887)
0810 04:41:36.186046 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0810 04:41:36.186048 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 04:41:36.186055 (+ 7us) server_negotiation.cc:407] Connection header received
0810 04:41:36.186075 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:41:36.186075 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:41:36.186085 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:41:36.186204 (+ 119us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 04:41:36.186346 (+ 142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:41:36.186980 (+ 634us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:41:36.187896 (+ 916us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:41:36.188423 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:41:36.188493 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:41:36.188502 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0810 04:41:36.188643 (+ 141us) server_negotiation.cc:299] Negotiation successful
0810 04:41:36.188683 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":101,"thread_start_us":81,"threads_started":1}
I20260810 04:41:36.188798 3433 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:41:36.185300 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59080 (local address 127.22.167.125:46451)
0810 04:41:36.185418 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0810 04:41:36.185420 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 04:41:36.185532 (+ 112us) server_negotiation.cc:407] Connection header received
0810 04:41:36.185562 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:41:36.185563 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:41:36.185574 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:41:36.185631 (+ 57us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 04:41:36.185984 (+ 353us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:41:36.186870 (+ 886us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:41:36.187872 (+ 1002us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:41:36.188423 (+ 551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:41:36.188493 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:41:36.188502 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0810 04:41:36.188643 (+ 141us) server_negotiation.cc:299] Negotiation successful
0810 04:41:36.188692 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":99,"thread_start_us":59,"threads_started":1}
I20260810 04:41:36.188800 3434 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:41:36.185717 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.124:39887 (local address 127.0.0.1:38528)
0810 04:41:36.185865 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0810 04:41:36.185874 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0810 04:41:36.185923 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 04:41:36.186217 (+ 294us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 04:41:36.186218 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 04:41:36.186221 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 04:41:36.186257 (+ 36us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:41:36.186258 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:41:36.187052 (+ 794us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:41:36.187052 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:41:36.187813 (+ 761us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:41:36.187816 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:41:36.188499 (+ 683us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:41:36.188500 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:41:36.188571 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:41:36.188575 (+ 4us) client_negotiation.cc:769] Sending connection context
0810 04:41:36.188638 (+ 63us) client_negotiation.cc:240] Negotiation successful
0810 04:41:36.188677 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":43,"threads_started":1}
I20260810 04:41:36.189045 2308 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "079feed1fb674f7fae482e7e9985a560" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "b421e727773044528cb946bc1c2b5b83" is_pre_election: true
I20260810 04:41:36.189126 2308 raft_consensus.cc:2454] T 00000000000000000000000000000000 P b421e727773044528cb946bc1c2b5b83 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 079feed1fb674f7fae482e7e9985a560 in term 1.
I20260810 04:41:36.189213 2593 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "b421e727773044528cb946bc1c2b5b83" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "079feed1fb674f7fae482e7e9985a560" is_pre_election: true
I20260810 04:41:36.189270 2556 leader_election.cc:302] T 00000000000000000000000000000000 P 079feed1fb674f7fae482e7e9985a560 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 079feed1fb674f7fae482e7e9985a560, b421e727773044528cb946bc1c2b5b83; no voters: b56636c33cce464eba3d4b547a7c704e
I20260810 04:41:36.189313 2593 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 079feed1fb674f7fae482e7e9985a560 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b421e727773044528cb946bc1c2b5b83 in term 1.
I20260810 04:41:36.189353 3426 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 079feed1fb674f7fae482e7e9985a560 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260810 04:41:36.189388 3426 raft_consensus.cc:482] T 00000000000000000000000000000000 P 079feed1fb674f7fae482e7e9985a560 [term 1 FOLLOWER]: Starting leader election (detected failure of leader b56636c33cce464eba3d4b547a7c704e)
I20260810 04:41:36.189405 3426 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 079feed1fb674f7fae482e7e9985a560 [term 1 FOLLOWER]: Advancing to term 2
I20260810 04:41:36.189443 2273 leader_election.cc:302] T 00000000000000000000000000000000 P b421e727773044528cb946bc1c2b5b83 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 079feed1fb674f7fae482e7e9985a560, b421e727773044528cb946bc1c2b5b83; no voters: b56636c33cce464eba3d4b547a7c704e
I20260810 04:41:36.189528 3427 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b421e727773044528cb946bc1c2b5b83 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260810 04:41:36.189558 3427 raft_consensus.cc:482] T 00000000000000000000000000000000 P b421e727773044528cb946bc1c2b5b83 [term 1 FOLLOWER]: Starting leader election (detected failure of leader b56636c33cce464eba3d4b547a7c704e)
I20260810 04:41:36.189575 3427 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b421e727773044528cb946bc1c2b5b83 [term 1 FOLLOWER]: Advancing to term 2
I20260810 04:41:36.190066 3426 raft_consensus.cc:504] T 00000000000000000000000000000000 P 079feed1fb674f7fae482e7e9985a560 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "b56636c33cce464eba3d4b547a7c704e" member_type: VOTER last_known_addr { host: "127.22.167.126" port: 41961 } } peers { permanent_uuid: "b421e727773044528cb946bc1c2b5b83" member_type: VOTER last_known_addr { host: "127.22.167.125" port: 46451 } attrs { promote: false } } peers { permanent_uuid: "079feed1fb674f7fae482e7e9985a560" member_type: VOTER last_known_addr { host: "127.22.167.124" port: 39887 } attrs { promote: false } }
I20260810 04:41:36.190191 3426 leader_election.cc:288] T 00000000000000000000000000000000 P 079feed1fb674f7fae482e7e9985a560 [CANDIDATE]: Term 2 election: Requested vote from peers b56636c33cce464eba3d4b547a7c704e (127.22.167.126:41961), b421e727773044528cb946bc1c2b5b83 (127.22.167.125:46451)
I20260810 04:41:36.190228 3427 raft_consensus.cc:504] T 00000000000000000000000000000000 P b421e727773044528cb946bc1c2b5b83 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "b56636c33cce464eba3d4b547a7c704e" member_type: VOTER last_known_addr { host: "127.22.167.126" port: 41961 } } peers { permanent_uuid: "b421e727773044528cb946bc1c2b5b83" member_type: VOTER last_known_addr { host: "127.22.167.125" port: 46451 } attrs { promote: false } } peers { permanent_uuid: "079feed1fb674f7fae482e7e9985a560" member_type: VOTER last_known_addr { host: "127.22.167.124" port: 39887 } attrs { promote: false } }
I20260810 04:41:36.190313 2308 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "079feed1fb674f7fae482e7e9985a560" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "b421e727773044528cb946bc1c2b5b83"
I20260810 04:41:36.190395 2308 raft_consensus.cc:2379] T 00000000000000000000000000000000 P b421e727773044528cb946bc1c2b5b83 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 079feed1fb674f7fae482e7e9985a560 in current term 2: Already voted for candidate b421e727773044528cb946bc1c2b5b83 in this term.
I20260810 04:41:36.190498 3434 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:41:36.190402 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.126:41961 (local address 127.0.0.1:40110)
0810 04:41:36.190446 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0810 04:41:36.190474 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.167.126:41961: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
I20260810 04:41:36.190524 3432 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:41:36.190272 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.126:41961 (local address 127.0.0.1:40108)
0810 04:41:36.190470 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0810 04:41:36.190500 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.167.126:41961: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":183}
I20260810 04:41:36.190603 2593 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "b421e727773044528cb946bc1c2b5b83" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "079feed1fb674f7fae482e7e9985a560"
I20260810 04:41:36.190685 2593 raft_consensus.cc:2379] T 00000000000000000000000000000000 P 079feed1fb674f7fae482e7e9985a560 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate b421e727773044528cb946bc1c2b5b83 in current term 2: Already voted for candidate 079feed1fb674f7fae482e7e9985a560 in this term.
I20260810 04:41:36.190675 3427 leader_election.cc:288] T 00000000000000000000000000000000 P b421e727773044528cb946bc1c2b5b83 [CANDIDATE]: Term 2 election: Requested vote from peers b56636c33cce464eba3d4b547a7c704e (127.22.167.126:41961), 079feed1fb674f7fae482e7e9985a560 (127.22.167.124:39887)
I20260810 04:41:36.190824 3432 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:41:36.190748 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.126:41961 (local address 127.0.0.1:40126)
0810 04:41:36.190777 (+ 29us) negotiation.cc:107] Waiting for socket to connect
0810 04:41:36.190802 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.167.126:41961: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":15}
I20260810 04:41:36.190860 3434 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:41:36.190721 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.126:41961 (local address 127.0.0.1:40142)
0810 04:41:36.190809 (+ 88us) negotiation.cc:107] Waiting for socket to connect
0810 04:41:36.190836 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.167.126:41961: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":75}
W20260810 04:41:36.190886 2558 leader_election.cc:334] T 00000000000000000000000000000000 P 079feed1fb674f7fae482e7e9985a560 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer b56636c33cce464eba3d4b547a7c704e (127.22.167.126:41961): Network error: Client connection negotiation failed: client connection to 127.22.167.126:41961: connect: Connection refused (error 111)
I20260810 04:41:36.190919 2558 leader_election.cc:302] T 00000000000000000000000000000000 P 079feed1fb674f7fae482e7e9985a560 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 079feed1fb674f7fae482e7e9985a560; no voters: b421e727773044528cb946bc1c2b5b83, b56636c33cce464eba3d4b547a7c704e
W20260810 04:41:36.190927 2273 leader_election.cc:334] T 00000000000000000000000000000000 P b421e727773044528cb946bc1c2b5b83 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer b56636c33cce464eba3d4b547a7c704e (127.22.167.126:41961): Network error: Client connection negotiation failed: client connection to 127.22.167.126:41961: connect: Connection refused (error 111)
I20260810 04:41:36.190963 2273 leader_election.cc:302] T 00000000000000000000000000000000 P b421e727773044528cb946bc1c2b5b83 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b421e727773044528cb946bc1c2b5b83; no voters: 079feed1fb674f7fae482e7e9985a560, b56636c33cce464eba3d4b547a7c704e
I20260810 04:41:36.190977 3426 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 079feed1fb674f7fae482e7e9985a560 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260810 04:41:36.191048 3427 raft_consensus.cc:2726] T 00000000000000000000000000000000 P b421e727773044528cb946bc1c2b5b83 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260810 04:41:36.681938 3445 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:41:36.681697 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.126:41961 (local address 127.22.167.65:46423)
0810 04:41:36.681833 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0810 04:41:36.681881 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.167.126:41961: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":70,"threads_started":1}
I20260810 04:41:36.681967 3444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:41:36.681700 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.126:41961 (local address 127.22.167.67:52295)
0810 04:41:36.681880 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0810 04:41:36.681918 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.167.126:41961: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":163,"thread_start_us":63,"threads_started":1}
I20260810 04:41:36.682228 3446 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:41:36.681964 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.126:41961 (local address 127.22.167.66:52663)
0810 04:41:36.682110 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0810 04:41:36.682186 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.167.126:41961: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":61,"threads_started":1}
I20260810 04:41:37.682757 3447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:41:37.682503 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.126:41961 (local address 127.22.167.65:55021)
0810 04:41:37.682651 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0810 04:41:37.682704 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.167.126:41961: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":71,"threads_started":1}
I20260810 04:41:37.682756 3448 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:41:37.682532 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.126:41961 (local address 127.22.167.66:51887)
0810 04:41:37.682654 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0810 04:41:37.682699 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.167.126:41961: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":99,"thread_start_us":44,"threads_started":1}
I20260810 04:41:37.682891 3450 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:41:37.682589 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.126:41961 (local address 127.22.167.67:56819)
0810 04:41:37.682819 (+ 230us) negotiation.cc:107] Waiting for socket to connect
0810 04:41:37.682851 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.167.126:41961: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":216,"thread_start_us":57,"threads_started":1}
I20260810 04:41:37.836874 3451 raft_consensus.cc:482] T 00000000000000000000000000000000 P b421e727773044528cb946bc1c2b5b83 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 04:41:37.836964 3451 raft_consensus.cc:504] T 00000000000000000000000000000000 P b421e727773044528cb946bc1c2b5b83 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "b56636c33cce464eba3d4b547a7c704e" member_type: VOTER last_known_addr { host: "127.22.167.126" port: 41961 } } peers { permanent_uuid: "b421e727773044528cb946bc1c2b5b83" member_type: VOTER last_known_addr { host: "127.22.167.125" port: 46451 } attrs { promote: false } } peers { permanent_uuid: "079feed1fb674f7fae482e7e9985a560" member_type: VOTER last_known_addr { host: "127.22.167.124" port: 39887 } attrs { promote: false } }
I20260810 04:41:37.837095 3451 leader_election.cc:288] T 00000000000000000000000000000000 P b421e727773044528cb946bc1c2b5b83 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers b56636c33cce464eba3d4b547a7c704e (127.22.167.126:41961), 079feed1fb674f7fae482e7e9985a560 (127.22.167.124:39887)
I20260810 04:41:37.837419 3452 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:41:37.837233 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.126:41961 (local address 127.0.0.1:40360)
0810 04:41:37.837330 (+ 97us) negotiation.cc:107] Waiting for socket to connect
0810 04:41:37.837373 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.167.126:41961: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":79,"thread_start_us":45,"threads_started":1}
I20260810 04:41:37.837486 2593 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "b421e727773044528cb946bc1c2b5b83" candidate_term: 3 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "079feed1fb674f7fae482e7e9985a560" is_pre_election: true
I20260810 04:41:37.837590 2593 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 079feed1fb674f7fae482e7e9985a560 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b421e727773044528cb946bc1c2b5b83 in term 2.
I20260810 04:41:37.837803 2273 leader_election.cc:302] T 00000000000000000000000000000000 P b421e727773044528cb946bc1c2b5b83 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 079feed1fb674f7fae482e7e9985a560, b421e727773044528cb946bc1c2b5b83; no voters:
I20260810 04:41:37.837852 3452 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:41:37.837756 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.126:41961 (local address 127.0.0.1:40374)
0810 04:41:37.837803 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0810 04:41:37.837832 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.167.126:41961: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
I20260810 04:41:37.837899 3451 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b421e727773044528cb946bc1c2b5b83 [term 2 FOLLOWER]: Leader pre-election won for term 3
W20260810 04:41:37.837905 2273 leader_election.cc:334] T 00000000000000000000000000000000 P b421e727773044528cb946bc1c2b5b83 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer b56636c33cce464eba3d4b547a7c704e (127.22.167.126:41961): Network error: Client connection negotiation failed: client connection to 127.22.167.126:41961: connect: Connection refused (error 111)
I20260810 04:41:37.837949 3451 raft_consensus.cc:482] T 00000000000000000000000000000000 P b421e727773044528cb946bc1c2b5b83 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 04:41:37.838009 3451 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b421e727773044528cb946bc1c2b5b83 [term 2 FOLLOWER]: Advancing to term 3
I20260810 04:41:37.838742 3451 raft_consensus.cc:504] T 00000000000000000000000000000000 P b421e727773044528cb946bc1c2b5b83 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "b56636c33cce464eba3d4b547a7c704e" member_type: VOTER last_known_addr { host: "127.22.167.126" port: 41961 } } peers { permanent_uuid: "b421e727773044528cb946bc1c2b5b83" member_type: VOTER last_known_addr { host: "127.22.167.125" port: 46451 } attrs { promote: false } } peers { permanent_uuid: "079feed1fb674f7fae482e7e9985a560" member_type: VOTER last_known_addr { host: "127.22.167.124" port: 39887 } attrs { promote: false } }
I20260810 04:41:37.838858 3451 leader_election.cc:288] T 00000000000000000000000000000000 P b421e727773044528cb946bc1c2b5b83 [CANDIDATE]: Term 3 election: Requested vote from peers b56636c33cce464eba3d4b547a7c704e (127.22.167.126:41961), 079feed1fb674f7fae482e7e9985a560 (127.22.167.124:39887)
I20260810 04:41:37.839040 3452 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:41:37.838952 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.126:41961 (local address 127.0.0.1:40382)
0810 04:41:37.838995 (+ 43us) negotiation.cc:107] Waiting for socket to connect
0810 04:41:37.839022 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.167.126:41961: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
I20260810 04:41:37.839104 2593 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "b421e727773044528cb946bc1c2b5b83" candidate_term: 3 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "079feed1fb674f7fae482e7e9985a560"
I20260810 04:41:37.839177 2593 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 079feed1fb674f7fae482e7e9985a560 [term 2 FOLLOWER]: Advancing to term 3
I20260810 04:41:37.839335 3452 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:41:37.839254 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.126:41961 (local address 127.0.0.1:40398)
0810 04:41:37.839287 (+ 33us) negotiation.cc:107] Waiting for socket to connect
0810 04:41:37.839315 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.167.126:41961: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":17}
W20260810 04:41:37.839406 2273 leader_election.cc:334] T 00000000000000000000000000000000 P b421e727773044528cb946bc1c2b5b83 [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer b56636c33cce464eba3d4b547a7c704e (127.22.167.126:41961): Network error: Client connection negotiation failed: client connection to 127.22.167.126:41961: connect: Connection refused (error 111)
I20260810 04:41:37.839867 2593 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 079feed1fb674f7fae482e7e9985a560 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b421e727773044528cb946bc1c2b5b83 in term 3.
I20260810 04:41:37.840016 2273 leader_election.cc:302] T 00000000000000000000000000000000 P b421e727773044528cb946bc1c2b5b83 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 079feed1fb674f7fae482e7e9985a560, b421e727773044528cb946bc1c2b5b83; no voters: b56636c33cce464eba3d4b547a7c704e
I20260810 04:41:37.840094 3451 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b421e727773044528cb946bc1c2b5b83 [term 3 FOLLOWER]: Leader election won for term 3
I20260810 04:41:37.840205 3451 raft_consensus.cc:686] T 00000000000000000000000000000000 P b421e727773044528cb946bc1c2b5b83 [term 3 LEADER]: Becoming Leader. State: Replica: b421e727773044528cb946bc1c2b5b83, State: Running, Role: LEADER
I20260810 04:41:37.840265 3451 consensus_queue.cc:237] T 00000000000000000000000000000000 P b421e727773044528cb946bc1c2b5b83 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 1.12, Last appended by leader: 12, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "b56636c33cce464eba3d4b547a7c704e" member_type: VOTER last_known_addr { host: "127.22.167.126" port: 41961 } } peers { permanent_uuid: "b421e727773044528cb946bc1c2b5b83" member_type: VOTER last_known_addr { host: "127.22.167.125" port: 46451 } attrs { promote: false } } peers { permanent_uuid: "079feed1fb674f7fae482e7e9985a560" member_type: VOTER last_known_addr { host: "127.22.167.124" port: 39887 } attrs { promote: false } }
I20260810 04:41:37.840619 3455 sys_catalog.cc:455] T 00000000000000000000000000000000 P b421e727773044528cb946bc1c2b5b83 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b421e727773044528cb946bc1c2b5b83. Latest consensus state: current_term: 3 leader_uuid: "b421e727773044528cb946bc1c2b5b83" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "b56636c33cce464eba3d4b547a7c704e" member_type: VOTER last_known_addr { host: "127.22.167.126" port: 41961 } } peers { permanent_uuid: "b421e727773044528cb946bc1c2b5b83" member_type: VOTER last_known_addr { host: "127.22.167.125" port: 46451 } attrs { promote: false } } peers { permanent_uuid: "079feed1fb674f7fae482e7e9985a560" member_type: VOTER last_known_addr { host: "127.22.167.124" port: 39887 } attrs { promote: false } } }
I20260810 04:41:37.840714 3455 sys_catalog.cc:458] T 00000000000000000000000000000000 P b421e727773044528cb946bc1c2b5b83 [sys.catalog]: This master's current role is: LEADER
I20260810 04:41:37.840893 3457 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 04:41:37.841362 3457 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 04:41:37.841439 3457 catalog_manager.cc:1259] Loaded cluster ID: 8f7937fec90543fcb67172c95103c8d3
I20260810 04:41:37.841471 3457 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 04:41:37.841655 3457 catalog_manager.cc:1504] Loading token signing keys...
I20260810 04:41:37.841722 3457 catalog_manager.cc:5879] T 00000000000000000000000000000000 P b421e727773044528cb946bc1c2b5b83: Loaded TSK: 0
I20260810 04:41:37.841862 3457 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 04:41:37.868111 2291 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39354:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 04:41:37.869693 2291 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 04:41:37.870586 3452 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:41:37.870487 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.126:41961 (local address 127.0.0.1:40416)
0810 04:41:37.870521 (+ 34us) negotiation.cc:107] Waiting for socket to connect
0810 04:41:37.870555 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.167.126:41961: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":23}
I20260810 04:41:37.870671 2593 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 079feed1fb674f7fae482e7e9985a560 [term 3 FOLLOWER]: Refusing update from remote peer b421e727773044528cb946bc1c2b5b83: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 3 index: 14. (index mismatch)
I20260810 04:41:37.870898 3452 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:41:37.870804 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.126:41961 (local address 127.0.0.1:40430)
0810 04:41:37.870845 (+ 41us) negotiation.cc:107] Waiting for socket to connect
0810 04:41:37.870873 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.167.126:41961: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
I20260810 04:41:37.870898 3451 consensus_queue.cc:1035] T 00000000000000000000000000000000 P b421e727773044528cb946bc1c2b5b83 [LEADER]: Connected to new peer: Peer: permanent_uuid: "079feed1fb674f7fae482e7e9985a560" member_type: VOTER last_known_addr { host: "127.22.167.124" port: 39887 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
W20260810 04:41:37.871037 2273 consensus_peers.cc:487] T 00000000000000000000000000000000 P b421e727773044528cb946bc1c2b5b83 -> Peer b56636c33cce464eba3d4b547a7c704e (127.22.167.126:41961): Couldn't send request to peer b56636c33cce464eba3d4b547a7c704e. Status: Network error: Client connection negotiation failed: client connection to 127.22.167.126:41961: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260810 04:41:37.871464 3462 sys_catalog.cc:455] T 00000000000000000000000000000000 P 079feed1fb674f7fae482e7e9985a560 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b421e727773044528cb946bc1c2b5b83. Latest consensus state: current_term: 3 leader_uuid: "b421e727773044528cb946bc1c2b5b83" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "b56636c33cce464eba3d4b547a7c704e" member_type: VOTER last_known_addr { host: "127.22.167.126" port: 41961 } } peers { permanent_uuid: "b421e727773044528cb946bc1c2b5b83" member_type: VOTER last_known_addr { host: "127.22.167.125" port: 46451 } attrs { promote: false } } peers { permanent_uuid: "079feed1fb674f7fae482e7e9985a560" member_type: VOTER last_known_addr { host: "127.22.167.124" port: 39887 } attrs { promote: false } } }
I20260810 04:41:37.871562 3462 sys_catalog.cc:458] T 00000000000000000000000000000000 P 079feed1fb674f7fae482e7e9985a560 [sys.catalog]: This master's current role is: FOLLOWER
I20260810 04:41:37.871994 3451 sys_catalog.cc:455] T 00000000000000000000000000000000 P b421e727773044528cb946bc1c2b5b83 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 3 leader_uuid: "b421e727773044528cb946bc1c2b5b83" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "b56636c33cce464eba3d4b547a7c704e" member_type: VOTER last_known_addr { host: "127.22.167.126" port: 41961 } } peers { permanent_uuid: "b421e727773044528cb946bc1c2b5b83" member_type: VOTER last_known_addr { host: "127.22.167.125" port: 46451 } attrs { promote: false } } peers { permanent_uuid: "079feed1fb674f7fae482e7e9985a560" member_type: VOTER last_known_addr { host: "127.22.167.124" port: 39887 } attrs { promote: false } } }
I20260810 04:41:37.872074 3451 sys_catalog.cc:458] T 00000000000000000000000000000000 P b421e727773044528cb946bc1c2b5b83 [sys.catalog]: This master's current role is: LEADER
I20260810 04:41:37.872092 3462 sys_catalog.cc:455] T 00000000000000000000000000000000 P 079feed1fb674f7fae482e7e9985a560 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 3 leader_uuid: "b421e727773044528cb946bc1c2b5b83" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "b56636c33cce464eba3d4b547a7c704e" member_type: VOTER last_known_addr { host: "127.22.167.126" port: 41961 } } peers { permanent_uuid: "b421e727773044528cb946bc1c2b5b83" member_type: VOTER last_known_addr { host: "127.22.167.125" port: 46451 } attrs { promote: false } } peers { permanent_uuid: "079feed1fb674f7fae482e7e9985a560" member_type: VOTER last_known_addr { host: "127.22.167.124" port: 39887 } attrs { promote: false } } }
I20260810 04:41:37.872172 3462 sys_catalog.cc:458] T 00000000000000000000000000000000 P 079feed1fb674f7fae482e7e9985a560 [sys.catalog]: This master's current role is: FOLLOWER
I20260810 04:41:37.872239 3464 mvcc.cc:205] Tried to move back new op lower bound from 7316835933676228608 to 7316835933554221056. Current Snapshot: MvccSnapshot[applied={T|T < 7316835933676228608 or (T in {7316835933676228608})}]
I20260810 04:41:37.876724 3452 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:41:37.873737 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.66:46237 (local address 127.0.0.1:33690)
0810 04:41:37.873805 (+ 68us) negotiation.cc:107] Waiting for socket to connect
0810 04:41:37.873816 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0810 04:41:37.873866 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 04:41:37.874092 (+ 226us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 04:41:37.874093 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 04:41:37.874094 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 04:41:37.874161 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:41:37.874164 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:41:37.874956 (+ 792us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:41:37.874956 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:41:37.875942 (+ 986us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:41:37.875944 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:41:37.876519 (+ 575us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:41:37.876520 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:41:37.876579 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:41:37.876583 (+ 4us) client_negotiation.cc:769] Sending connection context
0810 04:41:37.876627 (+ 44us) client_negotiation.cc:240] Negotiation successful
0810 04:41:37.876664 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":52}
I20260810 04:41:37.876984 3472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:41:37.873815 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33690 (local address 127.22.167.66:46237)
0810 04:41:37.873947 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0810 04:41:37.873949 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 04:41:37.873956 (+ 7us) server_negotiation.cc:407] Connection header received
0810 04:41:37.873982 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:41:37.873982 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:41:37.873992 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:41:37.874085 (+ 93us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 04:41:37.874217 (+ 132us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:41:37.874889 (+ 672us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:41:37.876031 (+ 1142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:41:37.876443 (+ 412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:41:37.876521 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:41:37.876532 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0810 04:41:37.876867 (+ 335us) server_negotiation.cc:299] Negotiation successful
0810 04:41:37.876915 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":114,"thread_start_us":79,"threads_started":1}
I20260810 04:41:37.877298 2976 tablet_service.cc:1467] Processing CreateTablet for tablet 8028ce9b119b45eb9d05f7d037224788 (DEFAULT_TABLE table=test-table [id=0924650b58aa4cafae9b4ca313a864ea]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 04:41:37.877326 3476 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:41:37.874033 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49790 (local address 127.22.167.65:43639)
0810 04:41:37.874163 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0810 04:41:37.874165 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 04:41:37.874172 (+ 7us) server_negotiation.cc:407] Connection header received
0810 04:41:37.874197 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:41:37.874198 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:41:37.874209 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:41:37.874277 (+ 68us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 04:41:37.874428 (+ 151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:41:37.875227 (+ 799us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:41:37.876143 (+ 916us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:41:37.876931 (+ 788us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:41:37.876994 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:41:37.877006 (+ 12us) server_negotiation.cc:1035] Waiting for connection context
0810 04:41:37.877216 (+ 210us) server_negotiation.cc:299] Negotiation successful
0810 04:41:37.877258 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":113,"thread_start_us":60,"threads_started":1}
I20260810 04:41:37.877331 3470 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:41:37.873799 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.65:43639 (local address 127.0.0.1:49790)
0810 04:41:37.874044 (+ 245us) negotiation.cc:107] Waiting for socket to connect
0810 04:41:37.874052 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0810 04:41:37.874085 (+ 33us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 04:41:37.874326 (+ 241us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 04:41:37.874327 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 04:41:37.874328 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 04:41:37.874374 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:41:37.874376 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:41:37.875308 (+ 932us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:41:37.875309 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:41:37.876059 (+ 750us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:41:37.876062 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:41:37.876999 (+ 937us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:41:37.877001 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:41:37.877146 (+ 145us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:41:37.877151 (+ 5us) client_negotiation.cc:769] Sending connection context
0810 04:41:37.877206 (+ 55us) client_negotiation.cc:240] Negotiation successful
0810 04:41:37.877249 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":230,"mutex_wait_us":19,"thread_start_us":87,"threads_started":1}
I20260810 04:41:37.877345 3471 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:41:37.873799 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.67:41131 (local address 127.0.0.1:46174)
0810 04:41:37.873920 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0810 04:41:37.873930 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0810 04:41:37.873977 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 04:41:37.874346 (+ 369us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 04:41:37.874347 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 04:41:37.874349 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 04:41:37.874419 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:41:37.874420 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:41:37.875289 (+ 869us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:41:37.875290 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:41:37.876390 (+ 1100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:41:37.876392 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:41:37.877090 (+ 698us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:41:37.877092 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:41:37.877165 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:41:37.877169 (+ 4us) client_negotiation.cc:769] Sending connection context
0810 04:41:37.877219 (+ 50us) client_negotiation.cc:240] Negotiation successful
0810 04:41:37.877257 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":103,"mutex_wait_us":26,"thread_start_us":69,"threads_started":1}
I20260810 04:41:37.877444 2976 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8028ce9b119b45eb9d05f7d037224788. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 04:41:37.877550 2765 tablet_service.cc:1467] Processing CreateTablet for tablet 8028ce9b119b45eb9d05f7d037224788 (DEFAULT_TABLE table=test-table [id=0924650b58aa4cafae9b4ca313a864ea]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 04:41:37.877588 3473 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:41:37.873994 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46174 (local address 127.22.167.67:41131)
0810 04:41:37.874218 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0810 04:41:37.874220 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 04:41:37.874228 (+ 8us) server_negotiation.cc:407] Connection header received
0810 04:41:37.874254 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:41:37.874255 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:41:37.874264 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:41:37.874334 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 04:41:37.874478 (+ 144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:41:37.875209 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:41:37.876473 (+ 1264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:41:37.876982 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:41:37.877078 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:41:37.877091 (+ 13us) server_negotiation.cc:1035] Waiting for connection context
0810 04:41:37.877475 (+ 384us) server_negotiation.cc:299] Negotiation successful
0810 04:41:37.877524 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":204,"thread_start_us":58,"threads_started":1}
I20260810 04:41:37.877650 2765 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8028ce9b119b45eb9d05f7d037224788. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 04:41:37.877774 3198 tablet_service.cc:1467] Processing CreateTablet for tablet 8028ce9b119b45eb9d05f7d037224788 (DEFAULT_TABLE table=test-table [id=0924650b58aa4cafae9b4ca313a864ea]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 04:41:37.877866 3198 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8028ce9b119b45eb9d05f7d037224788. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 04:41:37.879069 3480 tablet_bootstrap.cc:492] T 8028ce9b119b45eb9d05f7d037224788 P 5e88101ae4ec49b2a0be733b6f18ea08: Bootstrap starting.
I20260810 04:41:37.879395 3480 tablet_bootstrap.cc:654] T 8028ce9b119b45eb9d05f7d037224788 P 5e88101ae4ec49b2a0be733b6f18ea08: Neither blocks nor log segments found. Creating new log.
I20260810 04:41:37.879438 3477 tablet_bootstrap.cc:492] T 8028ce9b119b45eb9d05f7d037224788 P 7670dc9cfd0d4c25a8294f9115c4235b: Bootstrap starting.
I20260810 04:41:37.879765 3477 tablet_bootstrap.cc:654] T 8028ce9b119b45eb9d05f7d037224788 P 7670dc9cfd0d4c25a8294f9115c4235b: Neither blocks nor log segments found. Creating new log.
I20260810 04:41:37.879995 3480 tablet_bootstrap.cc:492] T 8028ce9b119b45eb9d05f7d037224788 P 5e88101ae4ec49b2a0be733b6f18ea08: No bootstrap required, opened a new log
I20260810 04:41:37.880038 3480 ts_tablet_manager.cc:1397] T 8028ce9b119b45eb9d05f7d037224788 P 5e88101ae4ec49b2a0be733b6f18ea08: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260810 04:41:37.880097 3479 tablet_bootstrap.cc:492] T 8028ce9b119b45eb9d05f7d037224788 P 1f3cb452053d447290369216f3e9ed93: Bootstrap starting.
I20260810 04:41:37.880127 3480 raft_consensus.cc:348] T 8028ce9b119b45eb9d05f7d037224788 P 5e88101ae4ec49b2a0be733b6f18ea08 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7670dc9cfd0d4c25a8294f9115c4235b" member_type: VOTER last_known_addr { host: "127.22.167.66" port: 46237 } } peers { permanent_uuid: "5e88101ae4ec49b2a0be733b6f18ea08" member_type: VOTER last_known_addr { host: "127.22.167.65" port: 43639 } } peers { permanent_uuid: "1f3cb452053d447290369216f3e9ed93" member_type: VOTER last_known_addr { host: "127.22.167.67" port: 41131 } }
I20260810 04:41:37.880173 3480 raft_consensus.cc:374] T 8028ce9b119b45eb9d05f7d037224788 P 5e88101ae4ec49b2a0be733b6f18ea08 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 04:41:37.880219 3480 raft_consensus.cc:729] T 8028ce9b119b45eb9d05f7d037224788 P 5e88101ae4ec49b2a0be733b6f18ea08 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5e88101ae4ec49b2a0be733b6f18ea08, State: Initialized, Role: FOLLOWER
I20260810 04:41:37.880277 3480 consensus_queue.cc:260] T 8028ce9b119b45eb9d05f7d037224788 P 5e88101ae4ec49b2a0be733b6f18ea08 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7670dc9cfd0d4c25a8294f9115c4235b" member_type: VOTER last_known_addr { host: "127.22.167.66" port: 46237 } } peers { permanent_uuid: "5e88101ae4ec49b2a0be733b6f18ea08" member_type: VOTER last_known_addr { host: "127.22.167.65" port: 43639 } } peers { permanent_uuid: "1f3cb452053d447290369216f3e9ed93" member_type: VOTER last_known_addr { host: "127.22.167.67" port: 41131 } }
I20260810 04:41:37.880429 3480 ts_tablet_manager.cc:1428] T 8028ce9b119b45eb9d05f7d037224788 P 5e88101ae4ec49b2a0be733b6f18ea08: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260810 04:41:37.880663 3447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:41:37.880576 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.126:41961 (local address 127.22.167.65:33593)
0810 04:41:37.880609 (+ 33us) negotiation.cc:107] Waiting for socket to connect
0810 04:41:37.880636 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.167.126:41961: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":18}
I20260810 04:41:37.880874 3477 tablet_bootstrap.cc:492] T 8028ce9b119b45eb9d05f7d037224788 P 7670dc9cfd0d4c25a8294f9115c4235b: No bootstrap required, opened a new log
I20260810 04:41:37.880916 3447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:41:37.880840 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.126:41961 (local address 127.22.167.65:42699)
0810 04:41:37.880875 (+ 35us) negotiation.cc:107] Waiting for socket to connect
0810 04:41:37.880896 (+ 21us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.167.126:41961: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":21}
I20260810 04:41:37.880929 3477 ts_tablet_manager.cc:1397] T 8028ce9b119b45eb9d05f7d037224788 P 7670dc9cfd0d4c25a8294f9115c4235b: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260810 04:41:37.881050 3477 raft_consensus.cc:348] T 8028ce9b119b45eb9d05f7d037224788 P 7670dc9cfd0d4c25a8294f9115c4235b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7670dc9cfd0d4c25a8294f9115c4235b" member_type: VOTER last_known_addr { host: "127.22.167.66" port: 46237 } } peers { permanent_uuid: "5e88101ae4ec49b2a0be733b6f18ea08" member_type: VOTER last_known_addr { host: "127.22.167.65" port: 43639 } } peers { permanent_uuid: "1f3cb452053d447290369216f3e9ed93" member_type: VOTER last_known_addr { host: "127.22.167.67" port: 41131 } }
I20260810 04:41:37.881111 2864 heartbeater.cc:503] Master 127.22.167.125:46451 was elected leader, sending a full tablet report...
I20260810 04:41:37.881121 3477 raft_consensus.cc:374] T 8028ce9b119b45eb9d05f7d037224788 P 7670dc9cfd0d4c25a8294f9115c4235b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 04:41:37.881141 3477 raft_consensus.cc:729] T 8028ce9b119b45eb9d05f7d037224788 P 7670dc9cfd0d4c25a8294f9115c4235b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7670dc9cfd0d4c25a8294f9115c4235b, State: Initialized, Role: FOLLOWER
I20260810 04:41:37.881234 3477 consensus_queue.cc:260] T 8028ce9b119b45eb9d05f7d037224788 P 7670dc9cfd0d4c25a8294f9115c4235b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7670dc9cfd0d4c25a8294f9115c4235b" member_type: VOTER last_known_addr { host: "127.22.167.66" port: 46237 } } peers { permanent_uuid: "5e88101ae4ec49b2a0be733b6f18ea08" member_type: VOTER last_known_addr { host: "127.22.167.65" port: 43639 } } peers { permanent_uuid: "1f3cb452053d447290369216f3e9ed93" member_type: VOTER last_known_addr { host: "127.22.167.67" port: 41131 } }
I20260810 04:41:37.881431 3477 ts_tablet_manager.cc:1428] T 8028ce9b119b45eb9d05f7d037224788 P 7670dc9cfd0d4c25a8294f9115c4235b: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260810 04:41:37.880494 3479 tablet_bootstrap.cc:654] T 8028ce9b119b45eb9d05f7d037224788 P 1f3cb452053d447290369216f3e9ed93: Neither blocks nor log segments found. Creating new log.
I20260810 04:41:37.881705 3448 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:41:37.881577 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.126:41961 (local address 127.22.167.66:51025)
0810 04:41:37.881642 (+ 65us) negotiation.cc:107] Waiting for socket to connect
0810 04:41:37.881678 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.167.126:41961: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
I20260810 04:41:37.882001 3448 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:41:37.881912 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.126:41961 (local address 127.22.167.66:40251)
0810 04:41:37.881948 (+ 36us) negotiation.cc:107] Waiting for socket to connect
0810 04:41:37.881977 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.167.126:41961: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":20}
I20260810 04:41:37.882205 3082 heartbeater.cc:503] Master 127.22.167.125:46451 was elected leader, sending a full tablet report...
I20260810 04:41:37.882239 3479 tablet_bootstrap.cc:492] T 8028ce9b119b45eb9d05f7d037224788 P 1f3cb452053d447290369216f3e9ed93: No bootstrap required, opened a new log
I20260810 04:41:37.882278 3479 ts_tablet_manager.cc:1397] T 8028ce9b119b45eb9d05f7d037224788 P 1f3cb452053d447290369216f3e9ed93: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260810 04:41:37.882388 3479 raft_consensus.cc:348] T 8028ce9b119b45eb9d05f7d037224788 P 1f3cb452053d447290369216f3e9ed93 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7670dc9cfd0d4c25a8294f9115c4235b" member_type: VOTER last_known_addr { host: "127.22.167.66" port: 46237 } } peers { permanent_uuid: "5e88101ae4ec49b2a0be733b6f18ea08" member_type: VOTER last_known_addr { host: "127.22.167.65" port: 43639 } } peers { permanent_uuid: "1f3cb452053d447290369216f3e9ed93" member_type: VOTER last_known_addr { host: "127.22.167.67" port: 41131 } }
I20260810 04:41:37.882444 3479 raft_consensus.cc:374] T 8028ce9b119b45eb9d05f7d037224788 P 1f3cb452053d447290369216f3e9ed93 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 04:41:37.882462 3479 raft_consensus.cc:729] T 8028ce9b119b45eb9d05f7d037224788 P 1f3cb452053d447290369216f3e9ed93 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1f3cb452053d447290369216f3e9ed93, State: Initialized, Role: FOLLOWER
I20260810 04:41:37.882514 3479 consensus_queue.cc:260] T 8028ce9b119b45eb9d05f7d037224788 P 1f3cb452053d447290369216f3e9ed93 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7670dc9cfd0d4c25a8294f9115c4235b" member_type: VOTER last_known_addr { host: "127.22.167.66" port: 46237 } } peers { permanent_uuid: "5e88101ae4ec49b2a0be733b6f18ea08" member_type: VOTER last_known_addr { host: "127.22.167.65" port: 43639 } } peers { permanent_uuid: "1f3cb452053d447290369216f3e9ed93" member_type: VOTER last_known_addr { host: "127.22.167.67" port: 41131 } }
I20260810 04:41:37.882668 3479 ts_tablet_manager.cc:1428] T 8028ce9b119b45eb9d05f7d037224788 P 1f3cb452053d447290369216f3e9ed93: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260810 04:41:37.882893 3450 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:41:37.882805 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.126:41961 (local address 127.22.167.67:59945)
0810 04:41:37.882837 (+ 32us) negotiation.cc:107] Waiting for socket to connect
0810 04:41:37.882870 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.167.126:41961: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":18}
I20260810 04:41:37.883093 3319 heartbeater.cc:503] Master 127.22.167.125:46451 was elected leader, sending a full tablet report...
I20260810 04:41:37.883177 3450 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:41:37.883080 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.126:41961 (local address 127.22.167.67:50157)
0810 04:41:37.883124 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0810 04:41:37.883151 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.167.126:41961: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
W20260810 04:41:37.966372 3089 tablet.cc:2369] T 8028ce9b119b45eb9d05f7d037224788 P 7670dc9cfd0d4c25a8294f9115c4235b: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260810 04:41:38.063374 2865 tablet.cc:2369] T 8028ce9b119b45eb9d05f7d037224788 P 5e88101ae4ec49b2a0be733b6f18ea08: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 04:41:38.065755 3485 raft_consensus.cc:482] T 8028ce9b119b45eb9d05f7d037224788 P 5e88101ae4ec49b2a0be733b6f18ea08 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 04:41:38.065837 3485 raft_consensus.cc:504] T 8028ce9b119b45eb9d05f7d037224788 P 5e88101ae4ec49b2a0be733b6f18ea08 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7670dc9cfd0d4c25a8294f9115c4235b" member_type: VOTER last_known_addr { host: "127.22.167.66" port: 46237 } } peers { permanent_uuid: "5e88101ae4ec49b2a0be733b6f18ea08" member_type: VOTER last_known_addr { host: "127.22.167.65" port: 43639 } } peers { permanent_uuid: "1f3cb452053d447290369216f3e9ed93" member_type: VOTER last_known_addr { host: "127.22.167.67" port: 41131 } }
I20260810 04:41:38.066006 3485 leader_election.cc:288] T 8028ce9b119b45eb9d05f7d037224788 P 5e88101ae4ec49b2a0be733b6f18ea08 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7670dc9cfd0d4c25a8294f9115c4235b (127.22.167.66:46237), 1f3cb452053d447290369216f3e9ed93 (127.22.167.67:41131)
I20260810 04:41:38.066262 3227 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8028ce9b119b45eb9d05f7d037224788" candidate_uuid: "5e88101ae4ec49b2a0be733b6f18ea08" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f3cb452053d447290369216f3e9ed93" is_pre_election: true
I20260810 04:41:38.066270 3007 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8028ce9b119b45eb9d05f7d037224788" candidate_uuid: "5e88101ae4ec49b2a0be733b6f18ea08" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7670dc9cfd0d4c25a8294f9115c4235b" is_pre_election: true
I20260810 04:41:38.066354 3227 raft_consensus.cc:2454] T 8028ce9b119b45eb9d05f7d037224788 P 1f3cb452053d447290369216f3e9ed93 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e88101ae4ec49b2a0be733b6f18ea08 in term 0.
I20260810 04:41:38.066376 3007 raft_consensus.cc:2454] T 8028ce9b119b45eb9d05f7d037224788 P 7670dc9cfd0d4c25a8294f9115c4235b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e88101ae4ec49b2a0be733b6f18ea08 in term 0.
I20260810 04:41:38.066509 2696 leader_election.cc:302] T 8028ce9b119b45eb9d05f7d037224788 P 5e88101ae4ec49b2a0be733b6f18ea08 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f3cb452053d447290369216f3e9ed93, 5e88101ae4ec49b2a0be733b6f18ea08; no voters:
I20260810 04:41:38.066620 3485 raft_consensus.cc:2781] T 8028ce9b119b45eb9d05f7d037224788 P 5e88101ae4ec49b2a0be733b6f18ea08 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 04:41:38.066675 3485 raft_consensus.cc:482] T 8028ce9b119b45eb9d05f7d037224788 P 5e88101ae4ec49b2a0be733b6f18ea08 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 04:41:38.066704 3485 raft_consensus.cc:3037] T 8028ce9b119b45eb9d05f7d037224788 P 5e88101ae4ec49b2a0be733b6f18ea08 [term 0 FOLLOWER]: Advancing to term 1
I20260810 04:41:38.067420 3485 raft_consensus.cc:504] T 8028ce9b119b45eb9d05f7d037224788 P 5e88101ae4ec49b2a0be733b6f18ea08 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7670dc9cfd0d4c25a8294f9115c4235b" member_type: VOTER last_known_addr { host: "127.22.167.66" port: 46237 } } peers { permanent_uuid: "5e88101ae4ec49b2a0be733b6f18ea08" member_type: VOTER last_known_addr { host: "127.22.167.65" port: 43639 } } peers { permanent_uuid: "1f3cb452053d447290369216f3e9ed93" member_type: VOTER last_known_addr { host: "127.22.167.67" port: 41131 } }
I20260810 04:41:38.067534 3485 leader_election.cc:288] T 8028ce9b119b45eb9d05f7d037224788 P 5e88101ae4ec49b2a0be733b6f18ea08 [CANDIDATE]: Term 1 election: Requested vote from peers 7670dc9cfd0d4c25a8294f9115c4235b (127.22.167.66:46237), 1f3cb452053d447290369216f3e9ed93 (127.22.167.67:41131)
I20260810 04:41:38.067708 3227 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8028ce9b119b45eb9d05f7d037224788" candidate_uuid: "5e88101ae4ec49b2a0be733b6f18ea08" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f3cb452053d447290369216f3e9ed93"
I20260810 04:41:38.067724 3007 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8028ce9b119b45eb9d05f7d037224788" candidate_uuid: "5e88101ae4ec49b2a0be733b6f18ea08" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7670dc9cfd0d4c25a8294f9115c4235b"
I20260810 04:41:38.067775 3227 raft_consensus.cc:3037] T 8028ce9b119b45eb9d05f7d037224788 P 1f3cb452053d447290369216f3e9ed93 [term 0 FOLLOWER]: Advancing to term 1
I20260810 04:41:38.067790 3007 raft_consensus.cc:3037] T 8028ce9b119b45eb9d05f7d037224788 P 7670dc9cfd0d4c25a8294f9115c4235b [term 0 FOLLOWER]: Advancing to term 1
I20260810 04:41:38.068342 3227 raft_consensus.cc:2454] T 8028ce9b119b45eb9d05f7d037224788 P 1f3cb452053d447290369216f3e9ed93 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e88101ae4ec49b2a0be733b6f18ea08 in term 1.
I20260810 04:41:38.068415 3007 raft_consensus.cc:2454] T 8028ce9b119b45eb9d05f7d037224788 P 7670dc9cfd0d4c25a8294f9115c4235b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e88101ae4ec49b2a0be733b6f18ea08 in term 1.
I20260810 04:41:38.068504 2696 leader_election.cc:302] T 8028ce9b119b45eb9d05f7d037224788 P 5e88101ae4ec49b2a0be733b6f18ea08 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f3cb452053d447290369216f3e9ed93, 5e88101ae4ec49b2a0be733b6f18ea08; no voters:
I20260810 04:41:38.068584 3485 raft_consensus.cc:2781] T 8028ce9b119b45eb9d05f7d037224788 P 5e88101ae4ec49b2a0be733b6f18ea08 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 04:41:38.068737 3485 raft_consensus.cc:686] T 8028ce9b119b45eb9d05f7d037224788 P 5e88101ae4ec49b2a0be733b6f18ea08 [term 1 LEADER]: Becoming Leader. State: Replica: 5e88101ae4ec49b2a0be733b6f18ea08, State: Running, Role: LEADER
I20260810 04:41:38.068783 3485 consensus_queue.cc:237] T 8028ce9b119b45eb9d05f7d037224788 P 5e88101ae4ec49b2a0be733b6f18ea08 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7670dc9cfd0d4c25a8294f9115c4235b" member_type: VOTER last_known_addr { host: "127.22.167.66" port: 46237 } } peers { permanent_uuid: "5e88101ae4ec49b2a0be733b6f18ea08" member_type: VOTER last_known_addr { host: "127.22.167.65" port: 43639 } } peers { permanent_uuid: "1f3cb452053d447290369216f3e9ed93" member_type: VOTER last_known_addr { host: "127.22.167.67" port: 41131 } }
I20260810 04:41:38.069053 3447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:41:38.068953 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.126:41961 (local address 127.22.167.65:59101)
0810 04:41:38.068990 (+ 37us) negotiation.cc:107] Waiting for socket to connect
0810 04:41:38.069021 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.167.126:41961: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":17}
I20260810 04:41:38.069415 2291 catalog_manager.cc:5495] T 8028ce9b119b45eb9d05f7d037224788 P 5e88101ae4ec49b2a0be733b6f18ea08 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5e88101ae4ec49b2a0be733b6f18ea08 (127.22.167.65). New cstate: current_term: 1 leader_uuid: "5e88101ae4ec49b2a0be733b6f18ea08" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7670dc9cfd0d4c25a8294f9115c4235b" member_type: VOTER last_known_addr { host: "127.22.167.66" port: 46237 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5e88101ae4ec49b2a0be733b6f18ea08" member_type: VOTER last_known_addr { host: "127.22.167.65" port: 43639 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1f3cb452053d447290369216f3e9ed93" member_type: VOTER last_known_addr { host: "127.22.167.67" port: 41131 } health_report { overall_health: UNKNOWN } } }
I20260810 04:41:38.100945 3476 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:41:38.098893 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49804 (local address 127.22.167.65:43639)
0810 04:41:38.098954 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0810 04:41:38.098956 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 04:41:38.098965 (+ 9us) server_negotiation.cc:407] Connection header received
0810 04:41:38.099020 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:41:38.099021 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:41:38.099030 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:41:38.099072 (+ 42us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 04:41:38.099180 (+ 108us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:41:38.099688 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:41:38.100447 (+ 759us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:41:38.100632 (+ 185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:41:38.100680 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:41:38.100781 (+ 101us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 04:41:38.100806 (+ 25us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 04:41:38.100838 (+ 32us) server_negotiation.cc:1035] Waiting for connection context
0810 04:41:38.100872 (+ 34us) server_negotiation.cc:299] Negotiation successful
0810 04:41:38.100890 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260810 04:41:38.101740 3227 raft_consensus.cc:1261] T 8028ce9b119b45eb9d05f7d037224788 P 1f3cb452053d447290369216f3e9ed93 [term 1 FOLLOWER]: Refusing update from remote peer 5e88101ae4ec49b2a0be733b6f18ea08: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 04:41:38.101763 3007 raft_consensus.cc:1261] T 8028ce9b119b45eb9d05f7d037224788 P 7670dc9cfd0d4c25a8294f9115c4235b [term 1 FOLLOWER]: Refusing update from remote peer 5e88101ae4ec49b2a0be733b6f18ea08: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 04:41:38.101960 3485 consensus_queue.cc:1035] T 8028ce9b119b45eb9d05f7d037224788 P 5e88101ae4ec49b2a0be733b6f18ea08 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7670dc9cfd0d4c25a8294f9115c4235b" member_type: VOTER last_known_addr { host: "127.22.167.66" port: 46237 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 04:41:38.102067 3490 consensus_queue.cc:1035] T 8028ce9b119b45eb9d05f7d037224788 P 5e88101ae4ec49b2a0be733b6f18ea08 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1f3cb452053d447290369216f3e9ed93" member_type: VOTER last_known_addr { host: "127.22.167.67" port: 41131 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 04:41:38.102708 3448 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:41:38.102603 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.126:41961 (local address 127.22.167.66:57631)
0810 04:41:38.102644 (+ 41us) negotiation.cc:107] Waiting for socket to connect
0810 04:41:38.102670 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.167.126:41961: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":26}
I20260810 04:41:38.103399 3450 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:41:38.103289 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.126:41961 (local address 127.22.167.67:40785)
0810 04:41:38.103328 (+ 39us) negotiation.cc:107] Waiting for socket to connect
0810 04:41:38.103364 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.167.126:41961: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":21}
I20260810 04:41:38.104463 3448 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:41:38.104376 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.126:41961 (local address 127.22.167.66:55631)
0810 04:41:38.104412 (+ 36us) negotiation.cc:107] Waiting for socket to connect
0810 04:41:38.104435 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.167.126:41961: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":24}
I20260810 04:41:38.104491 3447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:41:38.104316 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.126:41961 (local address 127.22.167.65:36231)
0810 04:41:38.104435 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0810 04:41:38.104464 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.167.126:41961: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":104}
I20260810 04:41:38.105506 3447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:41:38.105297 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.126:41961 (local address 127.22.167.65:34835)
0810 04:41:38.105454 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0810 04:41:38.105481 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.167.126:41961: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":145}
I20260810 04:41:38.105561 3497 mvcc.cc:205] Tried to move back new op lower bound from 7316835934623182848 to 7316835934490370048. Current Snapshot: MvccSnapshot[applied={T|T < 7316835934623182848 or (T in {7316835934623182848})}]
I20260810 04:41:38.105667 3450 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:41:38.105562 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.126:41961 (local address 127.22.167.67:53581)
0810 04:41:38.105615 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0810 04:41:38.105639 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.167.126:41961: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
I20260810 04:41:38.105908 2290 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39354:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 04:41:38.105999 2290 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39354:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 04:41:38.106245 3494 mvcc.cc:205] Tried to move back new op lower bound from 7316835934623182848 to 7316835934490370048. Current Snapshot: MvccSnapshot[applied={T|T < 7316835934623182848 or (T in {7316835934623182848})}]
I20260810 04:41:38.107158 2290 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b421e727773044528cb946bc1c2b5b83: Sending DeleteTablet for 3 replicas of tablet 8028ce9b119b45eb9d05f7d037224788
I20260810 04:41:38.107437 3198 tablet_service.cc:1514] Processing DeleteTablet for tablet 8028ce9b119b45eb9d05f7d037224788 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 04:41:38 UTC) from {username='oryx', principal='oryx/127.22.167.125@KRBTEST.COM'} at 127.0.0.1:46174
I20260810 04:41:38.107518 2976 tablet_service.cc:1514] Processing DeleteTablet for tablet 8028ce9b119b45eb9d05f7d037224788 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 04:41:38 UTC) from {username='oryx', principal='oryx/127.22.167.125@KRBTEST.COM'} at 127.0.0.1:33690
I20260810 04:41:38.107609 2765 tablet_service.cc:1514] Processing DeleteTablet for tablet 8028ce9b119b45eb9d05f7d037224788 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 04:41:38 UTC) from {username='oryx', principal='oryx/127.22.167.125@KRBTEST.COM'} at 127.0.0.1:49790
I20260810 04:41:38.107800 3511 tablet_replica.cc:330] T 8028ce9b119b45eb9d05f7d037224788 P 5e88101ae4ec49b2a0be733b6f18ea08: stopping tablet replica
I20260810 04:41:38.107810 3508 tablet_replica.cc:330] T 8028ce9b119b45eb9d05f7d037224788 P 1f3cb452053d447290369216f3e9ed93: stopping tablet replica
I20260810 04:41:38.107838 23197 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskemv_AM/build/release/bin/kudu with pid 2666
I20260810 04:41:38.107862 3508 raft_consensus.cc:2229] T 8028ce9b119b45eb9d05f7d037224788 P 1f3cb452053d447290369216f3e9ed93 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 04:41:38.107903 3511 raft_consensus.cc:2229] T 8028ce9b119b45eb9d05f7d037224788 P 5e88101ae4ec49b2a0be733b6f18ea08 [term 1 LEADER]: Raft consensus shutting down.
I20260810 04:41:38.107973 3511 raft_consensus.cc:2258] T 8028ce9b119b45eb9d05f7d037224788 P 5e88101ae4ec49b2a0be733b6f18ea08 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 04:41:38.108493 3508 raft_consensus.cc:2258] T 8028ce9b119b45eb9d05f7d037224788 P 1f3cb452053d447290369216f3e9ed93 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 04:41:38.111140 3508 ts_tablet_manager.cc:1905] T 8028ce9b119b45eb9d05f7d037224788 P 1f3cb452053d447290369216f3e9ed93: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 04:41:38.113348 3508 ts_tablet_manager.cc:1918] T 8028ce9b119b45eb9d05f7d037224788 P 1f3cb452053d447290369216f3e9ed93: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 04:41:38.113397 3508 log.cc:1196] T 8028ce9b119b45eb9d05f7d037224788 P 1f3cb452053d447290369216f3e9ed93: Deleting WAL directory at /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786336863388938-23197-0/minicluster-data/ts-2/wal/wals/8028ce9b119b45eb9d05f7d037224788
I20260810 04:41:38.113606 3508 ts_tablet_manager.cc:1939] T 8028ce9b119b45eb9d05f7d037224788 P 1f3cb452053d447290369216f3e9ed93: Deleting consensus metadata
I20260810 04:41:38.113725 3509 tablet_replica.cc:330] T 8028ce9b119b45eb9d05f7d037224788 P 7670dc9cfd0d4c25a8294f9115c4235b: stopping tablet replica
I20260810 04:41:38.113780 3509 raft_consensus.cc:2229] T 8028ce9b119b45eb9d05f7d037224788 P 7670dc9cfd0d4c25a8294f9115c4235b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 04:41:38.113817 3509 raft_consensus.cc:2258] T 8028ce9b119b45eb9d05f7d037224788 P 7670dc9cfd0d4c25a8294f9115c4235b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 04:41:38.116660 2272 catalog_manager.cc:4841] TS 1f3cb452053d447290369216f3e9ed93 (127.22.167.67:41131): tablet 8028ce9b119b45eb9d05f7d037224788 (table test-table [id=0924650b58aa4cafae9b4ca313a864ea]) successfully deleted
I20260810 04:41:38.116717 3509 ts_tablet_manager.cc:1905] T 8028ce9b119b45eb9d05f7d037224788 P 7670dc9cfd0d4c25a8294f9115c4235b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 04:41:38.117906 3509 ts_tablet_manager.cc:1918] T 8028ce9b119b45eb9d05f7d037224788 P 7670dc9cfd0d4c25a8294f9115c4235b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 04:41:38.117954 3509 log.cc:1196] T 8028ce9b119b45eb9d05f7d037224788 P 7670dc9cfd0d4c25a8294f9115c4235b: Deleting WAL directory at /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786336863388938-23197-0/minicluster-data/ts-1/wal/wals/8028ce9b119b45eb9d05f7d037224788
I20260810 04:41:38.118119 3509 ts_tablet_manager.cc:1939] T 8028ce9b119b45eb9d05f7d037224788 P 7670dc9cfd0d4c25a8294f9115c4235b: Deleting consensus metadata
I20260810 04:41:38.118440 2271 catalog_manager.cc:4841] TS 7670dc9cfd0d4c25a8294f9115c4235b (127.22.167.66:46237): tablet 8028ce9b119b45eb9d05f7d037224788 (table test-table [id=0924650b58aa4cafae9b4ca313a864ea]) successfully deleted
W20260810 04:41:38.123013 3470 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 04:41:38.122890 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.65:43639 (local address 127.0.0.1:49812)
0810 04:41:38.122935 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0810 04:41:38.122962 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.167.65:43639: connect: Connection reset by peer (error 104)
Metrics: {"client-negotiator.queue_time_us":24}
W20260810 04:41:38.123140 2270 catalog_manager.cc:4584] TS 5e88101ae4ec49b2a0be733b6f18ea08 (127.22.167.65:43639): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 8028ce9b119b45eb9d05f7d037224788: Network error: Client connection negotiation failed: client connection to 127.22.167.65:43639: connect: Connection reset by peer (error 104)
I20260810 04:41:38.123195 23197 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskemv_AM/build/release/bin/kudu with pid 2872
I20260810 04:41:38.130276 23197 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskemv_AM/build/release/bin/kudu with pid 3093
I20260810 04:41:38.139869 23197 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskemv_AM/build/release/bin/kudu with pid 2088
I20260810 04:41:38.147092 23197 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskemv_AM/build/release/bin/kudu with pid 2369
2026-08-10T04:41:38Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipalMultipleMaster (6488 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786336863388938-23197-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 04:41:38 dist-test-slave-bqcl krb5kdc[3517](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 04:41:38 dist-test-slave-bqcl krb5kdc[3517](info): set up 2 sockets
Aug 10 04:41:38 dist-test-slave-bqcl krb5kdc[3517](info): commencing operation
krb5kdc: starting...
W20260810 04:41:39.327261 23197 mini_kdc.cc:121] Time spent starting KDC: real 1.156s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786336863388938-23197-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786336863388938-23197-0/krb5kdc/test-user.keytab.
Aug 10 04:41:39 dist-test-slave-bqcl krb5kdc[3517](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786336899, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T04:41:39Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T04:41:39Z Disabled control of system clock
WARNING: no policy specified for oryx/127.22.167.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.22.167.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.22.167.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786336863388938-23197-0/krb5kdc/oryx_127.22.167.126.keytab.
Entry for principal oryx/127.22.167.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786336863388938-23197-0/krb5kdc/oryx_127.22.167.126.keytab.
I20260810 04:41:39.444470 23197 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskemv_AM/build/release/bin/kudu
/tmp/dist-test-taskemv_AM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786336863388938-23197-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786336863388938-23197-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786336863388938-23197-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786336863388938-23197-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.167.126:37713
--webserver_interface=127.22.167.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786336863388938-23197-0/krb5kdc/oryx_127.22.167.126.keytab
--principal=oryx/127.22.167.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.167.84:43155
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.167.126:37713
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786336863388938-23197-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786336863388938-23197-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786336863388938-23197-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 04:41:39.540748 3535 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 04:41:39.540947 3535 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 04:41:39.540968 3535 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 04:41:39.542030 3535 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 04:41:39.542063 3535 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260810 04:41:39.542075 3535 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 04:41:39.542086 3535 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 04:41:39.542101 3535 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 04:41:39.542112 3535 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 04:41:39.543603 3535 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.167.84:43155
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786336863388938-23197-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786336863388938-23197-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.22.167.126:37713
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786336863388938-23197-0/krb5kdc/oryx_127.22.167.126.keytab
--principal=oryx/127.22.167.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.167.126:37713
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786336863388938-23197-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.167.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3535
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786336863388938-23197-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 03:54:01 UTC on ccycloud.quasar-icijmt.root.comops.site
build id 4577009
I20260810 04:41:39.543797 3535 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 04:41:39.543973 3535 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 04:41:39.548568 3551 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 04:41:39.548579 3545 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 04:41:39.548604 3553 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 04:41:39.548755 3535 server_base.cc:1034] running on GCE node
I20260810 04:41:39.549021 3535 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 04:41:39.549309 3535 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 04:41:39.550447 3535 hybrid_clock.cc:648] HybridClock initialized: now 1786336899550430 us; error 28 us; skew 500 ppm
Aug 10 04:41:39 dist-test-slave-bqcl krb5kdc[3517](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786336899, etypes {rep=17 tkt=17 ses=17}, oryx/127.22.167.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 04:41:39.552770 3535 init.cc:377] Logged in from keytab as oryx/127.22.167.126@KRBTEST.COM (short username oryx)
I20260810 04:41:39.553164 3535 webserver.cc:460] Webserver started at http://127.22.167.126:42107/ using document root <none> and password file <none>
I20260810 04:41:39.553376 3535 fs_manager.cc:359] Metadata directory not provided
I20260810 04:41:39.553426 3535 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 04:41:39.553552 3535 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 04:41:39.554611 3535 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786336863388938-23197-0/minicluster-data/master-0/data/instance:
uuid: "b8bbdb416f794faeb39ac8593eebe708"
format_stamp: "Formatted at 2026-08-10 04:41:39 on dist-test-slave-bqcl"
server_key: "3fe48965eb7e9bbe05e7e12bcb15d3af"
server_key_iv: "02219009fa64adad25011be947bc8177"
server_key_version: "encryptionkey@0"
I20260810 04:41:39.554944 3535 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786336863388938-23197-0/minicluster-data/master-0/wal/instance:
uuid: "b8bbdb416f794faeb39ac8593eebe708"
format_stamp: "Formatted at 2026-08-10 04:41:39 on dist-test-slave-bqcl"
server_key: "3fe48965eb7e9bbe05e7e12bcb15d3af"
server_key_iv: "02219009fa64adad25011be947bc8177"
server_key_version: "encryptionkey@0"
I20260810 04:41:39.556149 3535 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260810 04:41:39.556770 3567 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 04:41:39.556917 3535 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260810 04:41:39.556999 3535 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786336863388938-23197-0/minicluster-data/master-0/data,/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786336863388938-23197-0/minicluster-data/master-0/wal
uuid: "b8bbdb416f794faeb39ac8593eebe708"
format_stamp: "Formatted at 2026-08-10 04:41:39 on dist-test-slave-bqcl"
server_key: "3fe48965eb7e9bbe05e7e12bcb15d3af"
server_key_iv: "02219009fa64adad25011be947bc8177"
server_key_version: "encryptionkey@0"
I20260810 04:41:39.557057 3535 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786336863388938-23197-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786336863388938-23197-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786336863388938-23197-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 04:41:39.577245 3535 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 04:41:39.578922 3535 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 04:41:39.579052 3535 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 04:41:39.583401 3535 rpc_server.cc:307] RPC server started. Bound to: 127.22.167.126:37713
I20260810 04:41:39.583475 3662 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.167.126:37713 every 8 connection(s)
I20260810 04:41:39.583745 3535 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786336863388938-23197-0/minicluster-data/master-0/data/info.pb
I20260810 04:41:39.584395 3663 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 04:41:39.586632 3663 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b8bbdb416f794faeb39ac8593eebe708: Bootstrap starting.
I20260810 04:41:39.587186 3663 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b8bbdb416f794faeb39ac8593eebe708: Neither blocks nor log segments found. Creating new log.
I20260810 04:41:39.587469 3663 log.cc:824] T 00000000000000000000000000000000 P b8bbdb416f794faeb39ac8593eebe708: Log is configured to *not* fsync() on all Append() calls
I20260810 04:41:39.588073 3663 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b8bbdb416f794faeb39ac8593eebe708: No bootstrap required, opened a new log
I20260810 04:41:39.588899 23197 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskemv_AM/build/release/bin/kudu as pid 3535
I20260810 04:41:39.588996 23197 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786336863388938-23197-0/minicluster-data/master-0/wal/instance
I20260810 04:41:39.589178 23197 external_mini_cluster.cc:1428] Setting key 15cea34fc154b1942fcdcb01e13ff985
I20260810 04:41:39.590325 3663 raft_consensus.cc:348] T 00000000000000000000000000000000 P b8bbdb416f794faeb39ac8593eebe708 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b8bbdb416f794faeb39ac8593eebe708" member_type: VOTER last_known_addr { host: "127.22.167.126" port: 37713 } }
I20260810 04:41:39.590487 3663 raft_consensus.cc:374] T 00000000000000000000000000000000 P b8bbdb416f794faeb39ac8593eebe708 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 04:41:39.590561 3663 raft_consensus.cc:729] T 00000000000000000000000000000000 P b8bbdb416f794faeb39ac8593eebe708 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b8bbdb416f794faeb39ac8593eebe708, State: Initialized, Role: FOLLOWER
I20260810 04:41:39.590689 3663 consensus_queue.cc:260] T 00000000000000000000000000000000 P b8bbdb416f794faeb39ac8593eebe708 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b8bbdb416f794faeb39ac8593eebe708" member_type: VOTER last_known_addr { host: "127.22.167.126" port: 37713 } }
I20260810 04:41:39.590770 3663 raft_consensus.cc:388] T 00000000000000000000000000000000 P b8bbdb416f794faeb39ac8593eebe708 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 04:41:39.590793 3663 raft_consensus.cc:482] T 00000000000000000000000000000000 P b8bbdb416f794faeb39ac8593eebe708 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 04:41:39.590837 3663 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b8bbdb416f794faeb39ac8593eebe708 [term 0 FOLLOWER]: Advancing to term 1
I20260810 04:41:39.591418 3663 raft_consensus.cc:504] T 00000000000000000000000000000000 P b8bbdb416f794faeb39ac8593eebe708 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b8bbdb416f794faeb39ac8593eebe708" member_type: VOTER last_known_addr { host: "127.22.167.126" port: 37713 } }
I20260810 04:41:39.591522 3663 leader_election.cc:302] T 00000000000000000000000000000000 P b8bbdb416f794faeb39ac8593eebe708 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b8bbdb416f794faeb39ac8593eebe708; no voters:
I20260810 04:41:39.591668 3663 leader_election.cc:288] T 00000000000000000000000000000000 P b8bbdb416f794faeb39ac8593eebe708 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 04:41:39.591717 3669 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b8bbdb416f794faeb39ac8593eebe708 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 04:41:39.591866 3669 raft_consensus.cc:686] T 00000000000000000000000000000000 P b8bbdb416f794faeb39ac8593eebe708 [term 1 LEADER]: Becoming Leader. State: Replica: b8bbdb416f794faeb39ac8593eebe708, State: Running, Role: LEADER
I20260810 04:41:39.591908 3663 sys_catalog.cc:564] T 00000000000000000000000000000000 P b8bbdb416f794faeb39ac8593eebe708 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 04:41:39.592017 3669 consensus_queue.cc:237] T 00000000000000000000000000000000 P b8bbdb416f794faeb39ac8593eebe708 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b8bbdb416f794faeb39ac8593eebe708" member_type: VOTER last_known_addr { host: "127.22.167.126" port: 37713 } }
I20260810 04:41:39.593564 3670 sys_catalog.cc:455] T 00000000000000000000000000000000 P b8bbdb416f794faeb39ac8593eebe708 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b8bbdb416f794faeb39ac8593eebe708. Latest consensus state: current_term: 1 leader_uuid: "b8bbdb416f794faeb39ac8593eebe708" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b8bbdb416f794faeb39ac8593eebe708" member_type: VOTER last_known_addr { host: "127.22.167.126" port: 37713 } } }
I20260810 04:41:39.593654 3670 sys_catalog.cc:458] T 00000000000000000000000000000000 P b8bbdb416f794faeb39ac8593eebe708 [sys.catalog]: This master's current role is: LEADER
I20260810 04:41:39.593906 3695 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 04:41:39.594197 3672 sys_catalog.cc:455] T 00000000000000000000000000000000 P b8bbdb416f794faeb39ac8593eebe708 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b8bbdb416f794faeb39ac8593eebe708" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b8bbdb416f794faeb39ac8593eebe708" member_type: VOTER last_known_addr { host: "127.22.167.126" port: 37713 } } }
I20260810 04:41:39.594271 3672 sys_catalog.cc:458] T 00000000000000000000000000000000 P b8bbdb416f794faeb39ac8593eebe708 [sys.catalog]: This master's current role is: LEADER
I20260810 04:41:39.594612 3695 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Aug 10 04:41:39 dist-test-slave-bqcl krb5kdc[3517](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786336899, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.22.167.126@KRBTEST.COM
I20260810 04:41:39.597298 3695 catalog_manager.cc:1347] Generated new cluster ID: 43aabd80f68747529e42bfcbce3d98ed
I20260810 04:41:39.597347 3695 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 04:41:39.599932 3668 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:41:39.589649 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52934 (local address 127.22.167.126:37713)
0810 04:41:39.589835 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0810 04:41:39.589844 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 04:41:39.589876 (+ 32us) server_negotiation.cc:407] Connection header received
0810 04:41:39.590302 (+ 426us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:41:39.590310 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:41:39.590372 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:41:39.590517 (+ 145us) server_negotiation.cc:226] Negotiated authn=SASL
0810 04:41:39.591195 (+ 678us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:41:39.592527 (+ 1332us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:41:39.593580 (+ 1053us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:41:39.594155 (+ 575us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:41:39.594268 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:41:39.596958 (+ 2690us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 04:41:39.596961 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 04:41:39.596962 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 04:41:39.596971 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0810 04:41:39.598537 (+ 1566us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 04:41:39.598856 (+ 319us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 04:41:39.598859 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 04:41:39.598859 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 04:41:39.598952 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 04:41:39.599133 (+ 181us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 04:41:39.599134 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 04:41:39.599134 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 04:41:39.599353 (+ 219us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 04:41:39.599498 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0810 04:41:39.599603 (+ 105us) server_negotiation.cc:299] Negotiation successful
0810 04:41:39.599731 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":128,"thread_start_us":79,"threads_started":1}
I20260810 04:41:39.607153 3695 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 04:41:39.607877 3695 catalog_manager.cc:1504] Loading token signing keys...
I20260810 04:41:39.614176 3695 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b8bbdb416f794faeb39ac8593eebe708: Generated new TSK 0
I20260810 04:41:39.614334 3695 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.22.167.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.22.167.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.22.167.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786336863388938-23197-0/krb5kdc/oryx_127.22.167.65.keytab.
Entry for principal oryx/127.22.167.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786336863388938-23197-0/krb5kdc/oryx_127.22.167.65.keytab.
I20260810 04:41:39.640533 23197 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskemv_AM/build/release/bin/kudu
/tmp/dist-test-taskemv_AM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786336863388938-23197-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786336863388938-23197-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786336863388938-23197-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786336863388938-23197-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.167.65:0
--local_ip_for_outbound_sockets=127.22.167.65
--webserver_interface=127.22.167.65
--webserver_port=0
--tserver_master_addrs=127.22.167.126:37713
--keytab_file=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786336863388938-23197-0/krb5kdc/oryx_127.22.167.65.keytab
--principal=oryx/127.22.167.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.167.84:43155
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786336863388938-23197-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786336863388938-23197-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786336863388938-23197-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 04:41:39.719967 3703 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 04:41:39.720156 3703 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 04:41:39.720188 3703 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 04:41:39.721293 3703 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 04:41:39.721333 3703 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 04:41:39.721359 3703 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260810 04:41:39.721381 3703 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.167.65
I20260810 04:41:39.722836 3703 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.167.84:43155
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786336863388938-23197-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786336863388938-23197-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786336863388938-23197-0/krb5kdc/oryx_127.22.167.65.keytab
--principal=oryx/127.22.167.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.167.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786336863388938-23197-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.22.167.65
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.22.167.126:37713
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3703
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.167.65
--log_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786336863388938-23197-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 03:54:01 UTC on ccycloud.quasar-icijmt.root.comops.site
build id 4577009
I20260810 04:41:39.723050 3703 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 04:41:39.723254 3703 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260810 04:41:39.728156 3703 server_base.cc:1034] running on GCE node
W20260810 04:41:39.728232 3719 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 04:41:39.728135 3712 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 04:41:39.728135 3716 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 04:41:39.728510 3703 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 04:41:39.728737 3703 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 04:41:39.729866 3703 hybrid_clock.cc:648] HybridClock initialized: now 1786336899729857 us; error 29 us; skew 500 ppm
Aug 10 04:41:39 dist-test-slave-bqcl krb5kdc[3517](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786336899, etypes {rep=17 tkt=17 ses=17}, oryx/127.22.167.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 04:41:39.732277 3703 init.cc:377] Logged in from keytab as oryx/127.22.167.65@KRBTEST.COM (short username oryx)
I20260810 04:41:39.732686 3703 webserver.cc:460] Webserver started at http://127.22.167.65:38487/ using document root <none> and password file <none>
I20260810 04:41:39.732877 3703 fs_manager.cc:359] Metadata directory not provided
I20260810 04:41:39.732923 3703 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 04:41:39.733043 3703 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 04:41:39.733992 3703 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786336863388938-23197-0/minicluster-data/ts-0/data/instance:
uuid: "84e7718d636e41efbd3a4b32db54e3f8"
format_stamp: "Formatted at 2026-08-10 04:41:39 on dist-test-slave-bqcl"
server_key: "582d99a9788e1bb3a8c5252af51fd630"
server_key_iv: "d8f3afdebebd1130d428ed5ccb5acbb8"
server_key_version: "encryptionkey@0"
I20260810 04:41:39.734357 3703 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786336863388938-23197-0/minicluster-data/ts-0/wal/instance:
uuid: "84e7718d636e41efbd3a4b32db54e3f8"
format_stamp: "Formatted at 2026-08-10 04:41:39 on dist-test-slave-bqcl"
server_key: "582d99a9788e1bb3a8c5252af51fd630"
server_key_iv: "d8f3afdebebd1130d428ed5ccb5acbb8"
server_key_version: "encryptionkey@0"
I20260810 04:41:39.735530 3703 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260810 04:41:39.736132 3730 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 04:41:39.736295 3703 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260810 04:41:39.736385 3703 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786336863388938-23197-0/minicluster-data/ts-0/data,/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786336863388938-23197-0/minicluster-data/ts-0/wal
uuid: "84e7718d636e41efbd3a4b32db54e3f8"
format_stamp: "Formatted at 2026-08-10 04:41:39 on dist-test-slave-bqcl"
server_key: "582d99a9788e1bb3a8c5252af51fd630"
server_key_iv: "d8f3afdebebd1130d428ed5ccb5acbb8"
server_key_version: "encryptionkey@0"
I20260810 04:41:39.736446 3703 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786336863388938-23197-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786336863388938-23197-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786336863388938-23197-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 04:41:39.777177 3703 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 04:41:39.779299 3703 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 04:41:39.779434 3703 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 04:41:39.779920 3703 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 04:41:39.779960 3703 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 04:41:39.779990 3703 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 04:41:39.780007 3703 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 10 04:41:39 dist-test-slave-bqcl krb5kdc[3517](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786336899, etypes {rep=17 tkt=17 ses=17}, oryx/127.22.167.65@KRBTEST.COM for oryx/127.22.167.126@KRBTEST.COM
I20260810 04:41:39.786501 3703 rpc_server.cc:307] RPC server started. Bound to: 127.22.167.65:40941
I20260810 04:41:39.786556 3913 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.167.65:40941 every 8 connection(s)
I20260810 04:41:39.786906 3703 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786336863388938-23197-0/minicluster-data/ts-0/data/info.pb
I20260810 04:41:39.788806 3668 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:41:39.780356 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.167.65:51541 (local address 127.22.167.126:37713)
0810 04:41:39.780409 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0810 04:41:39.780411 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 04:41:39.780621 (+ 210us) server_negotiation.cc:407] Connection header received
0810 04:41:39.781115 (+ 494us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:41:39.781117 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:41:39.781130 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:41:39.781183 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0810 04:41:39.782008 (+ 825us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:41:39.782794 (+ 786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:41:39.783805 (+ 1011us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:41:39.784026 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:41:39.784084 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:41:39.786117 (+ 2033us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 04:41:39.786142 (+ 25us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 04:41:39.786142 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 04:41:39.786154 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0810 04:41:39.787553 (+ 1399us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 04:41:39.787911 (+ 358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 04:41:39.787913 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 04:41:39.787913 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 04:41:39.787972 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 04:41:39.788190 (+ 218us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 04:41:39.788190 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 04:41:39.788191 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 04:41:39.788306 (+ 115us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 04:41:39.788362 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0810 04:41:39.788617 (+ 255us) server_negotiation.cc:299] Negotiation successful
0810 04:41:39.788717 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260810 04:41:39.788939 3745 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:41:39.780353 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.126:37713 (local address 127.22.167.65:51541)
0810 04:41:39.780497 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0810 04:41:39.780532 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0810 04:41:39.781012 (+ 480us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 04:41:39.781264 (+ 252us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 04:41:39.781269 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 04:41:39.781612 (+ 343us) client_negotiation.cc:189] Negotiated authn=SASL
0810 04:41:39.781886 (+ 274us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:41:39.781895 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:41:39.782918 (+ 1023us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:41:39.782920 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:41:39.783716 (+ 796us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:41:39.783719 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:41:39.784120 (+ 401us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:41:39.784121 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:41:39.784236 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:41:39.784703 (+ 467us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 04:41:39.784711 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0810 04:41:39.786025 (+ 1314us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 04:41:39.787625 (+ 1600us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 04:41:39.787632 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 04:41:39.787634 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 04:41:39.787843 (+ 209us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 04:41:39.788026 (+ 183us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 04:41:39.788027 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 04:41:39.788027 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 04:41:39.788133 (+ 106us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 04:41:39.788361 (+ 228us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 04:41:39.788368 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 04:41:39.788498 (+ 130us) client_negotiation.cc:769] Sending connection context
0810 04:41:39.788585 (+ 87us) client_negotiation.cc:240] Negotiation successful
0810 04:41:39.788735 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":76,"threads_started":1}
I20260810 04:41:39.789381 3915 heartbeater.cc:347] Connected to a master server at 127.22.167.126:37713
I20260810 04:41:39.789466 3915 heartbeater.cc:464] Registering TS with master...
I20260810 04:41:39.789633 3915 heartbeater.cc:511] Master 127.22.167.126:37713 requested a full tablet report, sending...
I20260810 04:41:39.790212 3596 ts_manager.cc:194] Registered new tserver with Master: 84e7718d636e41efbd3a4b32db54e3f8 (127.22.167.65:40941)
I20260810 04:41:39.790848 3596 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.22.167.65@KRBTEST.COM'} at 127.22.167.65:51541
I20260810 04:41:39.795290 23197 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskemv_AM/build/release/bin/kudu as pid 3703
I20260810 04:41:39.795396 23197 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786336863388938-23197-0/minicluster-data/ts-0/wal/instance
I20260810 04:41:39.795608 23197 external_mini_cluster.cc:1428] Setting key 7207b38352a4319982ef0f00df35fc1a
I20260810 04:41:39.795665 3928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:41:39.790323 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.126:37713 (local address 127.22.167.65:57305)
0810 04:41:39.790449 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0810 04:41:39.790459 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0810 04:41:39.790497 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 04:41:39.791114 (+ 617us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 04:41:39.791115 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 04:41:39.791300 (+ 185us) client_negotiation.cc:189] Negotiated authn=SASL
0810 04:41:39.791358 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:41:39.791360 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:41:39.792037 (+ 677us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:41:39.792038 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:41:39.792584 (+ 546us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:41:39.792586 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:41:39.792941 (+ 355us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:41:39.792942 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:41:39.793001 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:41:39.793311 (+ 310us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 04:41:39.793314 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0810 04:41:39.793579 (+ 265us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 04:41:39.794733 (+ 1154us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 04:41:39.794734 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 04:41:39.794734 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 04:41:39.794913 (+ 179us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 04:41:39.795080 (+ 167us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 04:41:39.795081 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 04:41:39.795081 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 04:41:39.795133 (+ 52us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 04:41:39.795336 (+ 203us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 04:41:39.795337 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 04:41:39.795382 (+ 45us) client_negotiation.cc:769] Sending connection context
0810 04:41:39.795471 (+ 89us) client_negotiation.cc:240] Negotiation successful
0810 04:41:39.795564 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":104,"thread_start_us":67,"threads_started":1}
I20260810 04:41:39.795692 3668 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:41:39.790382 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.167.65:57305 (local address 127.22.167.126:37713)
0810 04:41:39.791027 (+ 645us) server_negotiation.cc:206] Beginning negotiation
0810 04:41:39.791029 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 04:41:39.791038 (+ 9us) server_negotiation.cc:407] Connection header received
0810 04:41:39.791053 (+ 15us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:41:39.791054 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:41:39.791060 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:41:39.791104 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0810 04:41:39.791433 (+ 329us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:41:39.791951 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:41:39.792651 (+ 700us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:41:39.792861 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:41:39.792915 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:41:39.793652 (+ 737us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 04:41:39.793654 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 04:41:39.793654 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 04:41:39.793661 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 04:41:39.794662 (+ 1001us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 04:41:39.794975 (+ 313us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 04:41:39.794976 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 04:41:39.794976 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 04:41:39.795028 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 04:41:39.795183 (+ 155us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 04:41:39.795184 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 04:41:39.795184 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 04:41:39.795284 (+ 100us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 04:41:39.795337 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0810 04:41:39.795519 (+ 182us) server_negotiation.cc:299] Negotiation successful
0810 04:41:39.795600 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":626}
WARNING: no policy specified for oryx/127.22.167.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.22.167.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.22.167.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786336863388938-23197-0/krb5kdc/oryx_127.22.167.66.keytab.
Entry for principal oryx/127.22.167.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786336863388938-23197-0/krb5kdc/oryx_127.22.167.66.keytab.
I20260810 04:41:39.820509 23197 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskemv_AM/build/release/bin/kudu
/tmp/dist-test-taskemv_AM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786336863388938-23197-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786336863388938-23197-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786336863388938-23197-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786336863388938-23197-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.167.66:0
--local_ip_for_outbound_sockets=127.22.167.66
--webserver_interface=127.22.167.66
--webserver_port=0
--tserver_master_addrs=127.22.167.126:37713
--keytab_file=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786336863388938-23197-0/krb5kdc/oryx_127.22.167.66.keytab
--principal=oryx/127.22.167.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.167.84:43155
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786336863388938-23197-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786336863388938-23197-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786336863388938-23197-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 04:41:39.899559 3931 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 04:41:39.899752 3931 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 04:41:39.899775 3931 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 04:41:39.900861 3931 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 04:41:39.900900 3931 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 04:41:39.900921 3931 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260810 04:41:39.900952 3931 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.167.66
I20260810 04:41:39.902449 3931 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.167.84:43155
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786336863388938-23197-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786336863388938-23197-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786336863388938-23197-0/krb5kdc/oryx_127.22.167.66.keytab
--principal=oryx/127.22.167.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.167.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786336863388938-23197-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.22.167.66
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.22.167.126:37713
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3931
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.167.66
--log_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786336863388938-23197-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 03:54:01 UTC on ccycloud.quasar-icijmt.root.comops.site
build id 4577009
I20260810 04:41:39.902652 3931 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 04:41:39.902848 3931 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260810 04:41:39.907600 3931 server_base.cc:1034] running on GCE node
W20260810 04:41:39.907608 3940 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 04:41:39.907568 3939 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 04:41:39.907567 3942 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 04:41:39.908010 3931 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 04:41:39.908231 3931 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 04:41:39.909372 3931 hybrid_clock.cc:648] HybridClock initialized: now 1786336899909359 us; error 26 us; skew 500 ppm
Aug 10 04:41:39 dist-test-slave-bqcl krb5kdc[3517](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786336899, etypes {rep=17 tkt=17 ses=17}, oryx/127.22.167.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 04:41:39.911695 3931 init.cc:377] Logged in from keytab as oryx/127.22.167.66@KRBTEST.COM (short username oryx)
I20260810 04:41:39.912101 3931 webserver.cc:460] Webserver started at http://127.22.167.66:33419/ using document root <none> and password file <none>
I20260810 04:41:39.912288 3931 fs_manager.cc:359] Metadata directory not provided
I20260810 04:41:39.912334 3931 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 04:41:39.912459 3931 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 04:41:39.913456 3931 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786336863388938-23197-0/minicluster-data/ts-1/data/instance:
uuid: "603becb479b7443f854a91ab3530293a"
format_stamp: "Formatted at 2026-08-10 04:41:39 on dist-test-slave-bqcl"
server_key: "4505ed67ab9dc858e9f058be16ebed68"
server_key_iv: "61f5a11e1e30a0ee4b621c3bc7046dc9"
server_key_version: "encryptionkey@0"
I20260810 04:41:39.913800 3931 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786336863388938-23197-0/minicluster-data/ts-1/wal/instance:
uuid: "603becb479b7443f854a91ab3530293a"
format_stamp: "Formatted at 2026-08-10 04:41:39 on dist-test-slave-bqcl"
server_key: "4505ed67ab9dc858e9f058be16ebed68"
server_key_iv: "61f5a11e1e30a0ee4b621c3bc7046dc9"
server_key_version: "encryptionkey@0"
I20260810 04:41:39.915063 3931 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260810 04:41:39.915709 3955 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 04:41:39.915855 3931 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260810 04:41:39.915938 3931 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786336863388938-23197-0/minicluster-data/ts-1/data,/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786336863388938-23197-0/minicluster-data/ts-1/wal
uuid: "603becb479b7443f854a91ab3530293a"
format_stamp: "Formatted at 2026-08-10 04:41:39 on dist-test-slave-bqcl"
server_key: "4505ed67ab9dc858e9f058be16ebed68"
server_key_iv: "61f5a11e1e30a0ee4b621c3bc7046dc9"
server_key_version: "encryptionkey@0"
I20260810 04:41:39.915995 3931 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786336863388938-23197-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786336863388938-23197-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786336863388938-23197-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 04:41:39.927412 3931 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 04:41:39.929190 3931 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 04:41:39.929317 3931 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 04:41:39.929785 3931 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 04:41:39.929826 3931 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 04:41:39.929857 3931 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 04:41:39.929880 3931 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 10 04:41:39 dist-test-slave-bqcl krb5kdc[3517](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786336899, etypes {rep=17 tkt=17 ses=17}, oryx/127.22.167.66@KRBTEST.COM for oryx/127.22.167.126@KRBTEST.COM
I20260810 04:41:39.936676 3931 rpc_server.cc:307] RPC server started. Bound to: 127.22.167.66:34655
I20260810 04:41:39.936748 4136 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.167.66:34655 every 8 connection(s)
I20260810 04:41:39.937055 3931 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786336863388938-23197-0/minicluster-data/ts-1/data/info.pb
I20260810 04:41:39.939332 3668 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:41:39.930343 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.167.66:48213 (local address 127.22.167.126:37713)
0810 04:41:39.930409 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0810 04:41:39.930411 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 04:41:39.930637 (+ 226us) server_negotiation.cc:407] Connection header received
0810 04:41:39.931279 (+ 642us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:41:39.931280 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:41:39.931293 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:41:39.931361 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0810 04:41:39.932178 (+ 817us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:41:39.932943 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:41:39.933978 (+ 1035us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:41:39.934269 (+ 291us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:41:39.934338 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:41:39.936405 (+ 2067us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 04:41:39.936406 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 04:41:39.936407 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 04:41:39.936415 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0810 04:41:39.937820 (+ 1405us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 04:41:39.938268 (+ 448us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 04:41:39.938270 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 04:41:39.938270 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 04:41:39.938335 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 04:41:39.938637 (+ 302us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 04:41:39.938638 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 04:41:39.938639 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 04:41:39.938817 (+ 178us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 04:41:39.938890 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0810 04:41:39.939185 (+ 295us) server_negotiation.cc:299] Negotiation successful
0810 04:41:39.939269 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260810 04:41:39.939517 3971 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:41:39.930321 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.126:37713 (local address 127.22.167.66:48213)
0810 04:41:39.930491 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0810 04:41:39.930537 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0810 04:41:39.931149 (+ 612us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 04:41:39.931437 (+ 288us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 04:41:39.931441 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 04:41:39.931785 (+ 344us) client_negotiation.cc:189] Negotiated authn=SASL
0810 04:41:39.932075 (+ 290us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:41:39.932081 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:41:39.933037 (+ 956us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:41:39.933038 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:41:39.933895 (+ 857us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:41:39.933898 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:41:39.934361 (+ 463us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:41:39.934363 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:41:39.934519 (+ 156us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:41:39.934989 (+ 470us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 04:41:39.934997 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0810 04:41:39.936318 (+ 1321us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 04:41:39.937888 (+ 1570us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 04:41:39.937892 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 04:41:39.937893 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 04:41:39.938183 (+ 290us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 04:41:39.938397 (+ 214us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 04:41:39.938399 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 04:41:39.938399 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 04:41:39.938579 (+ 180us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 04:41:39.938885 (+ 306us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 04:41:39.938889 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 04:41:39.939053 (+ 164us) client_negotiation.cc:769] Sending connection context
0810 04:41:39.939158 (+ 105us) client_negotiation.cc:240] Negotiation successful
0810 04:41:39.939308 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":86,"threads_started":1}
I20260810 04:41:39.940003 4137 heartbeater.cc:347] Connected to a master server at 127.22.167.126:37713
I20260810 04:41:39.940084 4137 heartbeater.cc:464] Registering TS with master...
I20260810 04:41:39.940246 4137 heartbeater.cc:511] Master 127.22.167.126:37713 requested a full tablet report, sending...
I20260810 04:41:39.940618 3596 ts_manager.cc:194] Registered new tserver with Master: 603becb479b7443f854a91ab3530293a (127.22.167.66:34655)
I20260810 04:41:39.941059 3596 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.22.167.66@KRBTEST.COM'} at 127.22.167.66:48213
I20260810 04:41:39.944736 23197 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskemv_AM/build/release/bin/kudu as pid 3931
I20260810 04:41:39.944854 23197 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786336863388938-23197-0/minicluster-data/ts-1/wal/instance
I20260810 04:41:39.945058 23197 external_mini_cluster.cc:1428] Setting key 6f2fc74d81b7e272c3da72943cc1c742
I20260810 04:41:39.945978 4144 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:41:39.940867 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.126:37713 (local address 127.22.167.66:36995)
0810 04:41:39.940985 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0810 04:41:39.940996 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0810 04:41:39.941052 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 04:41:39.941284 (+ 232us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 04:41:39.941285 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 04:41:39.941436 (+ 151us) client_negotiation.cc:189] Negotiated authn=SASL
0810 04:41:39.941500 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:41:39.941501 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:41:39.942215 (+ 714us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:41:39.942216 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:41:39.942722 (+ 506us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:41:39.942724 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:41:39.943002 (+ 278us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:41:39.943003 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:41:39.943044 (+ 41us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:41:39.943339 (+ 295us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 04:41:39.943342 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0810 04:41:39.943578 (+ 236us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 04:41:39.944989 (+ 1411us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 04:41:39.944990 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 04:41:39.944990 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 04:41:39.945133 (+ 143us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 04:41:39.945386 (+ 253us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 04:41:39.945387 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 04:41:39.945387 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 04:41:39.945442 (+ 55us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 04:41:39.945746 (+ 304us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 04:41:39.945746 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 04:41:39.945784 (+ 38us) client_negotiation.cc:769] Sending connection context
0810 04:41:39.945838 (+ 54us) client_negotiation.cc:240] Negotiation successful
0810 04:41:39.945912 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":94,"thread_start_us":67,"threads_started":1}
I20260810 04:41:39.946024 3668 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:41:39.940935 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.167.66:36995 (local address 127.22.167.126:37713)
0810 04:41:39.941190 (+ 255us) server_negotiation.cc:206] Beginning negotiation
0810 04:41:39.941191 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0810 04:41:39.941198 (+ 7us) server_negotiation.cc:407] Connection header received
0810 04:41:39.941215 (+ 17us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:41:39.941215 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:41:39.941224 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:41:39.941275 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0810 04:41:39.941571 (+ 296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:41:39.942107 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:41:39.942788 (+ 681us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:41:39.942957 (+ 169us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:41:39.942998 (+ 41us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:41:39.943640 (+ 642us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 04:41:39.943641 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 04:41:39.943641 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 04:41:39.943646 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0810 04:41:39.944922 (+ 1276us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 04:41:39.945200 (+ 278us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 04:41:39.945201 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 04:41:39.945201 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 04:41:39.945272 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 04:41:39.945497 (+ 225us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 04:41:39.945498 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 04:41:39.945498 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 04:41:39.945677 (+ 179us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 04:41:39.945744 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0810 04:41:39.945878 (+ 134us) server_negotiation.cc:299] Negotiation successful
0810 04:41:39.945944 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":233}
WARNING: no policy specified for oryx/127.22.167.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.22.167.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.22.167.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786336863388938-23197-0/krb5kdc/oryx_127.22.167.67.keytab.
Entry for principal oryx/127.22.167.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786336863388938-23197-0/krb5kdc/oryx_127.22.167.67.keytab.
I20260810 04:41:39.969442 23197 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskemv_AM/build/release/bin/kudu
/tmp/dist-test-taskemv_AM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786336863388938-23197-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786336863388938-23197-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786336863388938-23197-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786336863388938-23197-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.167.67:0
--local_ip_for_outbound_sockets=127.22.167.67
--webserver_interface=127.22.167.67
--webserver_port=0
--tserver_master_addrs=127.22.167.126:37713
--keytab_file=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786336863388938-23197-0/krb5kdc/oryx_127.22.167.67.keytab
--principal=oryx/127.22.167.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.167.84:43155
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786336863388938-23197-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786336863388938-23197-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786336863388938-23197-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 04:41:40.049302 4148 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 04:41:40.049515 4148 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 04:41:40.049548 4148 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 04:41:40.050753 4148 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 04:41:40.050798 4148 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 04:41:40.050824 4148 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260810 04:41:40.050848 4148 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.167.67
I20260810 04:41:40.052345 4148 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.167.84:43155
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786336863388938-23197-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786336863388938-23197-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786336863388938-23197-0/krb5kdc/oryx_127.22.167.67.keytab
--principal=oryx/127.22.167.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.167.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786336863388938-23197-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.22.167.67
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.22.167.126:37713
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4148
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.167.67
--log_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786336863388938-23197-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 03:54:01 UTC on ccycloud.quasar-icijmt.root.comops.site
build id 4577009
I20260810 04:41:40.052587 4148 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 04:41:40.052810 4148 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 04:41:40.057552 4158 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 04:41:40.057567 4161 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 04:41:40.057614 4148 server_base.cc:1034] running on GCE node
W20260810 04:41:40.057526 4157 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 04:41:40.057904 4148 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 04:41:40.058174 4148 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 04:41:40.059319 4148 hybrid_clock.cc:648] HybridClock initialized: now 1786336900059302 us; error 27 us; skew 500 ppm
Aug 10 04:41:40 dist-test-slave-bqcl krb5kdc[3517](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786336900, etypes {rep=17 tkt=17 ses=17}, oryx/127.22.167.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 04:41:40.061740 4148 init.cc:377] Logged in from keytab as oryx/127.22.167.67@KRBTEST.COM (short username oryx)
I20260810 04:41:40.062211 4148 webserver.cc:460] Webserver started at http://127.22.167.67:34227/ using document root <none> and password file <none>
I20260810 04:41:40.062433 4148 fs_manager.cc:359] Metadata directory not provided
I20260810 04:41:40.062482 4148 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 04:41:40.062610 4148 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 04:41:40.063609 4148 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786336863388938-23197-0/minicluster-data/ts-2/data/instance:
uuid: "5fc86b14002f4fd3ae81df14a166c562"
format_stamp: "Formatted at 2026-08-10 04:41:40 on dist-test-slave-bqcl"
server_key: "e958e27aead906aa3790739af759840f"
server_key_iv: "c4496692ad22fc100688622fe22021cb"
server_key_version: "encryptionkey@0"
I20260810 04:41:40.063938 4148 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786336863388938-23197-0/minicluster-data/ts-2/wal/instance:
uuid: "5fc86b14002f4fd3ae81df14a166c562"
format_stamp: "Formatted at 2026-08-10 04:41:40 on dist-test-slave-bqcl"
server_key: "e958e27aead906aa3790739af759840f"
server_key_iv: "c4496692ad22fc100688622fe22021cb"
server_key_version: "encryptionkey@0"
I20260810 04:41:40.065153 4148 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260810 04:41:40.065809 4175 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 04:41:40.065961 4148 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260810 04:41:40.066043 4148 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786336863388938-23197-0/minicluster-data/ts-2/data,/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786336863388938-23197-0/minicluster-data/ts-2/wal
uuid: "5fc86b14002f4fd3ae81df14a166c562"
format_stamp: "Formatted at 2026-08-10 04:41:40 on dist-test-slave-bqcl"
server_key: "e958e27aead906aa3790739af759840f"
server_key_iv: "c4496692ad22fc100688622fe22021cb"
server_key_version: "encryptionkey@0"
I20260810 04:41:40.066111 4148 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786336863388938-23197-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786336863388938-23197-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786336863388938-23197-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 04:41:40.075284 4148 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 04:41:40.076858 4148 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 04:41:40.076987 4148 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 04:41:40.077481 4148 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 04:41:40.077521 4148 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 04:41:40.077549 4148 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 04:41:40.077567 4148 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 10 04:41:40 dist-test-slave-bqcl krb5kdc[3517](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786336900, etypes {rep=17 tkt=17 ses=17}, oryx/127.22.167.67@KRBTEST.COM for oryx/127.22.167.126@KRBTEST.COM
I20260810 04:41:40.084636 4148 rpc_server.cc:307] RPC server started. Bound to: 127.22.167.67:33991
I20260810 04:41:40.084713 4373 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.167.67:33991 every 8 connection(s)
I20260810 04:41:40.084983 4148 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786336863388938-23197-0/minicluster-data/ts-2/data/info.pb
I20260810 04:41:40.087044 3668 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:41:40.077839 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.167.67:33969 (local address 127.22.167.126:37713)
0810 04:41:40.077901 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0810 04:41:40.077903 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 04:41:40.078097 (+ 194us) server_negotiation.cc:407] Connection header received
0810 04:41:40.078744 (+ 647us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:41:40.078746 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:41:40.078756 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:41:40.078798 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0810 04:41:40.079625 (+ 827us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:41:40.080270 (+ 645us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:41:40.081726 (+ 1456us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:41:40.081985 (+ 259us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:41:40.082037 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:41:40.084394 (+ 2357us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 04:41:40.084396 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 04:41:40.084397 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 04:41:40.084405 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0810 04:41:40.085962 (+ 1557us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 04:41:40.086361 (+ 399us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 04:41:40.086361 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 04:41:40.086362 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 04:41:40.086411 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 04:41:40.086584 (+ 173us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 04:41:40.086585 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 04:41:40.086585 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 04:41:40.086677 (+ 92us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 04:41:40.086722 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0810 04:41:40.086913 (+ 191us) server_negotiation.cc:299] Negotiation successful
0810 04:41:40.086981 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260810 04:41:40.087182 4192 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:41:40.077810 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.126:37713 (local address 127.22.167.67:33969)
0810 04:41:40.077962 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0810 04:41:40.078010 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0810 04:41:40.078633 (+ 623us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 04:41:40.078876 (+ 243us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 04:41:40.078881 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 04:41:40.079230 (+ 349us) client_negotiation.cc:189] Negotiated authn=SASL
0810 04:41:40.079508 (+ 278us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:41:40.079515 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:41:40.080343 (+ 828us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:41:40.080344 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:41:40.081627 (+ 1283us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:41:40.081630 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:41:40.082076 (+ 446us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:41:40.082076 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:41:40.082259 (+ 183us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:41:40.082769 (+ 510us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 04:41:40.082779 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0810 04:41:40.084306 (+ 1527us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 04:41:40.086031 (+ 1725us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 04:41:40.086034 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 04:41:40.086036 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 04:41:40.086302 (+ 266us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 04:41:40.086456 (+ 154us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 04:41:40.086457 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 04:41:40.086457 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 04:41:40.086549 (+ 92us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 04:41:40.086717 (+ 168us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 04:41:40.086720 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 04:41:40.086826 (+ 106us) client_negotiation.cc:769] Sending connection context
0810 04:41:40.086893 (+ 67us) client_negotiation.cc:240] Negotiation successful
0810 04:41:40.087009 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":74,"threads_started":1}
I20260810 04:41:40.087613 4375 heartbeater.cc:347] Connected to a master server at 127.22.167.126:37713
I20260810 04:41:40.087690 4375 heartbeater.cc:464] Registering TS with master...
I20260810 04:41:40.087868 4375 heartbeater.cc:511] Master 127.22.167.126:37713 requested a full tablet report, sending...
I20260810 04:41:40.088229 3597 ts_manager.cc:194] Registered new tserver with Master: 5fc86b14002f4fd3ae81df14a166c562 (127.22.167.67:33991)
I20260810 04:41:40.088630 3597 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.22.167.67@KRBTEST.COM'} at 127.22.167.67:33969
I20260810 04:41:40.093348 3668 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:41:40.088620 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.167.67:38827 (local address 127.22.167.126:37713)
0810 04:41:40.088662 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0810 04:41:40.088664 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 04:41:40.088775 (+ 111us) server_negotiation.cc:407] Connection header received
0810 04:41:40.088829 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:41:40.088830 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:41:40.088840 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:41:40.088897 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0810 04:41:40.089240 (+ 343us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:41:40.089781 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:41:40.090476 (+ 695us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:41:40.090659 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:41:40.090714 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:41:40.091392 (+ 678us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 04:41:40.091393 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 04:41:40.091394 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 04:41:40.091399 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0810 04:41:40.092481 (+ 1082us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 04:41:40.092783 (+ 302us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 04:41:40.092783 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 04:41:40.092784 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 04:41:40.092831 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 04:41:40.092990 (+ 159us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 04:41:40.092990 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 04:41:40.092990 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 04:41:40.093081 (+ 91us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 04:41:40.093140 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0810 04:41:40.093236 (+ 96us) server_negotiation.cc:299] Negotiation successful
0810 04:41:40.093292 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260810 04:41:40.093354 4384 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:41:40.088547 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.126:37713 (local address 127.22.167.67:38827)
0810 04:41:40.088715 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0810 04:41:40.088726 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0810 04:41:40.088786 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 04:41:40.088901 (+ 115us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 04:41:40.088901 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 04:41:40.089086 (+ 185us) client_negotiation.cc:189] Negotiated authn=SASL
0810 04:41:40.089158 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:41:40.089160 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:41:40.089859 (+ 699us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:41:40.089859 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:41:40.090412 (+ 553us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:41:40.090414 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:41:40.090719 (+ 305us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:41:40.090720 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:41:40.090770 (+ 50us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:41:40.091070 (+ 300us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 04:41:40.091073 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0810 04:41:40.091329 (+ 256us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 04:41:40.092563 (+ 1234us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 04:41:40.092564 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 04:41:40.092564 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 04:41:40.092728 (+ 164us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 04:41:40.092900 (+ 172us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 04:41:40.092900 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 04:41:40.092901 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 04:41:40.092953 (+ 52us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 04:41:40.093130 (+ 177us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 04:41:40.093130 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 04:41:40.093162 (+ 32us) client_negotiation.cc:769] Sending connection context
0810 04:41:40.093216 (+ 54us) client_negotiation.cc:240] Negotiation successful
0810 04:41:40.093288 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":71,"threads_started":1}
I20260810 04:41:40.093787 23197 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskemv_AM/build/release/bin/kudu as pid 4148
I20260810 04:41:40.093933 23197 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786336863388938-23197-0/minicluster-data/ts-2/wal/instance
I20260810 04:41:40.094206 23197 external_mini_cluster.cc:1428] Setting key c372c850c0f32c801dba59b0dd73ae25
I20260810 04:41:40.095053 23197 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 10 04:41:40 dist-test-slave-bqcl krb5kdc[3517](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786336900, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 10 04:41:40 dist-test-slave-bqcl krb5kdc[3517](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.22.167.126@KRBTEST.COM, Server not found in Kerberos database
Aug 10 04:41:40 dist-test-slave-bqcl krb5kdc[3517](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.22.167.126@KRBTEST.COM, Server not found in Kerberos database
W20260810 04:41:40.115818 3668 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 04:41:40.110958 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52948 (local address 127.22.167.126:37713)
0810 04:41:40.111012 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0810 04:41:40.111014 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 04:41:40.111024 (+ 10us) server_negotiation.cc:407] Connection header received
0810 04:41:40.111047 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:41:40.111048 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:41:40.111057 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:41:40.111105 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0810 04:41:40.111623 (+ 518us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:41:40.112105 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:41:40.112859 (+ 754us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:41:40.113043 (+ 184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:41:40.113096 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:41:40.115742 (+ 2646us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:52948: BlockingRecv error: recv got EOF from 127.0.0.1:52948 (error 108)
Metrics: {"server-negotiator.queue_time_us":30}
Aug 10 04:41:40 dist-test-slave-bqcl krb5kdc[3517](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786336900, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for oryx/127.22.167.126@KRBTEST.COM
I20260810 04:41:40.122467 3668 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:41:40.116468 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52958 (local address 127.22.167.126:37713)
0810 04:41:40.116516 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0810 04:41:40.116518 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 04:41:40.116549 (+ 31us) server_negotiation.cc:407] Connection header received
0810 04:41:40.116595 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:41:40.116596 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:41:40.116602 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:41:40.116647 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0810 04:41:40.117045 (+ 398us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:41:40.117492 (+ 447us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:41:40.118415 (+ 923us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:41:40.118607 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:41:40.118682 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:41:40.120528 (+ 1846us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 04:41:40.120529 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 04:41:40.120530 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 04:41:40.120535 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0810 04:41:40.121691 (+ 1156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 04:41:40.121929 (+ 238us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 04:41:40.121930 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 04:41:40.121930 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 04:41:40.121973 (+ 43us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 04:41:40.122086 (+ 113us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 04:41:40.122086 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 04:41:40.122087 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 04:41:40.122204 (+ 117us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 04:41:40.122276 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0810 04:41:40.122354 (+ 78us) server_negotiation.cc:299] Negotiation successful
0810 04:41:40.122410 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260810 04:41:40.123826 3597 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:52958:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 04:41:40.125377 3597 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 04:41:40.131604 4406 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:41:40.127468 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.65:40941 (local address 127.0.0.1:50476)
0810 04:41:40.127775 (+ 307us) negotiation.cc:107] Waiting for socket to connect
0810 04:41:40.127819 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0810 04:41:40.127889 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 04:41:40.128366 (+ 477us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 04:41:40.128367 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 04:41:40.128369 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 04:41:40.128403 (+ 34us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:41:40.128404 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:41:40.129612 (+ 1208us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:41:40.129613 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:41:40.130678 (+ 1065us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:41:40.130680 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:41:40.131324 (+ 644us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:41:40.131325 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:41:40.131403 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:41:40.131417 (+ 14us) client_negotiation.cc:769] Sending connection context
0810 04:41:40.131482 (+ 65us) client_negotiation.cc:240] Negotiation successful
0810 04:41:40.131533 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":283,"mutex_wait_us":30,"thread_start_us":56,"threads_started":1}
I20260810 04:41:40.131641 4412 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:41:40.127568 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50476 (local address 127.22.167.65:40941)
0810 04:41:40.127782 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0810 04:41:40.127787 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 04:41:40.127971 (+ 184us) server_negotiation.cc:407] Connection header received
0810 04:41:40.128017 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:41:40.128019 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:41:40.128035 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:41:40.128108 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 04:41:40.128477 (+ 369us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:41:40.129537 (+ 1060us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:41:40.130748 (+ 1211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:41:40.131250 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:41:40.131312 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:41:40.131394 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0810 04:41:40.131507 (+ 113us) server_negotiation.cc:299] Negotiation successful
0810 04:41:40.131551 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":182,"thread_start_us":82,"threads_started":1}
I20260810 04:41:40.131824 4413 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:41:40.127593 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56580 (local address 127.22.167.67:33991)
0810 04:41:40.127794 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0810 04:41:40.127799 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 04:41:40.128112 (+ 313us) server_negotiation.cc:407] Connection header received
0810 04:41:40.128156 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:41:40.128159 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:41:40.128173 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:41:40.128242 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 04:41:40.128416 (+ 174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:41:40.129503 (+ 1087us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:41:40.130564 (+ 1061us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:41:40.131298 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:41:40.131367 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:41:40.131439 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0810 04:41:40.131685 (+ 246us) server_negotiation.cc:299] Negotiation successful
0810 04:41:40.131736 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":171,"thread_start_us":95,"threads_started":1}
I20260810 04:41:40.131721 4416 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:41:40.127662 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.66:34655 (local address 127.0.0.1:50370)
0810 04:41:40.127984 (+ 322us) negotiation.cc:107] Waiting for socket to connect
0810 04:41:40.127992 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0810 04:41:40.128037 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 04:41:40.128224 (+ 187us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 04:41:40.128227 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 04:41:40.128230 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 04:41:40.128300 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:41:40.128302 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:41:40.129277 (+ 975us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:41:40.129278 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:41:40.130462 (+ 1184us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:41:40.130465 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:41:40.131476 (+ 1011us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:41:40.131477 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:41:40.131547 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:41:40.131551 (+ 4us) client_negotiation.cc:769] Sending connection context
0810 04:41:40.131598 (+ 47us) client_negotiation.cc:240] Negotiation successful
0810 04:41:40.131641 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":308,"mutex_wait_us":3,"thread_start_us":56,"threads_started":1}
I20260810 04:41:40.131938 4421 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:41:40.127760 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50370 (local address 127.22.167.66:34655)
0810 04:41:40.127896 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0810 04:41:40.127900 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 04:41:40.128040 (+ 140us) server_negotiation.cc:407] Connection header received
0810 04:41:40.128104 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:41:40.128105 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:41:40.128117 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:41:40.128193 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 04:41:40.128494 (+ 301us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:41:40.129194 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:41:40.130551 (+ 1357us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:41:40.131299 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:41:40.131367 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:41:40.131439 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0810 04:41:40.131792 (+ 353us) server_negotiation.cc:299] Negotiation successful
0810 04:41:40.131839 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":109,"thread_start_us":79,"threads_started":1}
I20260810 04:41:40.132087 4411 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:41:40.127485 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.67:33991 (local address 127.0.0.1:56580)
0810 04:41:40.127826 (+ 341us) negotiation.cc:107] Waiting for socket to connect
0810 04:41:40.127834 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0810 04:41:40.127889 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 04:41:40.128251 (+ 362us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 04:41:40.128252 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 04:41:40.128254 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 04:41:40.128301 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:41:40.128303 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:41:40.129602 (+ 1299us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:41:40.129603 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:41:40.130481 (+ 878us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:41:40.130484 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:41:40.131479 (+ 995us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:41:40.131479 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:41:40.131556 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:41:40.131559 (+ 3us) client_negotiation.cc:769] Sending connection context
0810 04:41:40.131598 (+ 39us) client_negotiation.cc:240] Negotiation successful
0810 04:41:40.131642 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":324,"mutex_wait_us":10,"thread_start_us":74,"threads_started":1}
I20260810 04:41:40.132647 4031 tablet_service.cc:1467] Processing CreateTablet for tablet c8f88edaa4234f1db6f47968f1cfe707 (DEFAULT_TABLE table=test-table [id=b1a86c7cb541492cb22af120dca04630]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 04:41:40.132691 4267 tablet_service.cc:1467] Processing CreateTablet for tablet c8f88edaa4234f1db6f47968f1cfe707 (DEFAULT_TABLE table=test-table [id=b1a86c7cb541492cb22af120dca04630]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 04:41:40.132642 3815 tablet_service.cc:1467] Processing CreateTablet for tablet c8f88edaa4234f1db6f47968f1cfe707 (DEFAULT_TABLE table=test-table [id=b1a86c7cb541492cb22af120dca04630]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 04:41:40.132988 4031 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c8f88edaa4234f1db6f47968f1cfe707. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 04:41:40.133002 3815 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c8f88edaa4234f1db6f47968f1cfe707. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 04:41:40.133002 4267 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c8f88edaa4234f1db6f47968f1cfe707. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 04:41:40.136686 4425 tablet_bootstrap.cc:492] T c8f88edaa4234f1db6f47968f1cfe707 P 5fc86b14002f4fd3ae81df14a166c562: Bootstrap starting.
I20260810 04:41:40.136822 4426 tablet_bootstrap.cc:492] T c8f88edaa4234f1db6f47968f1cfe707 P 84e7718d636e41efbd3a4b32db54e3f8: Bootstrap starting.
I20260810 04:41:40.136907 4422 tablet_bootstrap.cc:492] T c8f88edaa4234f1db6f47968f1cfe707 P 603becb479b7443f854a91ab3530293a: Bootstrap starting.
I20260810 04:41:40.137341 4425 tablet_bootstrap.cc:654] T c8f88edaa4234f1db6f47968f1cfe707 P 5fc86b14002f4fd3ae81df14a166c562: Neither blocks nor log segments found. Creating new log.
I20260810 04:41:40.137341 4426 tablet_bootstrap.cc:654] T c8f88edaa4234f1db6f47968f1cfe707 P 84e7718d636e41efbd3a4b32db54e3f8: Neither blocks nor log segments found. Creating new log.
I20260810 04:41:40.137405 4422 tablet_bootstrap.cc:654] T c8f88edaa4234f1db6f47968f1cfe707 P 603becb479b7443f854a91ab3530293a: Neither blocks nor log segments found. Creating new log.
I20260810 04:41:40.137669 4425 log.cc:824] T c8f88edaa4234f1db6f47968f1cfe707 P 5fc86b14002f4fd3ae81df14a166c562: Log is configured to *not* fsync() on all Append() calls
I20260810 04:41:40.137686 4422 log.cc:824] T c8f88edaa4234f1db6f47968f1cfe707 P 603becb479b7443f854a91ab3530293a: Log is configured to *not* fsync() on all Append() calls
I20260810 04:41:40.137669 4426 log.cc:824] T c8f88edaa4234f1db6f47968f1cfe707 P 84e7718d636e41efbd3a4b32db54e3f8: Log is configured to *not* fsync() on all Append() calls
I20260810 04:41:40.138455 4425 tablet_bootstrap.cc:492] T c8f88edaa4234f1db6f47968f1cfe707 P 5fc86b14002f4fd3ae81df14a166c562: No bootstrap required, opened a new log
I20260810 04:41:40.138521 4425 ts_tablet_manager.cc:1397] T c8f88edaa4234f1db6f47968f1cfe707 P 5fc86b14002f4fd3ae81df14a166c562: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260810 04:41:40.138600 4426 tablet_bootstrap.cc:492] T c8f88edaa4234f1db6f47968f1cfe707 P 84e7718d636e41efbd3a4b32db54e3f8: No bootstrap required, opened a new log
I20260810 04:41:40.138701 4426 ts_tablet_manager.cc:1397] T c8f88edaa4234f1db6f47968f1cfe707 P 84e7718d636e41efbd3a4b32db54e3f8: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260810 04:41:40.138732 4422 tablet_bootstrap.cc:492] T c8f88edaa4234f1db6f47968f1cfe707 P 603becb479b7443f854a91ab3530293a: No bootstrap required, opened a new log
I20260810 04:41:40.138784 4422 ts_tablet_manager.cc:1397] T c8f88edaa4234f1db6f47968f1cfe707 P 603becb479b7443f854a91ab3530293a: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260810 04:41:40.140095 4425 raft_consensus.cc:348] T c8f88edaa4234f1db6f47968f1cfe707 P 5fc86b14002f4fd3ae81df14a166c562 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84e7718d636e41efbd3a4b32db54e3f8" member_type: VOTER last_known_addr { host: "127.22.167.65" port: 40941 } } peers { permanent_uuid: "5fc86b14002f4fd3ae81df14a166c562" member_type: VOTER last_known_addr { host: "127.22.167.67" port: 33991 } } peers { permanent_uuid: "603becb479b7443f854a91ab3530293a" member_type: VOTER last_known_addr { host: "127.22.167.66" port: 34655 } }
I20260810 04:41:40.140240 4425 raft_consensus.cc:374] T c8f88edaa4234f1db6f47968f1cfe707 P 5fc86b14002f4fd3ae81df14a166c562 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 04:41:40.140281 4425 raft_consensus.cc:729] T c8f88edaa4234f1db6f47968f1cfe707 P 5fc86b14002f4fd3ae81df14a166c562 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5fc86b14002f4fd3ae81df14a166c562, State: Initialized, Role: FOLLOWER
I20260810 04:41:40.140403 4425 consensus_queue.cc:260] T c8f88edaa4234f1db6f47968f1cfe707 P 5fc86b14002f4fd3ae81df14a166c562 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84e7718d636e41efbd3a4b32db54e3f8" member_type: VOTER last_known_addr { host: "127.22.167.65" port: 40941 } } peers { permanent_uuid: "5fc86b14002f4fd3ae81df14a166c562" member_type: VOTER last_known_addr { host: "127.22.167.67" port: 33991 } } peers { permanent_uuid: "603becb479b7443f854a91ab3530293a" member_type: VOTER last_known_addr { host: "127.22.167.66" port: 34655 } }
I20260810 04:41:40.140638 4425 ts_tablet_manager.cc:1428] T c8f88edaa4234f1db6f47968f1cfe707 P 5fc86b14002f4fd3ae81df14a166c562: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260810 04:41:40.140744 4375 heartbeater.cc:503] Master 127.22.167.126:37713 was elected leader, sending a full tablet report...
I20260810 04:41:40.140705 4426 raft_consensus.cc:348] T c8f88edaa4234f1db6f47968f1cfe707 P 84e7718d636e41efbd3a4b32db54e3f8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84e7718d636e41efbd3a4b32db54e3f8" member_type: VOTER last_known_addr { host: "127.22.167.65" port: 40941 } } peers { permanent_uuid: "5fc86b14002f4fd3ae81df14a166c562" member_type: VOTER last_known_addr { host: "127.22.167.67" port: 33991 } } peers { permanent_uuid: "603becb479b7443f854a91ab3530293a" member_type: VOTER last_known_addr { host: "127.22.167.66" port: 34655 } }
I20260810 04:41:40.140738 4422 raft_consensus.cc:348] T c8f88edaa4234f1db6f47968f1cfe707 P 603becb479b7443f854a91ab3530293a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84e7718d636e41efbd3a4b32db54e3f8" member_type: VOTER last_known_addr { host: "127.22.167.65" port: 40941 } } peers { permanent_uuid: "5fc86b14002f4fd3ae81df14a166c562" member_type: VOTER last_known_addr { host: "127.22.167.67" port: 33991 } } peers { permanent_uuid: "603becb479b7443f854a91ab3530293a" member_type: VOTER last_known_addr { host: "127.22.167.66" port: 34655 } }
I20260810 04:41:40.140869 4426 raft_consensus.cc:374] T c8f88edaa4234f1db6f47968f1cfe707 P 84e7718d636e41efbd3a4b32db54e3f8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 04:41:40.140902 4422 raft_consensus.cc:374] T c8f88edaa4234f1db6f47968f1cfe707 P 603becb479b7443f854a91ab3530293a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 04:41:40.140908 4426 raft_consensus.cc:729] T c8f88edaa4234f1db6f47968f1cfe707 P 84e7718d636e41efbd3a4b32db54e3f8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84e7718d636e41efbd3a4b32db54e3f8, State: Initialized, Role: FOLLOWER
I20260810 04:41:40.141011 4426 consensus_queue.cc:260] T c8f88edaa4234f1db6f47968f1cfe707 P 84e7718d636e41efbd3a4b32db54e3f8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84e7718d636e41efbd3a4b32db54e3f8" member_type: VOTER last_known_addr { host: "127.22.167.65" port: 40941 } } peers { permanent_uuid: "5fc86b14002f4fd3ae81df14a166c562" member_type: VOTER last_known_addr { host: "127.22.167.67" port: 33991 } } peers { permanent_uuid: "603becb479b7443f854a91ab3530293a" member_type: VOTER last_known_addr { host: "127.22.167.66" port: 34655 } }
I20260810 04:41:40.140941 4422 raft_consensus.cc:729] T c8f88edaa4234f1db6f47968f1cfe707 P 603becb479b7443f854a91ab3530293a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 603becb479b7443f854a91ab3530293a, State: Initialized, Role: FOLLOWER
I20260810 04:41:40.141213 4422 consensus_queue.cc:260] T c8f88edaa4234f1db6f47968f1cfe707 P 603becb479b7443f854a91ab3530293a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84e7718d636e41efbd3a4b32db54e3f8" member_type: VOTER last_known_addr { host: "127.22.167.65" port: 40941 } } peers { permanent_uuid: "5fc86b14002f4fd3ae81df14a166c562" member_type: VOTER last_known_addr { host: "127.22.167.67" port: 33991 } } peers { permanent_uuid: "603becb479b7443f854a91ab3530293a" member_type: VOTER last_known_addr { host: "127.22.167.66" port: 34655 } }
I20260810 04:41:40.141275 4426 ts_tablet_manager.cc:1428] T c8f88edaa4234f1db6f47968f1cfe707 P 84e7718d636e41efbd3a4b32db54e3f8: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260810 04:41:40.141404 3915 heartbeater.cc:503] Master 127.22.167.126:37713 was elected leader, sending a full tablet report...
I20260810 04:41:40.141461 4422 ts_tablet_manager.cc:1428] T c8f88edaa4234f1db6f47968f1cfe707 P 603becb479b7443f854a91ab3530293a: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260810 04:41:40.141651 4137 heartbeater.cc:503] Master 127.22.167.126:37713 was elected leader, sending a full tablet report...
W20260810 04:41:40.187587 4138 tablet.cc:2369] T c8f88edaa4234f1db6f47968f1cfe707 P 603becb479b7443f854a91ab3530293a: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260810 04:41:40.287694 3919 tablet.cc:2369] T c8f88edaa4234f1db6f47968f1cfe707 P 84e7718d636e41efbd3a4b32db54e3f8: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260810 04:41:40.335623 4376 tablet.cc:2369] T c8f88edaa4234f1db6f47968f1cfe707 P 5fc86b14002f4fd3ae81df14a166c562: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 04:41:40.392719 4434 raft_consensus.cc:482] T c8f88edaa4234f1db6f47968f1cfe707 P 84e7718d636e41efbd3a4b32db54e3f8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 04:41:40.392870 4434 raft_consensus.cc:504] T c8f88edaa4234f1db6f47968f1cfe707 P 84e7718d636e41efbd3a4b32db54e3f8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84e7718d636e41efbd3a4b32db54e3f8" member_type: VOTER last_known_addr { host: "127.22.167.65" port: 40941 } } peers { permanent_uuid: "5fc86b14002f4fd3ae81df14a166c562" member_type: VOTER last_known_addr { host: "127.22.167.67" port: 33991 } } peers { permanent_uuid: "603becb479b7443f854a91ab3530293a" member_type: VOTER last_known_addr { host: "127.22.167.66" port: 34655 } }
I20260810 04:41:40.393226 4434 leader_election.cc:288] T c8f88edaa4234f1db6f47968f1cfe707 P 84e7718d636e41efbd3a4b32db54e3f8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5fc86b14002f4fd3ae81df14a166c562 (127.22.167.67:33991), 603becb479b7443f854a91ab3530293a (127.22.167.66:34655)
I20260810 04:41:40.396063 4413 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:41:40.393500 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.167.65:49947 (local address 127.22.167.67:33991)
0810 04:41:40.393551 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0810 04:41:40.393553 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 04:41:40.393635 (+ 82us) server_negotiation.cc:407] Connection header received
0810 04:41:40.393748 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:41:40.393749 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:41:40.393763 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:41:40.393801 (+ 38us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 04:41:40.393953 (+ 152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:41:40.394469 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:41:40.395412 (+ 943us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:41:40.395756 (+ 344us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:41:40.395816 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:41:40.395825 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0810 04:41:40.395960 (+ 135us) server_negotiation.cc:299] Negotiation successful
0810 04:41:40.395998 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260810 04:41:40.396066 4436 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:41:40.393393 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.67:33991 (local address 127.22.167.65:49947)
0810 04:41:40.393560 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0810 04:41:40.393571 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0810 04:41:40.393672 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 04:41:40.393813 (+ 141us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 04:41:40.393814 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 04:41:40.393816 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 04:41:40.393892 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:41:40.393894 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:41:40.394545 (+ 651us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:41:40.394545 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:41:40.395326 (+ 781us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:41:40.395328 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:41:40.395830 (+ 502us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:41:40.395831 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:41:40.395897 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:41:40.395900 (+ 3us) client_negotiation.cc:769] Sending connection context
0810 04:41:40.395945 (+ 45us) client_negotiation.cc:240] Negotiation successful
0810 04:41:40.395987 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":107,"threads_started":1}
I20260810 04:41:40.396121 4421 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:41:40.393715 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.167.65:49211 (local address 127.22.167.66:34655)
0810 04:41:40.393771 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0810 04:41:40.393772 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0810 04:41:40.393871 (+ 99us) server_negotiation.cc:407] Connection header received
0810 04:41:40.393933 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:41:40.393934 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:41:40.393943 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:41:40.393986 (+ 43us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 04:41:40.394119 (+ 133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:41:40.394594 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:41:40.395455 (+ 861us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:41:40.395818 (+ 363us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:41:40.395879 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:41:40.395886 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0810 04:41:40.396024 (+ 138us) server_negotiation.cc:299] Negotiation successful
0810 04:41:40.396058 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260810 04:41:40.396173 4437 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:41:40.393614 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.66:34655 (local address 127.22.167.65:49211)
0810 04:41:40.393723 (+ 109us) negotiation.cc:107] Waiting for socket to connect
0810 04:41:40.393734 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0810 04:41:40.393879 (+ 145us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 04:41:40.393999 (+ 120us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 04:41:40.394000 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 04:41:40.394002 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 04:41:40.394062 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:41:40.394064 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:41:40.394672 (+ 608us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:41:40.394673 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:41:40.395379 (+ 706us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:41:40.395381 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:41:40.395893 (+ 512us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:41:40.395894 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:41:40.395965 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:41:40.395968 (+ 3us) client_negotiation.cc:769] Sending connection context
0810 04:41:40.396013 (+ 45us) client_negotiation.cc:240] Negotiation successful
0810 04:41:40.396051 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":94,"thread_start_us":62,"threads_started":1}
I20260810 04:41:40.396525 4055 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c8f88edaa4234f1db6f47968f1cfe707" candidate_uuid: "84e7718d636e41efbd3a4b32db54e3f8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "603becb479b7443f854a91ab3530293a" is_pre_election: true
I20260810 04:41:40.396548 4300 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c8f88edaa4234f1db6f47968f1cfe707" candidate_uuid: "84e7718d636e41efbd3a4b32db54e3f8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5fc86b14002f4fd3ae81df14a166c562" is_pre_election: true
I20260810 04:41:40.396682 4300 raft_consensus.cc:2454] T c8f88edaa4234f1db6f47968f1cfe707 P 5fc86b14002f4fd3ae81df14a166c562 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84e7718d636e41efbd3a4b32db54e3f8 in term 0.
I20260810 04:41:40.396678 4055 raft_consensus.cc:2454] T c8f88edaa4234f1db6f47968f1cfe707 P 603becb479b7443f854a91ab3530293a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84e7718d636e41efbd3a4b32db54e3f8 in term 0.
I20260810 04:41:40.396904 3731 leader_election.cc:302] T c8f88edaa4234f1db6f47968f1cfe707 P 84e7718d636e41efbd3a4b32db54e3f8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 603becb479b7443f854a91ab3530293a, 84e7718d636e41efbd3a4b32db54e3f8; no voters:
I20260810 04:41:40.397027 4434 raft_consensus.cc:2781] T c8f88edaa4234f1db6f47968f1cfe707 P 84e7718d636e41efbd3a4b32db54e3f8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 04:41:40.397085 4434 raft_consensus.cc:482] T c8f88edaa4234f1db6f47968f1cfe707 P 84e7718d636e41efbd3a4b32db54e3f8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 04:41:40.397114 4434 raft_consensus.cc:3037] T c8f88edaa4234f1db6f47968f1cfe707 P 84e7718d636e41efbd3a4b32db54e3f8 [term 0 FOLLOWER]: Advancing to term 1
I20260810 04:41:40.397769 4434 raft_consensus.cc:504] T c8f88edaa4234f1db6f47968f1cfe707 P 84e7718d636e41efbd3a4b32db54e3f8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84e7718d636e41efbd3a4b32db54e3f8" member_type: VOTER last_known_addr { host: "127.22.167.65" port: 40941 } } peers { permanent_uuid: "5fc86b14002f4fd3ae81df14a166c562" member_type: VOTER last_known_addr { host: "127.22.167.67" port: 33991 } } peers { permanent_uuid: "603becb479b7443f854a91ab3530293a" member_type: VOTER last_known_addr { host: "127.22.167.66" port: 34655 } }
I20260810 04:41:40.397868 4434 leader_election.cc:288] T c8f88edaa4234f1db6f47968f1cfe707 P 84e7718d636e41efbd3a4b32db54e3f8 [CANDIDATE]: Term 1 election: Requested vote from peers 5fc86b14002f4fd3ae81df14a166c562 (127.22.167.67:33991), 603becb479b7443f854a91ab3530293a (127.22.167.66:34655)
I20260810 04:41:40.398046 4300 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c8f88edaa4234f1db6f47968f1cfe707" candidate_uuid: "84e7718d636e41efbd3a4b32db54e3f8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5fc86b14002f4fd3ae81df14a166c562"
I20260810 04:41:40.398073 4055 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c8f88edaa4234f1db6f47968f1cfe707" candidate_uuid: "84e7718d636e41efbd3a4b32db54e3f8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "603becb479b7443f854a91ab3530293a"
I20260810 04:41:40.398116 4300 raft_consensus.cc:3037] T c8f88edaa4234f1db6f47968f1cfe707 P 5fc86b14002f4fd3ae81df14a166c562 [term 0 FOLLOWER]: Advancing to term 1
I20260810 04:41:40.398191 4055 raft_consensus.cc:3037] T c8f88edaa4234f1db6f47968f1cfe707 P 603becb479b7443f854a91ab3530293a [term 0 FOLLOWER]: Advancing to term 1
I20260810 04:41:40.398815 4300 raft_consensus.cc:2454] T c8f88edaa4234f1db6f47968f1cfe707 P 5fc86b14002f4fd3ae81df14a166c562 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 84e7718d636e41efbd3a4b32db54e3f8 in term 1.
I20260810 04:41:40.398932 4055 raft_consensus.cc:2454] T c8f88edaa4234f1db6f47968f1cfe707 P 603becb479b7443f854a91ab3530293a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 84e7718d636e41efbd3a4b32db54e3f8 in term 1.
I20260810 04:41:40.398978 3731 leader_election.cc:302] T c8f88edaa4234f1db6f47968f1cfe707 P 84e7718d636e41efbd3a4b32db54e3f8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5fc86b14002f4fd3ae81df14a166c562, 84e7718d636e41efbd3a4b32db54e3f8; no voters:
I20260810 04:41:40.399081 4434 raft_consensus.cc:2781] T c8f88edaa4234f1db6f47968f1cfe707 P 84e7718d636e41efbd3a4b32db54e3f8 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 04:41:40.399219 4434 raft_consensus.cc:686] T c8f88edaa4234f1db6f47968f1cfe707 P 84e7718d636e41efbd3a4b32db54e3f8 [term 1 LEADER]: Becoming Leader. State: Replica: 84e7718d636e41efbd3a4b32db54e3f8, State: Running, Role: LEADER
I20260810 04:41:40.399317 4434 consensus_queue.cc:237] T c8f88edaa4234f1db6f47968f1cfe707 P 84e7718d636e41efbd3a4b32db54e3f8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84e7718d636e41efbd3a4b32db54e3f8" member_type: VOTER last_known_addr { host: "127.22.167.65" port: 40941 } } peers { permanent_uuid: "5fc86b14002f4fd3ae81df14a166c562" member_type: VOTER last_known_addr { host: "127.22.167.67" port: 33991 } } peers { permanent_uuid: "603becb479b7443f854a91ab3530293a" member_type: VOTER last_known_addr { host: "127.22.167.66" port: 34655 } }
I20260810 04:41:40.400066 3597 catalog_manager.cc:5495] T c8f88edaa4234f1db6f47968f1cfe707 P 84e7718d636e41efbd3a4b32db54e3f8 reported cstate change: term changed from 0 to 1, leader changed from <none> to 84e7718d636e41efbd3a4b32db54e3f8 (127.22.167.65). New cstate: current_term: 1 leader_uuid: "84e7718d636e41efbd3a4b32db54e3f8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84e7718d636e41efbd3a4b32db54e3f8" member_type: VOTER last_known_addr { host: "127.22.167.65" port: 40941 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5fc86b14002f4fd3ae81df14a166c562" member_type: VOTER last_known_addr { host: "127.22.167.67" port: 33991 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "603becb479b7443f854a91ab3530293a" member_type: VOTER last_known_addr { host: "127.22.167.66" port: 34655 } health_report { overall_health: UNKNOWN } } }
I20260810 04:41:40.410324 4412 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:41:40.408350 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50484 (local address 127.22.167.65:40941)
0810 04:41:40.408395 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0810 04:41:40.408397 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 04:41:40.408404 (+ 7us) server_negotiation.cc:407] Connection header received
0810 04:41:40.408449 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:41:40.408449 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:41:40.408464 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:41:40.408520 (+ 56us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 04:41:40.408622 (+ 102us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:41:40.409053 (+ 431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:41:40.409721 (+ 668us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:41:40.409890 (+ 169us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:41:40.409931 (+ 41us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:41:40.410031 (+ 100us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 04:41:40.410146 (+ 115us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 04:41:40.410221 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0810 04:41:40.410242 (+ 21us) server_negotiation.cc:299] Negotiation successful
0810 04:41:40.410261 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260810 04:41:40.411978 4300 raft_consensus.cc:1261] T c8f88edaa4234f1db6f47968f1cfe707 P 5fc86b14002f4fd3ae81df14a166c562 [term 1 FOLLOWER]: Refusing update from remote peer 84e7718d636e41efbd3a4b32db54e3f8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 04:41:40.411978 4055 raft_consensus.cc:1261] T c8f88edaa4234f1db6f47968f1cfe707 P 603becb479b7443f854a91ab3530293a [term 1 FOLLOWER]: Refusing update from remote peer 84e7718d636e41efbd3a4b32db54e3f8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 04:41:40.412292 4438 consensus_queue.cc:1035] T c8f88edaa4234f1db6f47968f1cfe707 P 84e7718d636e41efbd3a4b32db54e3f8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "603becb479b7443f854a91ab3530293a" member_type: VOTER last_known_addr { host: "127.22.167.66" port: 34655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 04:41:40.412412 4434 consensus_queue.cc:1035] T c8f88edaa4234f1db6f47968f1cfe707 P 84e7718d636e41efbd3a4b32db54e3f8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5fc86b14002f4fd3ae81df14a166c562" member_type: VOTER last_known_addr { host: "127.22.167.67" port: 33991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 04:41:40.414983 4450 mvcc.cc:205] Tried to move back new op lower bound from 7316835944084992000 to 7316835944036159488. Current Snapshot: MvccSnapshot[applied={T|T < 7316835944084992000}]
I20260810 04:41:40.415133 4446 mvcc.cc:205] Tried to move back new op lower bound from 7316835944084992000 to 7316835944036159488. Current Snapshot: MvccSnapshot[applied={T|T < 7316835944084992000}]
I20260810 04:41:40.417128 3596 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:52958:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 04:41:40.417214 3596 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:52958:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 04:41:40.417896 3596 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b8bbdb416f794faeb39ac8593eebe708: Sending DeleteTablet for 3 replicas of tablet c8f88edaa4234f1db6f47968f1cfe707
I20260810 04:41:40.418195 3815 tablet_service.cc:1514] Processing DeleteTablet for tablet c8f88edaa4234f1db6f47968f1cfe707 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 04:41:40 UTC) from {username='oryx', principal='oryx/127.22.167.126@KRBTEST.COM'} at 127.0.0.1:50476
I20260810 04:41:40.418251 4267 tablet_service.cc:1514] Processing DeleteTablet for tablet c8f88edaa4234f1db6f47968f1cfe707 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 04:41:40 UTC) from {username='oryx', principal='oryx/127.22.167.126@KRBTEST.COM'} at 127.0.0.1:56580
I20260810 04:41:40.418315 4031 tablet_service.cc:1514] Processing DeleteTablet for tablet c8f88edaa4234f1db6f47968f1cfe707 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 04:41:40 UTC) from {username='oryx', principal='oryx/127.22.167.126@KRBTEST.COM'} at 127.0.0.1:50370
I20260810 04:41:40.418414 4466 tablet_replica.cc:330] T c8f88edaa4234f1db6f47968f1cfe707 P 84e7718d636e41efbd3a4b32db54e3f8: stopping tablet replica
I20260810 04:41:40.418530 4466 raft_consensus.cc:2229] T c8f88edaa4234f1db6f47968f1cfe707 P 84e7718d636e41efbd3a4b32db54e3f8 [term 1 LEADER]: Raft consensus shutting down.
I20260810 04:41:40.418651 4466 raft_consensus.cc:2258] T c8f88edaa4234f1db6f47968f1cfe707 P 84e7718d636e41efbd3a4b32db54e3f8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 04:41:40.418645 3596 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:52958:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 04:41:40.418804 3596 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 04:41:40.419203 4466 ts_tablet_manager.cc:1905] T c8f88edaa4234f1db6f47968f1cfe707 P 84e7718d636e41efbd3a4b32db54e3f8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 04:41:40.420102 3815 tablet_service.cc:1467] Processing CreateTablet for tablet d9572be79ddc45e98a527612a51cdaf9 (DEFAULT_TABLE table=test-table [id=0107be0e7f5347308877d96fd1cbe0df]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 04:41:40.420147 4267 tablet_service.cc:1467] Processing CreateTablet for tablet d9572be79ddc45e98a527612a51cdaf9 (DEFAULT_TABLE table=test-table [id=0107be0e7f5347308877d96fd1cbe0df]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 04:41:40.420188 4031 tablet_service.cc:1467] Processing CreateTablet for tablet d9572be79ddc45e98a527612a51cdaf9 (DEFAULT_TABLE table=test-table [id=0107be0e7f5347308877d96fd1cbe0df]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 04:41:40.420279 4031 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d9572be79ddc45e98a527612a51cdaf9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 04:41:40.420279 4267 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d9572be79ddc45e98a527612a51cdaf9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 04:41:40.420279 3815 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d9572be79ddc45e98a527612a51cdaf9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 04:41:40.420944 4466 ts_tablet_manager.cc:1918] T c8f88edaa4234f1db6f47968f1cfe707 P 84e7718d636e41efbd3a4b32db54e3f8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 04:41:40.421006 4466 log.cc:1196] T c8f88edaa4234f1db6f47968f1cfe707 P 84e7718d636e41efbd3a4b32db54e3f8: Deleting WAL directory at /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786336863388938-23197-0/minicluster-data/ts-0/wal/wals/c8f88edaa4234f1db6f47968f1cfe707
I20260810 04:41:40.421239 4466 ts_tablet_manager.cc:1939] T c8f88edaa4234f1db6f47968f1cfe707 P 84e7718d636e41efbd3a4b32db54e3f8: Deleting consensus metadata
I20260810 04:41:40.421378 4422 tablet_bootstrap.cc:492] T d9572be79ddc45e98a527612a51cdaf9 P 603becb479b7443f854a91ab3530293a: Bootstrap starting.
I20260810 04:41:40.421515 4426 tablet_bootstrap.cc:492] T d9572be79ddc45e98a527612a51cdaf9 P 84e7718d636e41efbd3a4b32db54e3f8: Bootstrap starting.
I20260810 04:41:40.421602 3573 catalog_manager.cc:4841] TS 84e7718d636e41efbd3a4b32db54e3f8 (127.22.167.65:40941): tablet c8f88edaa4234f1db6f47968f1cfe707 (table test-table [id=b1a86c7cb541492cb22af120dca04630]) successfully deleted
I20260810 04:41:40.421929 4426 tablet_bootstrap.cc:654] T d9572be79ddc45e98a527612a51cdaf9 P 84e7718d636e41efbd3a4b32db54e3f8: Neither blocks nor log segments found. Creating new log.
I20260810 04:41:40.421967 4422 tablet_bootstrap.cc:654] T d9572be79ddc45e98a527612a51cdaf9 P 603becb479b7443f854a91ab3530293a: Neither blocks nor log segments found. Creating new log.
I20260810 04:41:40.422300 4467 tablet_replica.cc:330] T c8f88edaa4234f1db6f47968f1cfe707 P 5fc86b14002f4fd3ae81df14a166c562: stopping tablet replica
I20260810 04:41:40.422456 4467 raft_consensus.cc:2229] T c8f88edaa4234f1db6f47968f1cfe707 P 5fc86b14002f4fd3ae81df14a166c562 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 04:41:40.422616 4467 raft_consensus.cc:2258] T c8f88edaa4234f1db6f47968f1cfe707 P 5fc86b14002f4fd3ae81df14a166c562 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 04:41:40.422818 4468 tablet_replica.cc:330] T c8f88edaa4234f1db6f47968f1cfe707 P 603becb479b7443f854a91ab3530293a: stopping tablet replica
I20260810 04:41:40.422914 4422 tablet_bootstrap.cc:492] T d9572be79ddc45e98a527612a51cdaf9 P 603becb479b7443f854a91ab3530293a: No bootstrap required, opened a new log
I20260810 04:41:40.422948 4468 raft_consensus.cc:2229] T c8f88edaa4234f1db6f47968f1cfe707 P 603becb479b7443f854a91ab3530293a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 04:41:40.422959 4422 ts_tablet_manager.cc:1397] T d9572be79ddc45e98a527612a51cdaf9 P 603becb479b7443f854a91ab3530293a: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260810 04:41:40.423102 4468 raft_consensus.cc:2258] T c8f88edaa4234f1db6f47968f1cfe707 P 603becb479b7443f854a91ab3530293a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 04:41:40.423174 4422 raft_consensus.cc:348] T d9572be79ddc45e98a527612a51cdaf9 P 603becb479b7443f854a91ab3530293a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84e7718d636e41efbd3a4b32db54e3f8" member_type: VOTER last_known_addr { host: "127.22.167.65" port: 40941 } } peers { permanent_uuid: "603becb479b7443f854a91ab3530293a" member_type: VOTER last_known_addr { host: "127.22.167.66" port: 34655 } } peers { permanent_uuid: "5fc86b14002f4fd3ae81df14a166c562" member_type: VOTER last_known_addr { host: "127.22.167.67" port: 33991 } }
I20260810 04:41:40.423238 4422 raft_consensus.cc:374] T d9572be79ddc45e98a527612a51cdaf9 P 603becb479b7443f854a91ab3530293a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 04:41:40.423257 4422 raft_consensus.cc:729] T d9572be79ddc45e98a527612a51cdaf9 P 603becb479b7443f854a91ab3530293a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 603becb479b7443f854a91ab3530293a, State: Initialized, Role: FOLLOWER
I20260810 04:41:40.423297 4422 consensus_queue.cc:260] T d9572be79ddc45e98a527612a51cdaf9 P 603becb479b7443f854a91ab3530293a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84e7718d636e41efbd3a4b32db54e3f8" member_type: VOTER last_known_addr { host: "127.22.167.65" port: 40941 } } peers { permanent_uuid: "603becb479b7443f854a91ab3530293a" member_type: VOTER last_known_addr { host: "127.22.167.66" port: 34655 } } peers { permanent_uuid: "5fc86b14002f4fd3ae81df14a166c562" member_type: VOTER last_known_addr { host: "127.22.167.67" port: 33991 } }
I20260810 04:41:40.423337 4425 tablet_bootstrap.cc:492] T d9572be79ddc45e98a527612a51cdaf9 P 5fc86b14002f4fd3ae81df14a166c562: Bootstrap starting.
I20260810 04:41:40.423378 4422 ts_tablet_manager.cc:1428] T d9572be79ddc45e98a527612a51cdaf9 P 603becb479b7443f854a91ab3530293a: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260810 04:41:40.423734 4425 tablet_bootstrap.cc:654] T d9572be79ddc45e98a527612a51cdaf9 P 5fc86b14002f4fd3ae81df14a166c562: Neither blocks nor log segments found. Creating new log.
I20260810 04:41:40.424165 4426 tablet_bootstrap.cc:492] T d9572be79ddc45e98a527612a51cdaf9 P 84e7718d636e41efbd3a4b32db54e3f8: No bootstrap required, opened a new log
I20260810 04:41:40.424212 4426 ts_tablet_manager.cc:1397] T d9572be79ddc45e98a527612a51cdaf9 P 84e7718d636e41efbd3a4b32db54e3f8: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.001s
I20260810 04:41:40.424342 4468 ts_tablet_manager.cc:1905] T c8f88edaa4234f1db6f47968f1cfe707 P 603becb479b7443f854a91ab3530293a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 04:41:40.424351 4467 ts_tablet_manager.cc:1905] T c8f88edaa4234f1db6f47968f1cfe707 P 5fc86b14002f4fd3ae81df14a166c562: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 04:41:40.424347 4426 raft_consensus.cc:348] T d9572be79ddc45e98a527612a51cdaf9 P 84e7718d636e41efbd3a4b32db54e3f8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84e7718d636e41efbd3a4b32db54e3f8" member_type: VOTER last_known_addr { host: "127.22.167.65" port: 40941 } } peers { permanent_uuid: "603becb479b7443f854a91ab3530293a" member_type: VOTER last_known_addr { host: "127.22.167.66" port: 34655 } } peers { permanent_uuid: "5fc86b14002f4fd3ae81df14a166c562" member_type: VOTER last_known_addr { host: "127.22.167.67" port: 33991 } }
I20260810 04:41:40.424424 4426 raft_consensus.cc:374] T d9572be79ddc45e98a527612a51cdaf9 P 84e7718d636e41efbd3a4b32db54e3f8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 04:41:40.424443 4426 raft_consensus.cc:729] T d9572be79ddc45e98a527612a51cdaf9 P 84e7718d636e41efbd3a4b32db54e3f8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84e7718d636e41efbd3a4b32db54e3f8, State: Initialized, Role: FOLLOWER
I20260810 04:41:40.424484 4426 consensus_queue.cc:260] T d9572be79ddc45e98a527612a51cdaf9 P 84e7718d636e41efbd3a4b32db54e3f8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84e7718d636e41efbd3a4b32db54e3f8" member_type: VOTER last_known_addr { host: "127.22.167.65" port: 40941 } } peers { permanent_uuid: "603becb479b7443f854a91ab3530293a" member_type: VOTER last_known_addr { host: "127.22.167.66" port: 34655 } } peers { permanent_uuid: "5fc86b14002f4fd3ae81df14a166c562" member_type: VOTER last_known_addr { host: "127.22.167.67" port: 33991 } }
I20260810 04:41:40.424566 4426 ts_tablet_manager.cc:1428] T d9572be79ddc45e98a527612a51cdaf9 P 84e7718d636e41efbd3a4b32db54e3f8: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260810 04:41:40.425699 4467 ts_tablet_manager.cc:1918] T c8f88edaa4234f1db6f47968f1cfe707 P 5fc86b14002f4fd3ae81df14a166c562: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 04:41:40.425765 4467 log.cc:1196] T c8f88edaa4234f1db6f47968f1cfe707 P 5fc86b14002f4fd3ae81df14a166c562: Deleting WAL directory at /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786336863388938-23197-0/minicluster-data/ts-2/wal/wals/c8f88edaa4234f1db6f47968f1cfe707
I20260810 04:41:40.425810 4468 ts_tablet_manager.cc:1918] T c8f88edaa4234f1db6f47968f1cfe707 P 603becb479b7443f854a91ab3530293a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 04:41:40.425886 4468 log.cc:1196] T c8f88edaa4234f1db6f47968f1cfe707 P 603becb479b7443f854a91ab3530293a: Deleting WAL directory at /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786336863388938-23197-0/minicluster-data/ts-1/wal/wals/c8f88edaa4234f1db6f47968f1cfe707
I20260810 04:41:40.426002 4467 ts_tablet_manager.cc:1939] T c8f88edaa4234f1db6f47968f1cfe707 P 5fc86b14002f4fd3ae81df14a166c562: Deleting consensus metadata
I20260810 04:41:40.426165 4468 ts_tablet_manager.cc:1939] T c8f88edaa4234f1db6f47968f1cfe707 P 603becb479b7443f854a91ab3530293a: Deleting consensus metadata
I20260810 04:41:40.426409 3571 catalog_manager.cc:4841] TS 5fc86b14002f4fd3ae81df14a166c562 (127.22.167.67:33991): tablet c8f88edaa4234f1db6f47968f1cfe707 (table test-table [id=b1a86c7cb541492cb22af120dca04630]) successfully deleted
I20260810 04:41:40.426491 3571 catalog_manager.cc:4841] TS 603becb479b7443f854a91ab3530293a (127.22.167.66:34655): tablet c8f88edaa4234f1db6f47968f1cfe707 (table test-table [id=b1a86c7cb541492cb22af120dca04630]) successfully deleted
I20260810 04:41:40.426743 4425 tablet_bootstrap.cc:492] T d9572be79ddc45e98a527612a51cdaf9 P 5fc86b14002f4fd3ae81df14a166c562: No bootstrap required, opened a new log
I20260810 04:41:40.426783 4425 ts_tablet_manager.cc:1397] T d9572be79ddc45e98a527612a51cdaf9 P 5fc86b14002f4fd3ae81df14a166c562: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.001s
I20260810 04:41:40.426906 4425 raft_consensus.cc:348] T d9572be79ddc45e98a527612a51cdaf9 P 5fc86b14002f4fd3ae81df14a166c562 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84e7718d636e41efbd3a4b32db54e3f8" member_type: VOTER last_known_addr { host: "127.22.167.65" port: 40941 } } peers { permanent_uuid: "603becb479b7443f854a91ab3530293a" member_type: VOTER last_known_addr { host: "127.22.167.66" port: 34655 } } peers { permanent_uuid: "5fc86b14002f4fd3ae81df14a166c562" member_type: VOTER last_known_addr { host: "127.22.167.67" port: 33991 } }
I20260810 04:41:40.426959 4425 raft_consensus.cc:374] T d9572be79ddc45e98a527612a51cdaf9 P 5fc86b14002f4fd3ae81df14a166c562 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 04:41:40.426972 4425 raft_consensus.cc:729] T d9572be79ddc45e98a527612a51cdaf9 P 5fc86b14002f4fd3ae81df14a166c562 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5fc86b14002f4fd3ae81df14a166c562, State: Initialized, Role: FOLLOWER
I20260810 04:41:40.427006 4425 consensus_queue.cc:260] T d9572be79ddc45e98a527612a51cdaf9 P 5fc86b14002f4fd3ae81df14a166c562 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84e7718d636e41efbd3a4b32db54e3f8" member_type: VOTER last_known_addr { host: "127.22.167.65" port: 40941 } } peers { permanent_uuid: "603becb479b7443f854a91ab3530293a" member_type: VOTER last_known_addr { host: "127.22.167.66" port: 34655 } } peers { permanent_uuid: "5fc86b14002f4fd3ae81df14a166c562" member_type: VOTER last_known_addr { host: "127.22.167.67" port: 33991 } }
I20260810 04:41:40.427086 4425 ts_tablet_manager.cc:1428] T d9572be79ddc45e98a527612a51cdaf9 P 5fc86b14002f4fd3ae81df14a166c562: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260810 04:41:40.704831 4458 raft_consensus.cc:482] T d9572be79ddc45e98a527612a51cdaf9 P 84e7718d636e41efbd3a4b32db54e3f8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 04:41:40.704945 4458 raft_consensus.cc:504] T d9572be79ddc45e98a527612a51cdaf9 P 84e7718d636e41efbd3a4b32db54e3f8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84e7718d636e41efbd3a4b32db54e3f8" member_type: VOTER last_known_addr { host: "127.22.167.65" port: 40941 } } peers { permanent_uuid: "603becb479b7443f854a91ab3530293a" member_type: VOTER last_known_addr { host: "127.22.167.66" port: 34655 } } peers { permanent_uuid: "5fc86b14002f4fd3ae81df14a166c562" member_type: VOTER last_known_addr { host: "127.22.167.67" port: 33991 } }
I20260810 04:41:40.705098 4458 leader_election.cc:288] T d9572be79ddc45e98a527612a51cdaf9 P 84e7718d636e41efbd3a4b32db54e3f8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 603becb479b7443f854a91ab3530293a (127.22.167.66:34655), 5fc86b14002f4fd3ae81df14a166c562 (127.22.167.67:33991)
I20260810 04:41:40.705392 4055 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d9572be79ddc45e98a527612a51cdaf9" candidate_uuid: "84e7718d636e41efbd3a4b32db54e3f8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "603becb479b7443f854a91ab3530293a" is_pre_election: true
I20260810 04:41:40.705493 4055 raft_consensus.cc:2454] T d9572be79ddc45e98a527612a51cdaf9 P 603becb479b7443f854a91ab3530293a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84e7718d636e41efbd3a4b32db54e3f8 in term 0.
I20260810 04:41:40.705632 3731 leader_election.cc:302] T d9572be79ddc45e98a527612a51cdaf9 P 84e7718d636e41efbd3a4b32db54e3f8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 603becb479b7443f854a91ab3530293a, 84e7718d636e41efbd3a4b32db54e3f8; no voters:
I20260810 04:41:40.705729 4300 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d9572be79ddc45e98a527612a51cdaf9" candidate_uuid: "84e7718d636e41efbd3a4b32db54e3f8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5fc86b14002f4fd3ae81df14a166c562" is_pre_election: true
I20260810 04:41:40.705766 4458 raft_consensus.cc:2781] T d9572be79ddc45e98a527612a51cdaf9 P 84e7718d636e41efbd3a4b32db54e3f8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 04:41:40.705811 4300 raft_consensus.cc:2454] T d9572be79ddc45e98a527612a51cdaf9 P 5fc86b14002f4fd3ae81df14a166c562 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84e7718d636e41efbd3a4b32db54e3f8 in term 0.
I20260810 04:41:40.705824 4458 raft_consensus.cc:482] T d9572be79ddc45e98a527612a51cdaf9 P 84e7718d636e41efbd3a4b32db54e3f8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 04:41:40.705843 4458 raft_consensus.cc:3037] T d9572be79ddc45e98a527612a51cdaf9 P 84e7718d636e41efbd3a4b32db54e3f8 [term 0 FOLLOWER]: Advancing to term 1
I20260810 04:41:40.706637 4458 raft_consensus.cc:504] T d9572be79ddc45e98a527612a51cdaf9 P 84e7718d636e41efbd3a4b32db54e3f8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84e7718d636e41efbd3a4b32db54e3f8" member_type: VOTER last_known_addr { host: "127.22.167.65" port: 40941 } } peers { permanent_uuid: "603becb479b7443f854a91ab3530293a" member_type: VOTER last_known_addr { host: "127.22.167.66" port: 34655 } } peers { permanent_uuid: "5fc86b14002f4fd3ae81df14a166c562" member_type: VOTER last_known_addr { host: "127.22.167.67" port: 33991 } }
I20260810 04:41:40.706764 4458 leader_election.cc:288] T d9572be79ddc45e98a527612a51cdaf9 P 84e7718d636e41efbd3a4b32db54e3f8 [CANDIDATE]: Term 1 election: Requested vote from peers 603becb479b7443f854a91ab3530293a (127.22.167.66:34655), 5fc86b14002f4fd3ae81df14a166c562 (127.22.167.67:33991)
I20260810 04:41:40.706934 4055 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d9572be79ddc45e98a527612a51cdaf9" candidate_uuid: "84e7718d636e41efbd3a4b32db54e3f8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "603becb479b7443f854a91ab3530293a"
I20260810 04:41:40.706956 4300 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d9572be79ddc45e98a527612a51cdaf9" candidate_uuid: "84e7718d636e41efbd3a4b32db54e3f8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5fc86b14002f4fd3ae81df14a166c562"
I20260810 04:41:40.707008 4055 raft_consensus.cc:3037] T d9572be79ddc45e98a527612a51cdaf9 P 603becb479b7443f854a91ab3530293a [term 0 FOLLOWER]: Advancing to term 1
I20260810 04:41:40.707022 4300 raft_consensus.cc:3037] T d9572be79ddc45e98a527612a51cdaf9 P 5fc86b14002f4fd3ae81df14a166c562 [term 0 FOLLOWER]: Advancing to term 1
I20260810 04:41:40.707659 4055 raft_consensus.cc:2454] T d9572be79ddc45e98a527612a51cdaf9 P 603becb479b7443f854a91ab3530293a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 84e7718d636e41efbd3a4b32db54e3f8 in term 1.
I20260810 04:41:40.707741 4300 raft_consensus.cc:2454] T d9572be79ddc45e98a527612a51cdaf9 P 5fc86b14002f4fd3ae81df14a166c562 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 84e7718d636e41efbd3a4b32db54e3f8 in term 1.
I20260810 04:41:40.707819 3731 leader_election.cc:302] T d9572be79ddc45e98a527612a51cdaf9 P 84e7718d636e41efbd3a4b32db54e3f8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 603becb479b7443f854a91ab3530293a, 84e7718d636e41efbd3a4b32db54e3f8; no voters:
I20260810 04:41:40.707912 4458 raft_consensus.cc:2781] T d9572be79ddc45e98a527612a51cdaf9 P 84e7718d636e41efbd3a4b32db54e3f8 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 04:41:40.707965 4458 raft_consensus.cc:686] T d9572be79ddc45e98a527612a51cdaf9 P 84e7718d636e41efbd3a4b32db54e3f8 [term 1 LEADER]: Becoming Leader. State: Replica: 84e7718d636e41efbd3a4b32db54e3f8, State: Running, Role: LEADER
I20260810 04:41:40.708014 4458 consensus_queue.cc:237] T d9572be79ddc45e98a527612a51cdaf9 P 84e7718d636e41efbd3a4b32db54e3f8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84e7718d636e41efbd3a4b32db54e3f8" member_type: VOTER last_known_addr { host: "127.22.167.65" port: 40941 } } peers { permanent_uuid: "603becb479b7443f854a91ab3530293a" member_type: VOTER last_known_addr { host: "127.22.167.66" port: 34655 } } peers { permanent_uuid: "5fc86b14002f4fd3ae81df14a166c562" member_type: VOTER last_known_addr { host: "127.22.167.67" port: 33991 } }
I20260810 04:41:40.708549 3597 catalog_manager.cc:5495] T d9572be79ddc45e98a527612a51cdaf9 P 84e7718d636e41efbd3a4b32db54e3f8 reported cstate change: term changed from 0 to 1, leader changed from <none> to 84e7718d636e41efbd3a4b32db54e3f8 (127.22.167.65). New cstate: current_term: 1 leader_uuid: "84e7718d636e41efbd3a4b32db54e3f8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84e7718d636e41efbd3a4b32db54e3f8" member_type: VOTER last_known_addr { host: "127.22.167.65" port: 40941 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "603becb479b7443f854a91ab3530293a" member_type: VOTER last_known_addr { host: "127.22.167.66" port: 34655 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5fc86b14002f4fd3ae81df14a166c562" member_type: VOTER last_known_addr { host: "127.22.167.67" port: 33991 } health_report { overall_health: UNKNOWN } } }
Aug 10 04:41:40 dist-test-slave-bqcl krb5kdc[3517](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786336899, etypes {rep=17 tkt=17 ses=17}, oryx/127.22.167.126@KRBTEST.COM for oryx/127.22.167.126@KRBTEST.COM
I20260810 04:41:40.800783 4484 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:41:40.794778 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52970 (local address 127.22.167.126:37713)
0810 04:41:40.794911 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0810 04:41:40.794913 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 04:41:40.794925 (+ 12us) server_negotiation.cc:407] Connection header received
0810 04:41:40.794988 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:41:40.794988 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:41:40.794997 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:41:40.795053 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0810 04:41:40.795472 (+ 419us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:41:40.795959 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:41:40.796584 (+ 625us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:41:40.796759 (+ 175us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:41:40.796798 (+ 39us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:41:40.798635 (+ 1837us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 04:41:40.798636 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 04:41:40.798637 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 04:41:40.798643 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0810 04:41:40.799921 (+ 1278us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 04:41:40.800232 (+ 311us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 04:41:40.800232 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 04:41:40.800232 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 04:41:40.800285 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 04:41:40.800421 (+ 136us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 04:41:40.800421 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 04:41:40.800421 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 04:41:40.800512 (+ 91us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 04:41:40.800556 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0810 04:41:40.800655 (+ 99us) server_negotiation.cc:299] Negotiation successful
0810 04:41:40.800719 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":110,"thread_start_us":88,"threads_started":1}
I20260810 04:41:40.800861 4483 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:41:40.794703 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.126:37713 (local address 127.0.0.1:52970)
0810 04:41:40.794873 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0810 04:41:40.794885 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0810 04:41:40.794941 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 04:41:40.795063 (+ 122us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 04:41:40.795064 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 04:41:40.795364 (+ 300us) client_negotiation.cc:189] Negotiated authn=SASL
0810 04:41:40.795418 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:41:40.795420 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:41:40.796010 (+ 590us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:41:40.796010 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:41:40.796535 (+ 525us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:41:40.796536 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:41:40.796802 (+ 266us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:41:40.796803 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:41:40.796837 (+ 34us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:41:40.797282 (+ 445us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 04:41:40.797286 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0810 04:41:40.798557 (+ 1271us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 04:41:40.799985 (+ 1428us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 04:41:40.799990 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 04:41:40.799992 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 04:41:40.800182 (+ 190us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 04:41:40.800328 (+ 146us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 04:41:40.800329 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 04:41:40.800329 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 04:41:40.800381 (+ 52us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 04:41:40.800551 (+ 170us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 04:41:40.800555 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 04:41:40.800592 (+ 37us) client_negotiation.cc:769] Sending connection context
0810 04:41:40.800638 (+ 46us) client_negotiation.cc:240] Negotiation successful
0810 04:41:40.800766 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":130,"mutex_wait_us":48,"thread_start_us":83,"threads_started":1}
I20260810 04:41:40.802518 3597 catalog_manager.cc:1908] Servicing CreateTable request from {username='oryx', principal='oryx/127.22.167.126@KRBTEST.COM'} at 127.0.0.1:52970:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260810 04:41:40.802696 3597 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 04:41:40.803977 3815 tablet_service.cc:1467] Processing CreateTablet for tablet fcb915290a1144a89651d649fce7b8ec (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=bb01fea3b1c74d0a974f0a86026517b3]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260810 04:41:40.804042 4031 tablet_service.cc:1467] Processing CreateTablet for tablet fcb915290a1144a89651d649fce7b8ec (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=bb01fea3b1c74d0a974f0a86026517b3]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260810 04:41:40.804023 4267 tablet_service.cc:1467] Processing CreateTablet for tablet fcb915290a1144a89651d649fce7b8ec (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=bb01fea3b1c74d0a974f0a86026517b3]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260810 04:41:40.804126 3815 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fcb915290a1144a89651d649fce7b8ec. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 04:41:40.804176 4267 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fcb915290a1144a89651d649fce7b8ec. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 04:41:40.804183 4031 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fcb915290a1144a89651d649fce7b8ec. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 04:41:40.805645 4422 tablet_bootstrap.cc:492] T fcb915290a1144a89651d649fce7b8ec P 603becb479b7443f854a91ab3530293a: Bootstrap starting.
I20260810 04:41:40.805645 4425 tablet_bootstrap.cc:492] T fcb915290a1144a89651d649fce7b8ec P 5fc86b14002f4fd3ae81df14a166c562: Bootstrap starting.
I20260810 04:41:40.805663 4426 tablet_bootstrap.cc:492] T fcb915290a1144a89651d649fce7b8ec P 84e7718d636e41efbd3a4b32db54e3f8: Bootstrap starting.
I20260810 04:41:40.805989 4426 tablet_bootstrap.cc:654] T fcb915290a1144a89651d649fce7b8ec P 84e7718d636e41efbd3a4b32db54e3f8: Neither blocks nor log segments found. Creating new log.
I20260810 04:41:40.806022 4422 tablet_bootstrap.cc:654] T fcb915290a1144a89651d649fce7b8ec P 603becb479b7443f854a91ab3530293a: Neither blocks nor log segments found. Creating new log.
I20260810 04:41:40.806022 4425 tablet_bootstrap.cc:654] T fcb915290a1144a89651d649fce7b8ec P 5fc86b14002f4fd3ae81df14a166c562: Neither blocks nor log segments found. Creating new log.
I20260810 04:41:40.806509 4426 tablet_bootstrap.cc:492] T fcb915290a1144a89651d649fce7b8ec P 84e7718d636e41efbd3a4b32db54e3f8: No bootstrap required, opened a new log
I20260810 04:41:40.806564 4426 ts_tablet_manager.cc:1397] T fcb915290a1144a89651d649fce7b8ec P 84e7718d636e41efbd3a4b32db54e3f8: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260810 04:41:40.806667 4425 tablet_bootstrap.cc:492] T fcb915290a1144a89651d649fce7b8ec P 5fc86b14002f4fd3ae81df14a166c562: No bootstrap required, opened a new log
I20260810 04:41:40.806717 4425 ts_tablet_manager.cc:1397] T fcb915290a1144a89651d649fce7b8ec P 5fc86b14002f4fd3ae81df14a166c562: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260810 04:41:40.806753 4426 raft_consensus.cc:348] T fcb915290a1144a89651d649fce7b8ec P 84e7718d636e41efbd3a4b32db54e3f8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84e7718d636e41efbd3a4b32db54e3f8" member_type: VOTER last_known_addr { host: "127.22.167.65" port: 40941 } } peers { permanent_uuid: "603becb479b7443f854a91ab3530293a" member_type: VOTER last_known_addr { host: "127.22.167.66" port: 34655 } } peers { permanent_uuid: "5fc86b14002f4fd3ae81df14a166c562" member_type: VOTER last_known_addr { host: "127.22.167.67" port: 33991 } }
I20260810 04:41:40.806797 4422 tablet_bootstrap.cc:492] T fcb915290a1144a89651d649fce7b8ec P 603becb479b7443f854a91ab3530293a: No bootstrap required, opened a new log
I20260810 04:41:40.806821 4426 raft_consensus.cc:374] T fcb915290a1144a89651d649fce7b8ec P 84e7718d636e41efbd3a4b32db54e3f8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 04:41:40.806833 4422 ts_tablet_manager.cc:1397] T fcb915290a1144a89651d649fce7b8ec P 603becb479b7443f854a91ab3530293a: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260810 04:41:40.806839 4426 raft_consensus.cc:729] T fcb915290a1144a89651d649fce7b8ec P 84e7718d636e41efbd3a4b32db54e3f8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84e7718d636e41efbd3a4b32db54e3f8, State: Initialized, Role: FOLLOWER
I20260810 04:41:40.806845 4425 raft_consensus.cc:348] T fcb915290a1144a89651d649fce7b8ec P 5fc86b14002f4fd3ae81df14a166c562 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84e7718d636e41efbd3a4b32db54e3f8" member_type: VOTER last_known_addr { host: "127.22.167.65" port: 40941 } } peers { permanent_uuid: "603becb479b7443f854a91ab3530293a" member_type: VOTER last_known_addr { host: "127.22.167.66" port: 34655 } } peers { permanent_uuid: "5fc86b14002f4fd3ae81df14a166c562" member_type: VOTER last_known_addr { host: "127.22.167.67" port: 33991 } }
I20260810 04:41:40.806900 4425 raft_consensus.cc:374] T fcb915290a1144a89651d649fce7b8ec P 5fc86b14002f4fd3ae81df14a166c562 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 04:41:40.806881 4426 consensus_queue.cc:260] T fcb915290a1144a89651d649fce7b8ec P 84e7718d636e41efbd3a4b32db54e3f8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84e7718d636e41efbd3a4b32db54e3f8" member_type: VOTER last_known_addr { host: "127.22.167.65" port: 40941 } } peers { permanent_uuid: "603becb479b7443f854a91ab3530293a" member_type: VOTER last_known_addr { host: "127.22.167.66" port: 34655 } } peers { permanent_uuid: "5fc86b14002f4fd3ae81df14a166c562" member_type: VOTER last_known_addr { host: "127.22.167.67" port: 33991 } }
I20260810 04:41:40.806918 4425 raft_consensus.cc:729] T fcb915290a1144a89651d649fce7b8ec P 5fc86b14002f4fd3ae81df14a166c562 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5fc86b14002f4fd3ae81df14a166c562, State: Initialized, Role: FOLLOWER
I20260810 04:41:40.806941 4422 raft_consensus.cc:348] T fcb915290a1144a89651d649fce7b8ec P 603becb479b7443f854a91ab3530293a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84e7718d636e41efbd3a4b32db54e3f8" member_type: VOTER last_known_addr { host: "127.22.167.65" port: 40941 } } peers { permanent_uuid: "603becb479b7443f854a91ab3530293a" member_type: VOTER last_known_addr { host: "127.22.167.66" port: 34655 } } peers { permanent_uuid: "5fc86b14002f4fd3ae81df14a166c562" member_type: VOTER last_known_addr { host: "127.22.167.67" port: 33991 } }
I20260810 04:41:40.806967 4426 ts_tablet_manager.cc:1428] T fcb915290a1144a89651d649fce7b8ec P 84e7718d636e41efbd3a4b32db54e3f8: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260810 04:41:40.806962 4425 consensus_queue.cc:260] T fcb915290a1144a89651d649fce7b8ec P 5fc86b14002f4fd3ae81df14a166c562 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84e7718d636e41efbd3a4b32db54e3f8" member_type: VOTER last_known_addr { host: "127.22.167.65" port: 40941 } } peers { permanent_uuid: "603becb479b7443f854a91ab3530293a" member_type: VOTER last_known_addr { host: "127.22.167.66" port: 34655 } } peers { permanent_uuid: "5fc86b14002f4fd3ae81df14a166c562" member_type: VOTER last_known_addr { host: "127.22.167.67" port: 33991 } }
I20260810 04:41:40.806995 4422 raft_consensus.cc:374] T fcb915290a1144a89651d649fce7b8ec P 603becb479b7443f854a91ab3530293a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 04:41:40.807015 4422 raft_consensus.cc:729] T fcb915290a1144a89651d649fce7b8ec P 603becb479b7443f854a91ab3530293a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 603becb479b7443f854a91ab3530293a, State: Initialized, Role: FOLLOWER
I20260810 04:41:40.807041 4425 ts_tablet_manager.cc:1428] T fcb915290a1144a89651d649fce7b8ec P 5fc86b14002f4fd3ae81df14a166c562: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260810 04:41:40.807053 4422 consensus_queue.cc:260] T fcb915290a1144a89651d649fce7b8ec P 603becb479b7443f854a91ab3530293a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84e7718d636e41efbd3a4b32db54e3f8" member_type: VOTER last_known_addr { host: "127.22.167.65" port: 40941 } } peers { permanent_uuid: "603becb479b7443f854a91ab3530293a" member_type: VOTER last_known_addr { host: "127.22.167.66" port: 34655 } } peers { permanent_uuid: "5fc86b14002f4fd3ae81df14a166c562" member_type: VOTER last_known_addr { host: "127.22.167.67" port: 33991 } }
I20260810 04:41:40.807058 4458 tablet_replica.cc:439] T fcb915290a1144a89651d649fce7b8ec P 84e7718d636e41efbd3a4b32db54e3f8: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84e7718d636e41efbd3a4b32db54e3f8" member_type: VOTER last_known_addr { host: "127.22.167.65" port: 40941 } } peers { permanent_uuid: "603becb479b7443f854a91ab3530293a" member_type: VOTER last_known_addr { host: "127.22.167.66" port: 34655 } } peers { permanent_uuid: "5fc86b14002f4fd3ae81df14a166c562" member_type: VOTER last_known_addr { host: "127.22.167.67" port: 33991 } } }
I20260810 04:41:40.807117 4458 tablet_replica.cc:442] T fcb915290a1144a89651d649fce7b8ec P 84e7718d636e41efbd3a4b32db54e3f8: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 04:41:40.807143 4422 ts_tablet_manager.cc:1428] T fcb915290a1144a89651d649fce7b8ec P 603becb479b7443f854a91ab3530293a: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260810 04:41:40.807204 4433 tablet_replica.cc:439] T fcb915290a1144a89651d649fce7b8ec P 5fc86b14002f4fd3ae81df14a166c562: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84e7718d636e41efbd3a4b32db54e3f8" member_type: VOTER last_known_addr { host: "127.22.167.65" port: 40941 } } peers { permanent_uuid: "603becb479b7443f854a91ab3530293a" member_type: VOTER last_known_addr { host: "127.22.167.66" port: 34655 } } peers { permanent_uuid: "5fc86b14002f4fd3ae81df14a166c562" member_type: VOTER last_known_addr { host: "127.22.167.67" port: 33991 } } }
I20260810 04:41:40.807300 4433 tablet_replica.cc:442] T fcb915290a1144a89651d649fce7b8ec P 5fc86b14002f4fd3ae81df14a166c562: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 04:41:40.807266 4435 tablet_replica.cc:439] T fcb915290a1144a89651d649fce7b8ec P 603becb479b7443f854a91ab3530293a: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84e7718d636e41efbd3a4b32db54e3f8" member_type: VOTER last_known_addr { host: "127.22.167.65" port: 40941 } } peers { permanent_uuid: "603becb479b7443f854a91ab3530293a" member_type: VOTER last_known_addr { host: "127.22.167.66" port: 34655 } } peers { permanent_uuid: "5fc86b14002f4fd3ae81df14a166c562" member_type: VOTER last_known_addr { host: "127.22.167.67" port: 33991 } } }
I20260810 04:41:40.807343 4435 tablet_replica.cc:442] T fcb915290a1144a89651d649fce7b8ec P 603becb479b7443f854a91ab3530293a: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 04:41:40.880299 4435 raft_consensus.cc:482] T fcb915290a1144a89651d649fce7b8ec P 603becb479b7443f854a91ab3530293a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 04:41:40.880389 4435 raft_consensus.cc:504] T fcb915290a1144a89651d649fce7b8ec P 603becb479b7443f854a91ab3530293a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84e7718d636e41efbd3a4b32db54e3f8" member_type: VOTER last_known_addr { host: "127.22.167.65" port: 40941 } } peers { permanent_uuid: "603becb479b7443f854a91ab3530293a" member_type: VOTER last_known_addr { host: "127.22.167.66" port: 34655 } } peers { permanent_uuid: "5fc86b14002f4fd3ae81df14a166c562" member_type: VOTER last_known_addr { host: "127.22.167.67" port: 33991 } }
I20260810 04:41:40.880721 4435 leader_election.cc:288] T fcb915290a1144a89651d649fce7b8ec P 603becb479b7443f854a91ab3530293a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 84e7718d636e41efbd3a4b32db54e3f8 (127.22.167.65:40941), 5fc86b14002f4fd3ae81df14a166c562 (127.22.167.67:33991)
I20260810 04:41:40.883538 4412 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:41:40.881030 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.167.66:34983 (local address 127.22.167.65:40941)
0810 04:41:40.881085 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0810 04:41:40.881086 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0810 04:41:40.881098 (+ 12us) server_negotiation.cc:407] Connection header received
0810 04:41:40.881125 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:41:40.881126 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:41:40.881137 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:41:40.881187 (+ 50us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 04:41:40.881323 (+ 136us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:41:40.881822 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:41:40.882807 (+ 985us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:41:40.883234 (+ 427us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:41:40.883285 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:41:40.883293 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0810 04:41:40.883436 (+ 143us) server_negotiation.cc:299] Negotiation successful
0810 04:41:40.883471 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260810 04:41:40.883548 4489 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:41:40.880853 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.65:40941 (local address 127.22.167.66:34983)
0810 04:41:40.880989 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0810 04:41:40.880999 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0810 04:41:40.881077 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 04:41:40.881205 (+ 128us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 04:41:40.881206 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 04:41:40.881208 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 04:41:40.881263 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:41:40.881265 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:41:40.881892 (+ 627us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:41:40.881894 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:41:40.882719 (+ 825us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:41:40.882721 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:41:40.883297 (+ 576us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:41:40.883298 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:41:40.883365 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:41:40.883370 (+ 5us) client_negotiation.cc:769] Sending connection context
0810 04:41:40.883422 (+ 52us) client_negotiation.cc:240] Negotiation successful
0810 04:41:40.883460 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":122,"mutex_wait_us":19,"thread_start_us":56,"threads_started":1}
I20260810 04:41:40.883809 4413 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:41:40.880949 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.167.66:57987 (local address 127.22.167.67:33991)
0810 04:41:40.880991 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0810 04:41:40.880993 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 04:41:40.881032 (+ 39us) server_negotiation.cc:407] Connection header received
0810 04:41:40.881128 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:41:40.881129 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:41:40.881139 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:41:40.881187 (+ 48us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 04:41:40.881377 (+ 190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:41:40.881836 (+ 459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:41:40.883012 (+ 1176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:41:40.883403 (+ 391us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:41:40.883463 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:41:40.883470 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0810 04:41:40.883695 (+ 225us) server_negotiation.cc:299] Negotiation successful
0810 04:41:40.883749 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260810 04:41:40.883824 3848 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fcb915290a1144a89651d649fce7b8ec" candidate_uuid: "603becb479b7443f854a91ab3530293a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84e7718d636e41efbd3a4b32db54e3f8" is_pre_election: true
I20260810 04:41:40.883960 3848 raft_consensus.cc:2454] T fcb915290a1144a89651d649fce7b8ec P 84e7718d636e41efbd3a4b32db54e3f8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 603becb479b7443f854a91ab3530293a in term 0.
I20260810 04:41:40.884027 4488 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:41:40.880854 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.67:33991 (local address 127.22.167.66:57987)
0810 04:41:40.880970 (+ 116us) negotiation.cc:107] Waiting for socket to connect
0810 04:41:40.880983 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0810 04:41:40.881053 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 04:41:40.881201 (+ 148us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 04:41:40.881202 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 04:41:40.881204 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 04:41:40.881313 (+ 109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:41:40.881315 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:41:40.881911 (+ 596us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:41:40.881912 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:41:40.882929 (+ 1017us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:41:40.882932 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:41:40.883476 (+ 544us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:41:40.883477 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:41:40.883596 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:41:40.883601 (+ 5us) client_negotiation.cc:769] Sending connection context
0810 04:41:40.883887 (+ 286us) client_negotiation.cc:240] Negotiation successful
0810 04:41:40.883932 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":98,"mutex_wait_us":2,"thread_start_us":67,"threads_started":1}
I20260810 04:41:40.884161 3960 leader_election.cc:302] T fcb915290a1144a89651d649fce7b8ec P 603becb479b7443f854a91ab3530293a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 603becb479b7443f854a91ab3530293a, 84e7718d636e41efbd3a4b32db54e3f8; no voters:
I20260810 04:41:40.884279 4435 raft_consensus.cc:2781] T fcb915290a1144a89651d649fce7b8ec P 603becb479b7443f854a91ab3530293a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 04:41:40.884325 4435 raft_consensus.cc:482] T fcb915290a1144a89651d649fce7b8ec P 603becb479b7443f854a91ab3530293a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 04:41:40.884270 4300 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fcb915290a1144a89651d649fce7b8ec" candidate_uuid: "603becb479b7443f854a91ab3530293a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5fc86b14002f4fd3ae81df14a166c562" is_pre_election: true
I20260810 04:41:40.884349 4435 raft_consensus.cc:3037] T fcb915290a1144a89651d649fce7b8ec P 603becb479b7443f854a91ab3530293a [term 0 FOLLOWER]: Advancing to term 1
I20260810 04:41:40.884357 4300 raft_consensus.cc:2454] T fcb915290a1144a89651d649fce7b8ec P 5fc86b14002f4fd3ae81df14a166c562 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 603becb479b7443f854a91ab3530293a in term 0.
I20260810 04:41:40.885185 4435 raft_consensus.cc:504] T fcb915290a1144a89651d649fce7b8ec P 603becb479b7443f854a91ab3530293a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84e7718d636e41efbd3a4b32db54e3f8" member_type: VOTER last_known_addr { host: "127.22.167.65" port: 40941 } } peers { permanent_uuid: "603becb479b7443f854a91ab3530293a" member_type: VOTER last_known_addr { host: "127.22.167.66" port: 34655 } } peers { permanent_uuid: "5fc86b14002f4fd3ae81df14a166c562" member_type: VOTER last_known_addr { host: "127.22.167.67" port: 33991 } }
I20260810 04:41:40.885308 4435 leader_election.cc:288] T fcb915290a1144a89651d649fce7b8ec P 603becb479b7443f854a91ab3530293a [CANDIDATE]: Term 1 election: Requested vote from peers 84e7718d636e41efbd3a4b32db54e3f8 (127.22.167.65:40941), 5fc86b14002f4fd3ae81df14a166c562 (127.22.167.67:33991)
I20260810 04:41:40.885437 3848 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fcb915290a1144a89651d649fce7b8ec" candidate_uuid: "603becb479b7443f854a91ab3530293a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84e7718d636e41efbd3a4b32db54e3f8"
I20260810 04:41:40.885478 4300 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fcb915290a1144a89651d649fce7b8ec" candidate_uuid: "603becb479b7443f854a91ab3530293a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5fc86b14002f4fd3ae81df14a166c562"
I20260810 04:41:40.885510 3848 raft_consensus.cc:3037] T fcb915290a1144a89651d649fce7b8ec P 84e7718d636e41efbd3a4b32db54e3f8 [term 0 FOLLOWER]: Advancing to term 1
I20260810 04:41:40.885540 4300 raft_consensus.cc:3037] T fcb915290a1144a89651d649fce7b8ec P 5fc86b14002f4fd3ae81df14a166c562 [term 0 FOLLOWER]: Advancing to term 1
I20260810 04:41:40.886137 3848 raft_consensus.cc:2454] T fcb915290a1144a89651d649fce7b8ec P 84e7718d636e41efbd3a4b32db54e3f8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 603becb479b7443f854a91ab3530293a in term 1.
I20260810 04:41:40.886266 4300 raft_consensus.cc:2454] T fcb915290a1144a89651d649fce7b8ec P 5fc86b14002f4fd3ae81df14a166c562 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 603becb479b7443f854a91ab3530293a in term 1.
I20260810 04:41:40.886283 3960 leader_election.cc:302] T fcb915290a1144a89651d649fce7b8ec P 603becb479b7443f854a91ab3530293a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 603becb479b7443f854a91ab3530293a, 84e7718d636e41efbd3a4b32db54e3f8; no voters:
I20260810 04:41:40.886354 4435 raft_consensus.cc:2781] T fcb915290a1144a89651d649fce7b8ec P 603becb479b7443f854a91ab3530293a [term 1 FOLLOWER]: Leader election won for term 1
I20260810 04:41:40.886474 4435 raft_consensus.cc:686] T fcb915290a1144a89651d649fce7b8ec P 603becb479b7443f854a91ab3530293a [term 1 LEADER]: Becoming Leader. State: Replica: 603becb479b7443f854a91ab3530293a, State: Running, Role: LEADER
I20260810 04:41:40.886554 4435 consensus_queue.cc:237] T fcb915290a1144a89651d649fce7b8ec P 603becb479b7443f854a91ab3530293a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84e7718d636e41efbd3a4b32db54e3f8" member_type: VOTER last_known_addr { host: "127.22.167.65" port: 40941 } } peers { permanent_uuid: "603becb479b7443f854a91ab3530293a" member_type: VOTER last_known_addr { host: "127.22.167.66" port: 34655 } } peers { permanent_uuid: "5fc86b14002f4fd3ae81df14a166c562" member_type: VOTER last_known_addr { host: "127.22.167.67" port: 33991 } }
I20260810 04:41:40.886881 4491 tablet_replica.cc:439] T fcb915290a1144a89651d649fce7b8ec P 603becb479b7443f854a91ab3530293a: TxnStatusTablet state changed. Reason: New leader 603becb479b7443f854a91ab3530293a. Latest consensus state: current_term: 1 leader_uuid: "603becb479b7443f854a91ab3530293a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84e7718d636e41efbd3a4b32db54e3f8" member_type: VOTER last_known_addr { host: "127.22.167.65" port: 40941 } } peers { permanent_uuid: "603becb479b7443f854a91ab3530293a" member_type: VOTER last_known_addr { host: "127.22.167.66" port: 34655 } } peers { permanent_uuid: "5fc86b14002f4fd3ae81df14a166c562" member_type: VOTER last_known_addr { host: "127.22.167.67" port: 33991 } } }
I20260810 04:41:40.886984 4491 tablet_replica.cc:442] T fcb915290a1144a89651d649fce7b8ec P 603becb479b7443f854a91ab3530293a: This TxnStatusTablet replica's current role is: LEADER
I20260810 04:41:40.887188 4495 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260810 04:41:40.887292 4495 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260810 04:41:40.887221 3593 catalog_manager.cc:5495] T fcb915290a1144a89651d649fce7b8ec P 603becb479b7443f854a91ab3530293a reported cstate change: term changed from 0 to 1, leader changed from <none> to 603becb479b7443f854a91ab3530293a (127.22.167.66). New cstate: current_term: 1 leader_uuid: "603becb479b7443f854a91ab3530293a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84e7718d636e41efbd3a4b32db54e3f8" member_type: VOTER last_known_addr { host: "127.22.167.65" port: 40941 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "603becb479b7443f854a91ab3530293a" member_type: VOTER last_known_addr { host: "127.22.167.66" port: 34655 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5fc86b14002f4fd3ae81df14a166c562" member_type: VOTER last_known_addr { host: "127.22.167.67" port: 33991 } health_report { overall_health: UNKNOWN } } }
I20260810 04:41:40.899098 4501 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:41:40.896477 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50382 (local address 127.22.167.66:34655)
0810 04:41:40.896633 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0810 04:41:40.896635 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 04:41:40.896641 (+ 6us) server_negotiation.cc:407] Connection header received
0810 04:41:40.896665 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:41:40.896666 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:41:40.896681 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:41:40.896744 (+ 63us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 04:41:40.896886 (+ 142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:41:40.897360 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:41:40.898412 (+ 1052us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:41:40.898604 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:41:40.898649 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:41:40.898833 (+ 184us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 04:41:40.898916 (+ 83us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 04:41:40.898961 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0810 04:41:40.899026 (+ 65us) server_negotiation.cc:299] Negotiation successful
0810 04:41:40.899042 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":126,"thread_start_us":36,"threads_started":1}
I20260810 04:41:40.899150 4483 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:41:40.896364 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.66:34655 (local address 127.0.0.1:50382)
0810 04:41:40.896431 (+ 67us) negotiation.cc:107] Waiting for socket to connect
0810 04:41:40.896443 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0810 04:41:40.896493 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 04:41:40.896771 (+ 278us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 04:41:40.896772 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 04:41:40.896774 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0810 04:41:40.896825 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:41:40.896827 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:41:40.897433 (+ 606us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:41:40.897435 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:41:40.898341 (+ 906us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:41:40.898345 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:41:40.898672 (+ 327us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:41:40.898674 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:41:40.898733 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:41:40.898753 (+ 20us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0810 04:41:40.898969 (+ 216us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0810 04:41:40.898975 (+ 6us) client_negotiation.cc:769] Sending connection context
0810 04:41:40.899025 (+ 50us) client_negotiation.cc:240] Negotiation successful
0810 04:41:40.899074 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":22}
I20260810 04:41:40.900228 3848 raft_consensus.cc:1261] T fcb915290a1144a89651d649fce7b8ec P 84e7718d636e41efbd3a4b32db54e3f8 [term 1 FOLLOWER]: Refusing update from remote peer 603becb479b7443f854a91ab3530293a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 04:41:40.900434 4300 raft_consensus.cc:1261] T fcb915290a1144a89651d649fce7b8ec P 5fc86b14002f4fd3ae81df14a166c562 [term 1 FOLLOWER]: Refusing update from remote peer 603becb479b7443f854a91ab3530293a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 04:41:40.900668 4491 consensus_queue.cc:1035] T fcb915290a1144a89651d649fce7b8ec P 603becb479b7443f854a91ab3530293a [LEADER]: Connected to new peer: Peer: permanent_uuid: "5fc86b14002f4fd3ae81df14a166c562" member_type: VOTER last_known_addr { host: "127.22.167.67" port: 33991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 04:41:40.900790 4435 consensus_queue.cc:1035] T fcb915290a1144a89651d649fce7b8ec P 603becb479b7443f854a91ab3530293a [LEADER]: Connected to new peer: Peer: permanent_uuid: "84e7718d636e41efbd3a4b32db54e3f8" member_type: VOTER last_known_addr { host: "127.22.167.65" port: 40941 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 04:41:40.901336 4433 tablet_replica.cc:439] T fcb915290a1144a89651d649fce7b8ec P 5fc86b14002f4fd3ae81df14a166c562: TxnStatusTablet state changed. Reason: New leader 603becb479b7443f854a91ab3530293a. Latest consensus state: current_term: 1 leader_uuid: "603becb479b7443f854a91ab3530293a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84e7718d636e41efbd3a4b32db54e3f8" member_type: VOTER last_known_addr { host: "127.22.167.65" port: 40941 } } peers { permanent_uuid: "603becb479b7443f854a91ab3530293a" member_type: VOTER last_known_addr { host: "127.22.167.66" port: 34655 } } peers { permanent_uuid: "5fc86b14002f4fd3ae81df14a166c562" member_type: VOTER last_known_addr { host: "127.22.167.67" port: 33991 } } }
I20260810 04:41:40.901419 4433 tablet_replica.cc:442] T fcb915290a1144a89651d649fce7b8ec P 5fc86b14002f4fd3ae81df14a166c562: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 04:41:40.901443 4458 tablet_replica.cc:439] T fcb915290a1144a89651d649fce7b8ec P 84e7718d636e41efbd3a4b32db54e3f8: TxnStatusTablet state changed. Reason: New leader 603becb479b7443f854a91ab3530293a. Latest consensus state: current_term: 1 leader_uuid: "603becb479b7443f854a91ab3530293a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84e7718d636e41efbd3a4b32db54e3f8" member_type: VOTER last_known_addr { host: "127.22.167.65" port: 40941 } } peers { permanent_uuid: "603becb479b7443f854a91ab3530293a" member_type: VOTER last_known_addr { host: "127.22.167.66" port: 34655 } } peers { permanent_uuid: "5fc86b14002f4fd3ae81df14a166c562" member_type: VOTER last_known_addr { host: "127.22.167.67" port: 33991 } } }
I20260810 04:41:40.901510 4458 tablet_replica.cc:442] T fcb915290a1144a89651d649fce7b8ec P 84e7718d636e41efbd3a4b32db54e3f8: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 04:41:40.902082 4435 tablet_replica.cc:439] T fcb915290a1144a89651d649fce7b8ec P 603becb479b7443f854a91ab3530293a: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "603becb479b7443f854a91ab3530293a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84e7718d636e41efbd3a4b32db54e3f8" member_type: VOTER last_known_addr { host: "127.22.167.65" port: 40941 } } peers { permanent_uuid: "603becb479b7443f854a91ab3530293a" member_type: VOTER last_known_addr { host: "127.22.167.66" port: 34655 } } peers { permanent_uuid: "5fc86b14002f4fd3ae81df14a166c562" member_type: VOTER last_known_addr { host: "127.22.167.67" port: 33991 } } }
I20260810 04:41:40.902180 4435 tablet_replica.cc:442] T fcb915290a1144a89651d649fce7b8ec P 603becb479b7443f854a91ab3530293a: This TxnStatusTablet replica's current role is: LEADER
I20260810 04:41:40.902423 4435 tablet_replica.cc:439] T fcb915290a1144a89651d649fce7b8ec P 603becb479b7443f854a91ab3530293a: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "603becb479b7443f854a91ab3530293a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84e7718d636e41efbd3a4b32db54e3f8" member_type: VOTER last_known_addr { host: "127.22.167.65" port: 40941 } } peers { permanent_uuid: "603becb479b7443f854a91ab3530293a" member_type: VOTER last_known_addr { host: "127.22.167.66" port: 34655 } } peers { permanent_uuid: "5fc86b14002f4fd3ae81df14a166c562" member_type: VOTER last_known_addr { host: "127.22.167.67" port: 33991 } } }
I20260810 04:41:40.902493 4435 tablet_replica.cc:442] T fcb915290a1144a89651d649fce7b8ec P 603becb479b7443f854a91ab3530293a: This TxnStatusTablet replica's current role is: LEADER
I20260810 04:41:40.902696 4458 tablet_replica.cc:439] T fcb915290a1144a89651d649fce7b8ec P 84e7718d636e41efbd3a4b32db54e3f8: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "603becb479b7443f854a91ab3530293a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84e7718d636e41efbd3a4b32db54e3f8" member_type: VOTER last_known_addr { host: "127.22.167.65" port: 40941 } } peers { permanent_uuid: "603becb479b7443f854a91ab3530293a" member_type: VOTER last_known_addr { host: "127.22.167.66" port: 34655 } } peers { permanent_uuid: "5fc86b14002f4fd3ae81df14a166c562" member_type: VOTER last_known_addr { host: "127.22.167.67" port: 33991 } } }
I20260810 04:41:40.902771 4458 tablet_replica.cc:442] T fcb915290a1144a89651d649fce7b8ec P 84e7718d636e41efbd3a4b32db54e3f8: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 04:41:40.902872 4451 mvcc.cc:205] Tried to move back new op lower bound from 7316835946085224448 to 7316835946031747072. Current Snapshot: MvccSnapshot[applied={T|T < 7316835946085224448}]
I20260810 04:41:40.902944 4433 tablet_replica.cc:439] T fcb915290a1144a89651d649fce7b8ec P 5fc86b14002f4fd3ae81df14a166c562: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "603becb479b7443f854a91ab3530293a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84e7718d636e41efbd3a4b32db54e3f8" member_type: VOTER last_known_addr { host: "127.22.167.65" port: 40941 } } peers { permanent_uuid: "603becb479b7443f854a91ab3530293a" member_type: VOTER last_known_addr { host: "127.22.167.66" port: 34655 } } peers { permanent_uuid: "5fc86b14002f4fd3ae81df14a166c562" member_type: VOTER last_known_addr { host: "127.22.167.67" port: 33991 } } }
I20260810 04:41:40.903003 4433 tablet_replica.cc:442] T fcb915290a1144a89651d649fce7b8ec P 5fc86b14002f4fd3ae81df14a166c562: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 04:41:40.912725 4501 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:41:40.906033 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.167.65:38357 (local address 127.22.167.66:34655)
0810 04:41:40.906074 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0810 04:41:40.906075 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0810 04:41:40.910607 (+ 4532us) server_negotiation.cc:407] Connection header received
0810 04:41:40.910657 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:41:40.910658 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:41:40.910667 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:41:40.910713 (+ 46us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 04:41:40.910812 (+ 99us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:41:40.911360 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:41:40.912089 (+ 729us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:41:40.912273 (+ 184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:41:40.912324 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:41:40.912514 (+ 190us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 04:41:40.912542 (+ 28us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 04:41:40.912582 (+ 40us) server_negotiation.cc:1035] Waiting for connection context
0810 04:41:40.912641 (+ 59us) server_negotiation.cc:299] Negotiation successful
0810 04:41:40.912660 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260810 04:41:40.912757 4510 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:41:40.905963 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.66:34655 (local address 127.22.167.65:38357)
0810 04:41:40.910532 (+ 4569us) negotiation.cc:107] Waiting for socket to connect
0810 04:41:40.910543 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0810 04:41:40.910600 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 04:41:40.910716 (+ 116us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 04:41:40.910717 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 04:41:40.910718 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0810 04:41:40.910763 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:41:40.910765 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:41:40.911432 (+ 667us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:41:40.911434 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:41:40.912016 (+ 582us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:41:40.912018 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:41:40.912332 (+ 314us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:41:40.912333 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:41:40.912424 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:41:40.912440 (+ 16us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0810 04:41:40.912589 (+ 149us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0810 04:41:40.912591 (+ 2us) client_negotiation.cc:769] Sending connection context
0810 04:41:40.912638 (+ 47us) client_negotiation.cc:240] Negotiation successful
0810 04:41:40.912681 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4534,"thread_start_us":94,"threads_started":1}
I20260810 04:41:40.914790 4055 raft_consensus.cc:1261] T d9572be79ddc45e98a527612a51cdaf9 P 603becb479b7443f854a91ab3530293a [term 1 FOLLOWER]: Refusing update from remote peer 84e7718d636e41efbd3a4b32db54e3f8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 04:41:40.914781 4300 raft_consensus.cc:1261] T d9572be79ddc45e98a527612a51cdaf9 P 5fc86b14002f4fd3ae81df14a166c562 [term 1 FOLLOWER]: Refusing update from remote peer 84e7718d636e41efbd3a4b32db54e3f8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 04:41:40.915005 4434 consensus_queue.cc:1035] T d9572be79ddc45e98a527612a51cdaf9 P 84e7718d636e41efbd3a4b32db54e3f8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5fc86b14002f4fd3ae81df14a166c562" member_type: VOTER last_known_addr { host: "127.22.167.67" port: 33991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 04:41:40.915119 4458 consensus_queue.cc:1035] T d9572be79ddc45e98a527612a51cdaf9 P 84e7718d636e41efbd3a4b32db54e3f8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "603becb479b7443f854a91ab3530293a" member_type: VOTER last_known_addr { host: "127.22.167.66" port: 34655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 04:41:40.923633 4412 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:41:40.920206 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.167.66:38913 (local address 127.22.167.65:40941)
0810 04:41:40.920244 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0810 04:41:40.920246 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 04:41:40.920622 (+ 376us) server_negotiation.cc:407] Connection header received
0810 04:41:40.920739 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:41:40.920739 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:41:40.920749 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:41:40.920807 (+ 58us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 04:41:40.921057 (+ 250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:41:40.921768 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:41:40.922859 (+ 1091us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:41:40.923137 (+ 278us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:41:40.923208 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:41:40.923429 (+ 221us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 04:41:40.923463 (+ 34us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 04:41:40.923515 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0810 04:41:40.923550 (+ 35us) server_negotiation.cc:299] Negotiation successful
0810 04:41:40.923569 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260810 04:41:40.923741 4520 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:41:40.920217 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.65:40941 (local address 127.22.167.66:38913)
0810 04:41:40.920552 (+ 335us) negotiation.cc:107] Waiting for socket to connect
0810 04:41:40.920563 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0810 04:41:40.920621 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 04:41:40.920809 (+ 188us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 04:41:40.920810 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 04:41:40.920812 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0810 04:41:40.920998 (+ 186us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:41:40.921000 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:41:40.921836 (+ 836us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:41:40.921837 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:41:40.922793 (+ 956us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:41:40.922796 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:41:40.923265 (+ 469us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:41:40.923265 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:41:40.923344 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:41:40.923358 (+ 14us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0810 04:41:40.923516 (+ 158us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0810 04:41:40.923517 (+ 1us) client_negotiation.cc:769] Sending connection context
0810 04:41:40.923545 (+ 28us) client_negotiation.cc:240] Negotiation successful
0810 04:41:40.923575 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":278,"thread_start_us":204,"threads_started":1}
I20260810 04:41:40.932636 3596 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:52958:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 04:41:40.932723 3596 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:52958:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 04:41:40.933284 3596 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b8bbdb416f794faeb39ac8593eebe708: Sending DeleteTablet for 3 replicas of tablet d9572be79ddc45e98a527612a51cdaf9
I20260810 04:41:40.933537 4031 tablet_service.cc:1514] Processing DeleteTablet for tablet d9572be79ddc45e98a527612a51cdaf9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 04:41:40 UTC) from {username='oryx', principal='oryx/127.22.167.126@KRBTEST.COM'} at 127.0.0.1:50370
I20260810 04:41:40.933542 4267 tablet_service.cc:1514] Processing DeleteTablet for tablet d9572be79ddc45e98a527612a51cdaf9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 04:41:40 UTC) from {username='oryx', principal='oryx/127.22.167.126@KRBTEST.COM'} at 127.0.0.1:56580
I20260810 04:41:40.933542 3815 tablet_service.cc:1514] Processing DeleteTablet for tablet d9572be79ddc45e98a527612a51cdaf9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 04:41:40 UTC) from {username='oryx', principal='oryx/127.22.167.126@KRBTEST.COM'} at 127.0.0.1:50476
I20260810 04:41:40.933729 4526 tablet_replica.cc:330] T d9572be79ddc45e98a527612a51cdaf9 P 84e7718d636e41efbd3a4b32db54e3f8: stopping tablet replica
I20260810 04:41:40.933787 4526 raft_consensus.cc:2229] T d9572be79ddc45e98a527612a51cdaf9 P 84e7718d636e41efbd3a4b32db54e3f8 [term 1 LEADER]: Raft consensus shutting down.
I20260810 04:41:40.933856 4526 raft_consensus.cc:2258] T d9572be79ddc45e98a527612a51cdaf9 P 84e7718d636e41efbd3a4b32db54e3f8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 04:41:40.933995 4529 tablet_replica.cc:330] T d9572be79ddc45e98a527612a51cdaf9 P 603becb479b7443f854a91ab3530293a: stopping tablet replica
I20260810 04:41:40.934039 4529 raft_consensus.cc:2229] T d9572be79ddc45e98a527612a51cdaf9 P 603becb479b7443f854a91ab3530293a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 04:41:40.934085 4529 raft_consensus.cc:2258] T d9572be79ddc45e98a527612a51cdaf9 P 603becb479b7443f854a91ab3530293a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 04:41:40.937583 4529 ts_tablet_manager.cc:1905] T d9572be79ddc45e98a527612a51cdaf9 P 603becb479b7443f854a91ab3530293a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 04:41:40.938901 4526 ts_tablet_manager.cc:1905] T d9572be79ddc45e98a527612a51cdaf9 P 84e7718d636e41efbd3a4b32db54e3f8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 04:41:40.939442 4529 ts_tablet_manager.cc:1918] T d9572be79ddc45e98a527612a51cdaf9 P 603becb479b7443f854a91ab3530293a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260810 04:41:40.939571 4529 log.cc:1196] T d9572be79ddc45e98a527612a51cdaf9 P 603becb479b7443f854a91ab3530293a: Deleting WAL directory at /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786336863388938-23197-0/minicluster-data/ts-1/wal/wals/d9572be79ddc45e98a527612a51cdaf9
I20260810 04:41:40.939862 4529 ts_tablet_manager.cc:1939] T d9572be79ddc45e98a527612a51cdaf9 P 603becb479b7443f854a91ab3530293a: Deleting consensus metadata
I20260810 04:41:40.940176 3571 catalog_manager.cc:4841] TS 603becb479b7443f854a91ab3530293a (127.22.167.66:34655): tablet d9572be79ddc45e98a527612a51cdaf9 (table test-table [id=0107be0e7f5347308877d96fd1cbe0df]) successfully deleted
I20260810 04:41:40.940361 4524 tablet_replica.cc:330] T d9572be79ddc45e98a527612a51cdaf9 P 5fc86b14002f4fd3ae81df14a166c562: stopping tablet replica
I20260810 04:41:40.940416 4524 raft_consensus.cc:2229] T d9572be79ddc45e98a527612a51cdaf9 P 5fc86b14002f4fd3ae81df14a166c562 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 04:41:40.940479 4524 raft_consensus.cc:2258] T d9572be79ddc45e98a527612a51cdaf9 P 5fc86b14002f4fd3ae81df14a166c562 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 04:41:40.940841 4526 ts_tablet_manager.cc:1918] T d9572be79ddc45e98a527612a51cdaf9 P 84e7718d636e41efbd3a4b32db54e3f8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260810 04:41:40.940905 4526 log.cc:1196] T d9572be79ddc45e98a527612a51cdaf9 P 84e7718d636e41efbd3a4b32db54e3f8: Deleting WAL directory at /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786336863388938-23197-0/minicluster-data/ts-0/wal/wals/d9572be79ddc45e98a527612a51cdaf9
I20260810 04:41:40.941136 4526 ts_tablet_manager.cc:1939] T d9572be79ddc45e98a527612a51cdaf9 P 84e7718d636e41efbd3a4b32db54e3f8: Deleting consensus metadata
I20260810 04:41:40.941427 3573 catalog_manager.cc:4841] TS 84e7718d636e41efbd3a4b32db54e3f8 (127.22.167.65:40941): tablet d9572be79ddc45e98a527612a51cdaf9 (table test-table [id=0107be0e7f5347308877d96fd1cbe0df]) successfully deleted
I20260810 04:41:40.944162 4524 ts_tablet_manager.cc:1905] T d9572be79ddc45e98a527612a51cdaf9 P 5fc86b14002f4fd3ae81df14a166c562: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 04:41:40.945641 4524 ts_tablet_manager.cc:1918] T d9572be79ddc45e98a527612a51cdaf9 P 5fc86b14002f4fd3ae81df14a166c562: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260810 04:41:40.945693 4524 log.cc:1196] T d9572be79ddc45e98a527612a51cdaf9 P 5fc86b14002f4fd3ae81df14a166c562: Deleting WAL directory at /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786336863388938-23197-0/minicluster-data/ts-2/wal/wals/d9572be79ddc45e98a527612a51cdaf9
I20260810 04:41:40.945887 4524 ts_tablet_manager.cc:1939] T d9572be79ddc45e98a527612a51cdaf9 P 5fc86b14002f4fd3ae81df14a166c562: Deleting consensus metadata
I20260810 04:41:40.946365 3571 catalog_manager.cc:4841] TS 5fc86b14002f4fd3ae81df14a166c562 (127.22.167.67:33991): tablet d9572be79ddc45e98a527612a51cdaf9 (table test-table [id=0107be0e7f5347308877d96fd1cbe0df]) successfully deleted
Aug 10 04:41:40 dist-test-slave-bqcl krb5kdc[3517](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786336900, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 10 04:41:40 dist-test-slave-bqcl krb5kdc[3517](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.22.167.126@KRBTEST.COM, Server not found in Kerberos database
Aug 10 04:41:40 dist-test-slave-bqcl krb5kdc[3517](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.22.167.126@KRBTEST.COM, Server not found in Kerberos database
W20260810 04:41:40.965716 4484 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 04:41:40.960531 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52974 (local address 127.22.167.126:37713)
0810 04:41:40.960596 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0810 04:41:40.960598 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 04:41:40.960718 (+ 120us) server_negotiation.cc:407] Connection header received
0810 04:41:40.960779 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:41:40.960780 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:41:40.960792 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:41:40.960846 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0810 04:41:40.961451 (+ 605us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:41:40.961961 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:41:40.962744 (+ 783us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:41:40.962952 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:41:40.963008 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:41:40.965655 (+ 2647us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:52974: BlockingRecv error: recv got EOF from 127.0.0.1:52974 (error 108)
Metrics: {"server-negotiator.queue_time_us":25}
Aug 10 04:41:40 dist-test-slave-bqcl krb5kdc[3517](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786336900, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.22.167.126@KRBTEST.COM
I20260810 04:41:40.972735 4484 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:41:40.966409 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52986 (local address 127.22.167.126:37713)
0810 04:41:40.966450 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0810 04:41:40.966452 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 04:41:40.966461 (+ 9us) server_negotiation.cc:407] Connection header received
0810 04:41:40.966516 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:41:40.966517 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:41:40.966529 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:41:40.966568 (+ 39us) server_negotiation.cc:226] Negotiated authn=SASL
0810 04:41:40.966985 (+ 417us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:41:40.967440 (+ 455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:41:40.968079 (+ 639us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:41:40.968265 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:41:40.968317 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:41:40.970554 (+ 2237us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 04:41:40.970556 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 04:41:40.970556 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 04:41:40.970563 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 04:41:40.971770 (+ 1207us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 04:41:40.972071 (+ 301us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 04:41:40.972072 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 04:41:40.972072 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 04:41:40.972123 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 04:41:40.972288 (+ 165us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 04:41:40.972288 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 04:41:40.972289 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 04:41:40.972386 (+ 97us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 04:41:40.972474 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0810 04:41:40.972566 (+ 92us) server_negotiation.cc:299] Negotiation successful
0810 04:41:40.972641 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260810 04:41:40.974078 3596 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52986:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 04:41:40.974261 3596 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 04:41:40.975514 3815 tablet_service.cc:1467] Processing CreateTablet for tablet 0d33d900130f4142b550f269264bfbea (DEFAULT_TABLE table=test-table [id=9297c5f97b394df59e8fd3cea596a3a1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 04:41:40.975550 4031 tablet_service.cc:1467] Processing CreateTablet for tablet 0d33d900130f4142b550f269264bfbea (DEFAULT_TABLE table=test-table [id=9297c5f97b394df59e8fd3cea596a3a1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 04:41:40.975621 4267 tablet_service.cc:1467] Processing CreateTablet for tablet 0d33d900130f4142b550f269264bfbea (DEFAULT_TABLE table=test-table [id=9297c5f97b394df59e8fd3cea596a3a1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 04:41:40.975674 3815 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0d33d900130f4142b550f269264bfbea. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 04:41:40.975677 4031 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0d33d900130f4142b550f269264bfbea. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 04:41:40.975723 4267 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0d33d900130f4142b550f269264bfbea. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 04:41:40.976948 4422 tablet_bootstrap.cc:492] T 0d33d900130f4142b550f269264bfbea P 603becb479b7443f854a91ab3530293a: Bootstrap starting.
I20260810 04:41:40.977137 4426 tablet_bootstrap.cc:492] T 0d33d900130f4142b550f269264bfbea P 84e7718d636e41efbd3a4b32db54e3f8: Bootstrap starting.
I20260810 04:41:40.977336 4422 tablet_bootstrap.cc:654] T 0d33d900130f4142b550f269264bfbea P 603becb479b7443f854a91ab3530293a: Neither blocks nor log segments found. Creating new log.
I20260810 04:41:40.977547 4426 tablet_bootstrap.cc:654] T 0d33d900130f4142b550f269264bfbea P 84e7718d636e41efbd3a4b32db54e3f8: Neither blocks nor log segments found. Creating new log.
I20260810 04:41:40.977676 4425 tablet_bootstrap.cc:492] T 0d33d900130f4142b550f269264bfbea P 5fc86b14002f4fd3ae81df14a166c562: Bootstrap starting.
I20260810 04:41:40.978037 4425 tablet_bootstrap.cc:654] T 0d33d900130f4142b550f269264bfbea P 5fc86b14002f4fd3ae81df14a166c562: Neither blocks nor log segments found. Creating new log.
I20260810 04:41:40.978050 4422 tablet_bootstrap.cc:492] T 0d33d900130f4142b550f269264bfbea P 603becb479b7443f854a91ab3530293a: No bootstrap required, opened a new log
I20260810 04:41:40.978087 4422 ts_tablet_manager.cc:1397] T 0d33d900130f4142b550f269264bfbea P 603becb479b7443f854a91ab3530293a: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260810 04:41:40.978407 4422 raft_consensus.cc:348] T 0d33d900130f4142b550f269264bfbea P 603becb479b7443f854a91ab3530293a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84e7718d636e41efbd3a4b32db54e3f8" member_type: VOTER last_known_addr { host: "127.22.167.65" port: 40941 } } peers { permanent_uuid: "603becb479b7443f854a91ab3530293a" member_type: VOTER last_known_addr { host: "127.22.167.66" port: 34655 } } peers { permanent_uuid: "5fc86b14002f4fd3ae81df14a166c562" member_type: VOTER last_known_addr { host: "127.22.167.67" port: 33991 } }
I20260810 04:41:40.978490 4422 raft_consensus.cc:374] T 0d33d900130f4142b550f269264bfbea P 603becb479b7443f854a91ab3530293a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 04:41:40.978510 4422 raft_consensus.cc:729] T 0d33d900130f4142b550f269264bfbea P 603becb479b7443f854a91ab3530293a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 603becb479b7443f854a91ab3530293a, State: Initialized, Role: FOLLOWER
I20260810 04:41:40.978559 4422 consensus_queue.cc:260] T 0d33d900130f4142b550f269264bfbea P 603becb479b7443f854a91ab3530293a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84e7718d636e41efbd3a4b32db54e3f8" member_type: VOTER last_known_addr { host: "127.22.167.65" port: 40941 } } peers { permanent_uuid: "603becb479b7443f854a91ab3530293a" member_type: VOTER last_known_addr { host: "127.22.167.66" port: 34655 } } peers { permanent_uuid: "5fc86b14002f4fd3ae81df14a166c562" member_type: VOTER last_known_addr { host: "127.22.167.67" port: 33991 } }
I20260810 04:41:40.978821 4422 ts_tablet_manager.cc:1428] T 0d33d900130f4142b550f269264bfbea P 603becb479b7443f854a91ab3530293a: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260810 04:41:40.978891 4425 tablet_bootstrap.cc:492] T 0d33d900130f4142b550f269264bfbea P 5fc86b14002f4fd3ae81df14a166c562: No bootstrap required, opened a new log
I20260810 04:41:40.978938 4425 ts_tablet_manager.cc:1397] T 0d33d900130f4142b550f269264bfbea P 5fc86b14002f4fd3ae81df14a166c562: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260810 04:41:40.979071 4425 raft_consensus.cc:348] T 0d33d900130f4142b550f269264bfbea P 5fc86b14002f4fd3ae81df14a166c562 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84e7718d636e41efbd3a4b32db54e3f8" member_type: VOTER last_known_addr { host: "127.22.167.65" port: 40941 } } peers { permanent_uuid: "603becb479b7443f854a91ab3530293a" member_type: VOTER last_known_addr { host: "127.22.167.66" port: 34655 } } peers { permanent_uuid: "5fc86b14002f4fd3ae81df14a166c562" member_type: VOTER last_known_addr { host: "127.22.167.67" port: 33991 } }
I20260810 04:41:40.979136 4425 raft_consensus.cc:374] T 0d33d900130f4142b550f269264bfbea P 5fc86b14002f4fd3ae81df14a166c562 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 04:41:40.979153 4425 raft_consensus.cc:729] T 0d33d900130f4142b550f269264bfbea P 5fc86b14002f4fd3ae81df14a166c562 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5fc86b14002f4fd3ae81df14a166c562, State: Initialized, Role: FOLLOWER
I20260810 04:41:40.979192 4425 consensus_queue.cc:260] T 0d33d900130f4142b550f269264bfbea P 5fc86b14002f4fd3ae81df14a166c562 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84e7718d636e41efbd3a4b32db54e3f8" member_type: VOTER last_known_addr { host: "127.22.167.65" port: 40941 } } peers { permanent_uuid: "603becb479b7443f854a91ab3530293a" member_type: VOTER last_known_addr { host: "127.22.167.66" port: 34655 } } peers { permanent_uuid: "5fc86b14002f4fd3ae81df14a166c562" member_type: VOTER last_known_addr { host: "127.22.167.67" port: 33991 } }
I20260810 04:41:40.979275 4425 ts_tablet_manager.cc:1428] T 0d33d900130f4142b550f269264bfbea P 5fc86b14002f4fd3ae81df14a166c562: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260810 04:41:40.979660 4426 tablet_bootstrap.cc:492] T 0d33d900130f4142b550f269264bfbea P 84e7718d636e41efbd3a4b32db54e3f8: No bootstrap required, opened a new log
I20260810 04:41:40.979708 4426 ts_tablet_manager.cc:1397] T 0d33d900130f4142b550f269264bfbea P 84e7718d636e41efbd3a4b32db54e3f8: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.001s
I20260810 04:41:40.979806 4426 raft_consensus.cc:348] T 0d33d900130f4142b550f269264bfbea P 84e7718d636e41efbd3a4b32db54e3f8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84e7718d636e41efbd3a4b32db54e3f8" member_type: VOTER last_known_addr { host: "127.22.167.65" port: 40941 } } peers { permanent_uuid: "603becb479b7443f854a91ab3530293a" member_type: VOTER last_known_addr { host: "127.22.167.66" port: 34655 } } peers { permanent_uuid: "5fc86b14002f4fd3ae81df14a166c562" member_type: VOTER last_known_addr { host: "127.22.167.67" port: 33991 } }
I20260810 04:41:40.979856 4426 raft_consensus.cc:374] T 0d33d900130f4142b550f269264bfbea P 84e7718d636e41efbd3a4b32db54e3f8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 04:41:40.979866 4426 raft_consensus.cc:729] T 0d33d900130f4142b550f269264bfbea P 84e7718d636e41efbd3a4b32db54e3f8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84e7718d636e41efbd3a4b32db54e3f8, State: Initialized, Role: FOLLOWER
I20260810 04:41:40.979899 4426 consensus_queue.cc:260] T 0d33d900130f4142b550f269264bfbea P 84e7718d636e41efbd3a4b32db54e3f8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84e7718d636e41efbd3a4b32db54e3f8" member_type: VOTER last_known_addr { host: "127.22.167.65" port: 40941 } } peers { permanent_uuid: "603becb479b7443f854a91ab3530293a" member_type: VOTER last_known_addr { host: "127.22.167.66" port: 34655 } } peers { permanent_uuid: "5fc86b14002f4fd3ae81df14a166c562" member_type: VOTER last_known_addr { host: "127.22.167.67" port: 33991 } }
I20260810 04:41:40.979979 4426 ts_tablet_manager.cc:1428] T 0d33d900130f4142b550f269264bfbea P 84e7718d636e41efbd3a4b32db54e3f8: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260810 04:41:41.158471 4434 raft_consensus.cc:482] T 0d33d900130f4142b550f269264bfbea P 84e7718d636e41efbd3a4b32db54e3f8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 04:41:41.158569 4434 raft_consensus.cc:504] T 0d33d900130f4142b550f269264bfbea P 84e7718d636e41efbd3a4b32db54e3f8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84e7718d636e41efbd3a4b32db54e3f8" member_type: VOTER last_known_addr { host: "127.22.167.65" port: 40941 } } peers { permanent_uuid: "603becb479b7443f854a91ab3530293a" member_type: VOTER last_known_addr { host: "127.22.167.66" port: 34655 } } peers { permanent_uuid: "5fc86b14002f4fd3ae81df14a166c562" member_type: VOTER last_known_addr { host: "127.22.167.67" port: 33991 } }
I20260810 04:41:41.158713 4434 leader_election.cc:288] T 0d33d900130f4142b550f269264bfbea P 84e7718d636e41efbd3a4b32db54e3f8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 603becb479b7443f854a91ab3530293a (127.22.167.66:34655), 5fc86b14002f4fd3ae81df14a166c562 (127.22.167.67:33991)
I20260810 04:41:41.158912 4055 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0d33d900130f4142b550f269264bfbea" candidate_uuid: "84e7718d636e41efbd3a4b32db54e3f8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "603becb479b7443f854a91ab3530293a" is_pre_election: true
I20260810 04:41:41.158931 4300 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0d33d900130f4142b550f269264bfbea" candidate_uuid: "84e7718d636e41efbd3a4b32db54e3f8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5fc86b14002f4fd3ae81df14a166c562" is_pre_election: true
I20260810 04:41:41.159015 4055 raft_consensus.cc:2454] T 0d33d900130f4142b550f269264bfbea P 603becb479b7443f854a91ab3530293a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84e7718d636e41efbd3a4b32db54e3f8 in term 0.
I20260810 04:41:41.159015 4300 raft_consensus.cc:2454] T 0d33d900130f4142b550f269264bfbea P 5fc86b14002f4fd3ae81df14a166c562 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84e7718d636e41efbd3a4b32db54e3f8 in term 0.
I20260810 04:41:41.159188 3731 leader_election.cc:302] T 0d33d900130f4142b550f269264bfbea P 84e7718d636e41efbd3a4b32db54e3f8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 603becb479b7443f854a91ab3530293a, 84e7718d636e41efbd3a4b32db54e3f8; no voters:
I20260810 04:41:41.159291 4434 raft_consensus.cc:2781] T 0d33d900130f4142b550f269264bfbea P 84e7718d636e41efbd3a4b32db54e3f8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 04:41:41.159361 4434 raft_consensus.cc:482] T 0d33d900130f4142b550f269264bfbea P 84e7718d636e41efbd3a4b32db54e3f8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 04:41:41.159384 4434 raft_consensus.cc:3037] T 0d33d900130f4142b550f269264bfbea P 84e7718d636e41efbd3a4b32db54e3f8 [term 0 FOLLOWER]: Advancing to term 1
I20260810 04:41:41.160069 4434 raft_consensus.cc:504] T 0d33d900130f4142b550f269264bfbea P 84e7718d636e41efbd3a4b32db54e3f8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84e7718d636e41efbd3a4b32db54e3f8" member_type: VOTER last_known_addr { host: "127.22.167.65" port: 40941 } } peers { permanent_uuid: "603becb479b7443f854a91ab3530293a" member_type: VOTER last_known_addr { host: "127.22.167.66" port: 34655 } } peers { permanent_uuid: "5fc86b14002f4fd3ae81df14a166c562" member_type: VOTER last_known_addr { host: "127.22.167.67" port: 33991 } }
I20260810 04:41:41.160184 4434 leader_election.cc:288] T 0d33d900130f4142b550f269264bfbea P 84e7718d636e41efbd3a4b32db54e3f8 [CANDIDATE]: Term 1 election: Requested vote from peers 603becb479b7443f854a91ab3530293a (127.22.167.66:34655), 5fc86b14002f4fd3ae81df14a166c562 (127.22.167.67:33991)
I20260810 04:41:41.160354 4300 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0d33d900130f4142b550f269264bfbea" candidate_uuid: "84e7718d636e41efbd3a4b32db54e3f8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5fc86b14002f4fd3ae81df14a166c562"
I20260810 04:41:41.160363 4055 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0d33d900130f4142b550f269264bfbea" candidate_uuid: "84e7718d636e41efbd3a4b32db54e3f8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "603becb479b7443f854a91ab3530293a"
I20260810 04:41:41.160434 4300 raft_consensus.cc:3037] T 0d33d900130f4142b550f269264bfbea P 5fc86b14002f4fd3ae81df14a166c562 [term 0 FOLLOWER]: Advancing to term 1
I20260810 04:41:41.160434 4055 raft_consensus.cc:3037] T 0d33d900130f4142b550f269264bfbea P 603becb479b7443f854a91ab3530293a [term 0 FOLLOWER]: Advancing to term 1
I20260810 04:41:41.161042 4055 raft_consensus.cc:2454] T 0d33d900130f4142b550f269264bfbea P 603becb479b7443f854a91ab3530293a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 84e7718d636e41efbd3a4b32db54e3f8 in term 1.
I20260810 04:41:41.161049 4300 raft_consensus.cc:2454] T 0d33d900130f4142b550f269264bfbea P 5fc86b14002f4fd3ae81df14a166c562 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 84e7718d636e41efbd3a4b32db54e3f8 in term 1.
I20260810 04:41:41.161217 3731 leader_election.cc:302] T 0d33d900130f4142b550f269264bfbea P 84e7718d636e41efbd3a4b32db54e3f8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 603becb479b7443f854a91ab3530293a, 84e7718d636e41efbd3a4b32db54e3f8; no voters:
I20260810 04:41:41.161319 4434 raft_consensus.cc:2781] T 0d33d900130f4142b550f269264bfbea P 84e7718d636e41efbd3a4b32db54e3f8 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 04:41:41.161384 4434 raft_consensus.cc:686] T 0d33d900130f4142b550f269264bfbea P 84e7718d636e41efbd3a4b32db54e3f8 [term 1 LEADER]: Becoming Leader. State: Replica: 84e7718d636e41efbd3a4b32db54e3f8, State: Running, Role: LEADER
I20260810 04:41:41.161437 4434 consensus_queue.cc:237] T 0d33d900130f4142b550f269264bfbea P 84e7718d636e41efbd3a4b32db54e3f8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84e7718d636e41efbd3a4b32db54e3f8" member_type: VOTER last_known_addr { host: "127.22.167.65" port: 40941 } } peers { permanent_uuid: "603becb479b7443f854a91ab3530293a" member_type: VOTER last_known_addr { host: "127.22.167.66" port: 34655 } } peers { permanent_uuid: "5fc86b14002f4fd3ae81df14a166c562" member_type: VOTER last_known_addr { host: "127.22.167.67" port: 33991 } }
I20260810 04:41:41.161996 3596 catalog_manager.cc:5495] T 0d33d900130f4142b550f269264bfbea P 84e7718d636e41efbd3a4b32db54e3f8 reported cstate change: term changed from 0 to 1, leader changed from <none> to 84e7718d636e41efbd3a4b32db54e3f8 (127.22.167.65). New cstate: current_term: 1 leader_uuid: "84e7718d636e41efbd3a4b32db54e3f8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84e7718d636e41efbd3a4b32db54e3f8" member_type: VOTER last_known_addr { host: "127.22.167.65" port: 40941 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "603becb479b7443f854a91ab3530293a" member_type: VOTER last_known_addr { host: "127.22.167.66" port: 34655 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5fc86b14002f4fd3ae81df14a166c562" member_type: VOTER last_known_addr { host: "127.22.167.67" port: 33991 } health_report { overall_health: UNKNOWN } } }
I20260810 04:41:41.204075 4412 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:41:41.202205 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50490 (local address 127.22.167.65:40941)
0810 04:41:41.202260 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0810 04:41:41.202262 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 04:41:41.202271 (+ 9us) server_negotiation.cc:407] Connection header received
0810 04:41:41.202293 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:41:41.202293 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:41:41.202305 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:41:41.202352 (+ 47us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 04:41:41.202460 (+ 108us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:41:41.202917 (+ 457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:41:41.203596 (+ 679us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:41:41.203766 (+ 170us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:41:41.203808 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:41:41.203904 (+ 96us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 04:41:41.203928 (+ 24us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 04:41:41.203958 (+ 30us) server_negotiation.cc:1035] Waiting for connection context
0810 04:41:41.203996 (+ 38us) server_negotiation.cc:299] Negotiation successful
0810 04:41:41.204014 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260810 04:41:41.204836 4055 raft_consensus.cc:1261] T 0d33d900130f4142b550f269264bfbea P 603becb479b7443f854a91ab3530293a [term 1 FOLLOWER]: Refusing update from remote peer 84e7718d636e41efbd3a4b32db54e3f8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 04:41:41.204843 4300 raft_consensus.cc:1261] T 0d33d900130f4142b550f269264bfbea P 5fc86b14002f4fd3ae81df14a166c562 [term 1 FOLLOWER]: Refusing update from remote peer 84e7718d636e41efbd3a4b32db54e3f8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 04:41:41.205052 4458 consensus_queue.cc:1035] T 0d33d900130f4142b550f269264bfbea P 84e7718d636e41efbd3a4b32db54e3f8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "603becb479b7443f854a91ab3530293a" member_type: VOTER last_known_addr { host: "127.22.167.66" port: 34655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 04:41:41.205158 4434 consensus_queue.cc:1035] T 0d33d900130f4142b550f269264bfbea P 84e7718d636e41efbd3a4b32db54e3f8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5fc86b14002f4fd3ae81df14a166c562" member_type: VOTER last_known_addr { host: "127.22.167.67" port: 33991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 04:41:41.207567 3597 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52986:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 04:41:41.207650 3597 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52986:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 04:41:41.208266 3597 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b8bbdb416f794faeb39ac8593eebe708: Sending DeleteTablet for 3 replicas of tablet 0d33d900130f4142b550f269264bfbea
I20260810 04:41:41.208496 4031 tablet_service.cc:1514] Processing DeleteTablet for tablet 0d33d900130f4142b550f269264bfbea with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 04:41:41 UTC) from {username='oryx', principal='oryx/127.22.167.126@KRBTEST.COM'} at 127.0.0.1:50370
I20260810 04:41:41.208520 4267 tablet_service.cc:1514] Processing DeleteTablet for tablet 0d33d900130f4142b550f269264bfbea with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 04:41:41 UTC) from {username='oryx', principal='oryx/127.22.167.126@KRBTEST.COM'} at 127.0.0.1:56580
I20260810 04:41:41.208590 4529 tablet_replica.cc:330] T 0d33d900130f4142b550f269264bfbea P 603becb479b7443f854a91ab3530293a: stopping tablet replica
I20260810 04:41:41.208590 4524 tablet_replica.cc:330] T 0d33d900130f4142b550f269264bfbea P 5fc86b14002f4fd3ae81df14a166c562: stopping tablet replica
I20260810 04:41:41.208630 4524 raft_consensus.cc:2229] T 0d33d900130f4142b550f269264bfbea P 5fc86b14002f4fd3ae81df14a166c562 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 04:41:41.208635 4529 raft_consensus.cc:2229] T 0d33d900130f4142b550f269264bfbea P 603becb479b7443f854a91ab3530293a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 04:41:41.208667 4524 raft_consensus.cc:2258] T 0d33d900130f4142b550f269264bfbea P 5fc86b14002f4fd3ae81df14a166c562 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 04:41:41.208628 3815 tablet_service.cc:1514] Processing DeleteTablet for tablet 0d33d900130f4142b550f269264bfbea with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 04:41:41 UTC) from {username='oryx', principal='oryx/127.22.167.126@KRBTEST.COM'} at 127.0.0.1:50476
I20260810 04:41:41.208675 4529 raft_consensus.cc:2258] T 0d33d900130f4142b550f269264bfbea P 603becb479b7443f854a91ab3530293a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 04:41:41.208750 4526 tablet_replica.cc:330] T 0d33d900130f4142b550f269264bfbea P 84e7718d636e41efbd3a4b32db54e3f8: stopping tablet replica
I20260810 04:41:41.208724 3597 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52986:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260810 04:41:41.208812 4526 raft_consensus.cc:2229] T 0d33d900130f4142b550f269264bfbea P 84e7718d636e41efbd3a4b32db54e3f8 [term 1 LEADER]: Raft consensus shutting down.
W20260810 04:41:41.208853 3597 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 04:41:41.208864 4526 raft_consensus.cc:2258] T 0d33d900130f4142b550f269264bfbea P 84e7718d636e41efbd3a4b32db54e3f8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 04:41:41.209213 4524 ts_tablet_manager.cc:1905] T 0d33d900130f4142b550f269264bfbea P 5fc86b14002f4fd3ae81df14a166c562: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 04:41:41.209229 4526 ts_tablet_manager.cc:1905] T 0d33d900130f4142b550f269264bfbea P 84e7718d636e41efbd3a4b32db54e3f8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 04:41:41.210320 4529 ts_tablet_manager.cc:1905] T 0d33d900130f4142b550f269264bfbea P 603becb479b7443f854a91ab3530293a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 04:41:41.210590 4526 ts_tablet_manager.cc:1918] T 0d33d900130f4142b550f269264bfbea P 84e7718d636e41efbd3a4b32db54e3f8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 04:41:41.210644 4526 log.cc:1196] T 0d33d900130f4142b550f269264bfbea P 84e7718d636e41efbd3a4b32db54e3f8: Deleting WAL directory at /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786336863388938-23197-0/minicluster-data/ts-0/wal/wals/0d33d900130f4142b550f269264bfbea
I20260810 04:41:41.210677 4524 ts_tablet_manager.cc:1918] T 0d33d900130f4142b550f269264bfbea P 5fc86b14002f4fd3ae81df14a166c562: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 04:41:41.210752 4524 log.cc:1196] T 0d33d900130f4142b550f269264bfbea P 5fc86b14002f4fd3ae81df14a166c562: Deleting WAL directory at /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786336863388938-23197-0/minicluster-data/ts-2/wal/wals/0d33d900130f4142b550f269264bfbea
I20260810 04:41:41.210832 4526 ts_tablet_manager.cc:1939] T 0d33d900130f4142b550f269264bfbea P 84e7718d636e41efbd3a4b32db54e3f8: Deleting consensus metadata
I20260810 04:41:41.210966 4524 ts_tablet_manager.cc:1939] T 0d33d900130f4142b550f269264bfbea P 5fc86b14002f4fd3ae81df14a166c562: Deleting consensus metadata
I20260810 04:41:41.210973 4267 tablet_service.cc:1467] Processing CreateTablet for tablet 8d49e1ccae4546f59c658baa66310f40 (DEFAULT_TABLE table=test-table [id=4a53590cd77c4a169e78e0dc82489f40]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 04:41:41.211133 4031 tablet_service.cc:1467] Processing CreateTablet for tablet 8d49e1ccae4546f59c658baa66310f40 (DEFAULT_TABLE table=test-table [id=4a53590cd77c4a169e78e0dc82489f40]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 04:41:41.211228 4031 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8d49e1ccae4546f59c658baa66310f40. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 04:41:41.211735 4529 ts_tablet_manager.cc:1918] T 0d33d900130f4142b550f269264bfbea P 603becb479b7443f854a91ab3530293a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 04:41:41.211787 4529 log.cc:1196] T 0d33d900130f4142b550f269264bfbea P 603becb479b7443f854a91ab3530293a: Deleting WAL directory at /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786336863388938-23197-0/minicluster-data/ts-1/wal/wals/0d33d900130f4142b550f269264bfbea
I20260810 04:41:41.211962 4529 ts_tablet_manager.cc:1939] T 0d33d900130f4142b550f269264bfbea P 603becb479b7443f854a91ab3530293a: Deleting consensus metadata
I20260810 04:41:41.212291 4422 tablet_bootstrap.cc:492] T 8d49e1ccae4546f59c658baa66310f40 P 603becb479b7443f854a91ab3530293a: Bootstrap starting.
I20260810 04:41:41.212330 3571 catalog_manager.cc:4841] TS 603becb479b7443f854a91ab3530293a (127.22.167.66:34655): tablet 0d33d900130f4142b550f269264bfbea (table test-table [id=9297c5f97b394df59e8fd3cea596a3a1]) successfully deleted
I20260810 04:41:41.212378 3573 catalog_manager.cc:4841] TS 84e7718d636e41efbd3a4b32db54e3f8 (127.22.167.65:40941): tablet 0d33d900130f4142b550f269264bfbea (table test-table [id=9297c5f97b394df59e8fd3cea596a3a1]) successfully deleted
I20260810 04:41:41.212450 3815 tablet_service.cc:1467] Processing CreateTablet for tablet 8d49e1ccae4546f59c658baa66310f40 (DEFAULT_TABLE table=test-table [id=4a53590cd77c4a169e78e0dc82489f40]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 04:41:41.212499 4267 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8d49e1ccae4546f59c658baa66310f40. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 04:41:41.212545 3815 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8d49e1ccae4546f59c658baa66310f40. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 04:41:41.212586 4422 tablet_bootstrap.cc:654] T 8d49e1ccae4546f59c658baa66310f40 P 603becb479b7443f854a91ab3530293a: Neither blocks nor log segments found. Creating new log.
I20260810 04:41:41.213151 4422 tablet_bootstrap.cc:492] T 8d49e1ccae4546f59c658baa66310f40 P 603becb479b7443f854a91ab3530293a: No bootstrap required, opened a new log
I20260810 04:41:41.213217 4422 ts_tablet_manager.cc:1397] T 8d49e1ccae4546f59c658baa66310f40 P 603becb479b7443f854a91ab3530293a: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260810 04:41:41.213335 4422 raft_consensus.cc:348] T 8d49e1ccae4546f59c658baa66310f40 P 603becb479b7443f854a91ab3530293a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84e7718d636e41efbd3a4b32db54e3f8" member_type: VOTER last_known_addr { host: "127.22.167.65" port: 40941 } } peers { permanent_uuid: "603becb479b7443f854a91ab3530293a" member_type: VOTER last_known_addr { host: "127.22.167.66" port: 34655 } } peers { permanent_uuid: "5fc86b14002f4fd3ae81df14a166c562" member_type: VOTER last_known_addr { host: "127.22.167.67" port: 33991 } }
I20260810 04:41:41.213389 4422 raft_consensus.cc:374] T 8d49e1ccae4546f59c658baa66310f40 P 603becb479b7443f854a91ab3530293a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 04:41:41.213407 4422 raft_consensus.cc:729] T 8d49e1ccae4546f59c658baa66310f40 P 603becb479b7443f854a91ab3530293a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 603becb479b7443f854a91ab3530293a, State: Initialized, Role: FOLLOWER
I20260810 04:41:41.213444 4422 consensus_queue.cc:260] T 8d49e1ccae4546f59c658baa66310f40 P 603becb479b7443f854a91ab3530293a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84e7718d636e41efbd3a4b32db54e3f8" member_type: VOTER last_known_addr { host: "127.22.167.65" port: 40941 } } peers { permanent_uuid: "603becb479b7443f854a91ab3530293a" member_type: VOTER last_known_addr { host: "127.22.167.66" port: 34655 } } peers { permanent_uuid: "5fc86b14002f4fd3ae81df14a166c562" member_type: VOTER last_known_addr { host: "127.22.167.67" port: 33991 } }
I20260810 04:41:41.213469 4426 tablet_bootstrap.cc:492] T 8d49e1ccae4546f59c658baa66310f40 P 84e7718d636e41efbd3a4b32db54e3f8: Bootstrap starting.
I20260810 04:41:41.213524 4422 ts_tablet_manager.cc:1428] T 8d49e1ccae4546f59c658baa66310f40 P 603becb479b7443f854a91ab3530293a: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260810 04:41:41.213696 4425 tablet_bootstrap.cc:492] T 8d49e1ccae4546f59c658baa66310f40 P 5fc86b14002f4fd3ae81df14a166c562: Bootstrap starting.
I20260810 04:41:41.213757 4426 tablet_bootstrap.cc:654] T 8d49e1ccae4546f59c658baa66310f40 P 84e7718d636e41efbd3a4b32db54e3f8: Neither blocks nor log segments found. Creating new log.
I20260810 04:41:41.213783 3571 catalog_manager.cc:4841] TS 5fc86b14002f4fd3ae81df14a166c562 (127.22.167.67:33991): tablet 0d33d900130f4142b550f269264bfbea (table test-table [id=9297c5f97b394df59e8fd3cea596a3a1]) successfully deleted
I20260810 04:41:41.214051 4425 tablet_bootstrap.cc:654] T 8d49e1ccae4546f59c658baa66310f40 P 5fc86b14002f4fd3ae81df14a166c562: Neither blocks nor log segments found. Creating new log.
I20260810 04:41:41.214277 4426 tablet_bootstrap.cc:492] T 8d49e1ccae4546f59c658baa66310f40 P 84e7718d636e41efbd3a4b32db54e3f8: No bootstrap required, opened a new log
I20260810 04:41:41.214318 4426 ts_tablet_manager.cc:1397] T 8d49e1ccae4546f59c658baa66310f40 P 84e7718d636e41efbd3a4b32db54e3f8: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260810 04:41:41.214411 4426 raft_consensus.cc:348] T 8d49e1ccae4546f59c658baa66310f40 P 84e7718d636e41efbd3a4b32db54e3f8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84e7718d636e41efbd3a4b32db54e3f8" member_type: VOTER last_known_addr { host: "127.22.167.65" port: 40941 } } peers { permanent_uuid: "603becb479b7443f854a91ab3530293a" member_type: VOTER last_known_addr { host: "127.22.167.66" port: 34655 } } peers { permanent_uuid: "5fc86b14002f4fd3ae81df14a166c562" member_type: VOTER last_known_addr { host: "127.22.167.67" port: 33991 } }
I20260810 04:41:41.214478 4426 raft_consensus.cc:374] T 8d49e1ccae4546f59c658baa66310f40 P 84e7718d636e41efbd3a4b32db54e3f8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 04:41:41.214496 4426 raft_consensus.cc:729] T 8d49e1ccae4546f59c658baa66310f40 P 84e7718d636e41efbd3a4b32db54e3f8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84e7718d636e41efbd3a4b32db54e3f8, State: Initialized, Role: FOLLOWER
I20260810 04:41:41.214535 4426 consensus_queue.cc:260] T 8d49e1ccae4546f59c658baa66310f40 P 84e7718d636e41efbd3a4b32db54e3f8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84e7718d636e41efbd3a4b32db54e3f8" member_type: VOTER last_known_addr { host: "127.22.167.65" port: 40941 } } peers { permanent_uuid: "603becb479b7443f854a91ab3530293a" member_type: VOTER last_known_addr { host: "127.22.167.66" port: 34655 } } peers { permanent_uuid: "5fc86b14002f4fd3ae81df14a166c562" member_type: VOTER last_known_addr { host: "127.22.167.67" port: 33991 } }
I20260810 04:41:41.214620 4426 ts_tablet_manager.cc:1428] T 8d49e1ccae4546f59c658baa66310f40 P 84e7718d636e41efbd3a4b32db54e3f8: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260810 04:41:41.214638 4425 tablet_bootstrap.cc:492] T 8d49e1ccae4546f59c658baa66310f40 P 5fc86b14002f4fd3ae81df14a166c562: No bootstrap required, opened a new log
I20260810 04:41:41.214675 4425 ts_tablet_manager.cc:1397] T 8d49e1ccae4546f59c658baa66310f40 P 5fc86b14002f4fd3ae81df14a166c562: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260810 04:41:41.214800 4425 raft_consensus.cc:348] T 8d49e1ccae4546f59c658baa66310f40 P 5fc86b14002f4fd3ae81df14a166c562 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84e7718d636e41efbd3a4b32db54e3f8" member_type: VOTER last_known_addr { host: "127.22.167.65" port: 40941 } } peers { permanent_uuid: "603becb479b7443f854a91ab3530293a" member_type: VOTER last_known_addr { host: "127.22.167.66" port: 34655 } } peers { permanent_uuid: "5fc86b14002f4fd3ae81df14a166c562" member_type: VOTER last_known_addr { host: "127.22.167.67" port: 33991 } }
I20260810 04:41:41.214854 4425 raft_consensus.cc:374] T 8d49e1ccae4546f59c658baa66310f40 P 5fc86b14002f4fd3ae81df14a166c562 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 04:41:41.214871 4425 raft_consensus.cc:729] T 8d49e1ccae4546f59c658baa66310f40 P 5fc86b14002f4fd3ae81df14a166c562 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5fc86b14002f4fd3ae81df14a166c562, State: Initialized, Role: FOLLOWER
I20260810 04:41:41.214907 4425 consensus_queue.cc:260] T 8d49e1ccae4546f59c658baa66310f40 P 5fc86b14002f4fd3ae81df14a166c562 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84e7718d636e41efbd3a4b32db54e3f8" member_type: VOTER last_known_addr { host: "127.22.167.65" port: 40941 } } peers { permanent_uuid: "603becb479b7443f854a91ab3530293a" member_type: VOTER last_known_addr { host: "127.22.167.66" port: 34655 } } peers { permanent_uuid: "5fc86b14002f4fd3ae81df14a166c562" member_type: VOTER last_known_addr { host: "127.22.167.67" port: 33991 } }
I20260810 04:41:41.214993 4425 ts_tablet_manager.cc:1428] T 8d49e1ccae4546f59c658baa66310f40 P 5fc86b14002f4fd3ae81df14a166c562: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260810 04:41:41.249914 4433 raft_consensus.cc:482] T 8d49e1ccae4546f59c658baa66310f40 P 5fc86b14002f4fd3ae81df14a166c562 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 04:41:41.250010 4433 raft_consensus.cc:504] T 8d49e1ccae4546f59c658baa66310f40 P 5fc86b14002f4fd3ae81df14a166c562 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84e7718d636e41efbd3a4b32db54e3f8" member_type: VOTER last_known_addr { host: "127.22.167.65" port: 40941 } } peers { permanent_uuid: "603becb479b7443f854a91ab3530293a" member_type: VOTER last_known_addr { host: "127.22.167.66" port: 34655 } } peers { permanent_uuid: "5fc86b14002f4fd3ae81df14a166c562" member_type: VOTER last_known_addr { host: "127.22.167.67" port: 33991 } }
I20260810 04:41:41.250397 4433 leader_election.cc:288] T 8d49e1ccae4546f59c658baa66310f40 P 5fc86b14002f4fd3ae81df14a166c562 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 84e7718d636e41efbd3a4b32db54e3f8 (127.22.167.65:40941), 603becb479b7443f854a91ab3530293a (127.22.167.66:34655)
I20260810 04:41:41.253258 4412 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:41:41.250606 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.167.67:55425 (local address 127.22.167.65:40941)
0810 04:41:41.250652 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0810 04:41:41.250655 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 04:41:41.250752 (+ 97us) server_negotiation.cc:407] Connection header received
0810 04:41:41.250808 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:41:41.250809 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:41:41.250818 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:41:41.250864 (+ 46us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 04:41:41.250981 (+ 117us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:41:41.251449 (+ 468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:41:41.252589 (+ 1140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:41:41.252951 (+ 362us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:41:41.253012 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:41:41.253022 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0810 04:41:41.253150 (+ 128us) server_negotiation.cc:299] Negotiation successful
0810 04:41:41.253187 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260810 04:41:41.253271 4570 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:41:41.250543 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.65:40941 (local address 127.22.167.67:55425)
0810 04:41:41.250683 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0810 04:41:41.250694 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0810 04:41:41.250756 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 04:41:41.250871 (+ 115us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 04:41:41.250872 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 04:41:41.250874 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 04:41:41.250930 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:41:41.250931 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:41:41.251573 (+ 642us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:41:41.251574 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:41:41.252503 (+ 929us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:41:41.252506 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:41:41.253037 (+ 531us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:41:41.253039 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:41:41.253091 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:41:41.253095 (+ 4us) client_negotiation.cc:769] Sending connection context
0810 04:41:41.253139 (+ 44us) client_negotiation.cc:240] Negotiation successful
0810 04:41:41.253179 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":126,"mutex_wait_us":47,"thread_start_us":65,"threads_started":1}
I20260810 04:41:41.253525 3848 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8d49e1ccae4546f59c658baa66310f40" candidate_uuid: "5fc86b14002f4fd3ae81df14a166c562" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84e7718d636e41efbd3a4b32db54e3f8" is_pre_election: true
I20260810 04:41:41.253580 4501 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:41:41.250631 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.167.67:33421 (local address 127.22.167.66:34655)
0810 04:41:41.250705 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0810 04:41:41.250707 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 04:41:41.250760 (+ 53us) server_negotiation.cc:407] Connection header received
0810 04:41:41.250834 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:41:41.250835 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:41:41.250847 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:41:41.250887 (+ 40us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 04:41:41.251032 (+ 145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:41:41.251487 (+ 455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:41:41.252874 (+ 1387us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:41:41.253239 (+ 365us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:41:41.253301 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:41:41.253309 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0810 04:41:41.253478 (+ 169us) server_negotiation.cc:299] Negotiation successful
0810 04:41:41.253525 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260810 04:41:41.253620 3848 raft_consensus.cc:2454] T 8d49e1ccae4546f59c658baa66310f40 P 84e7718d636e41efbd3a4b32db54e3f8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5fc86b14002f4fd3ae81df14a166c562 in term 0.
I20260810 04:41:41.253803 4567 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:41:41.250563 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.66:34655 (local address 127.22.167.67:33421)
0810 04:41:41.250692 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0810 04:41:41.250703 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0810 04:41:41.250774 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 04:41:41.250895 (+ 121us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 04:41:41.250896 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 04:41:41.250898 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 04:41:41.250972 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:41:41.250974 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:41:41.251575 (+ 601us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:41:41.251576 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:41:41.252779 (+ 1203us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:41:41.252781 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:41:41.253309 (+ 528us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:41:41.253310 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:41:41.253390 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:41:41.253393 (+ 3us) client_negotiation.cc:769] Sending connection context
0810 04:41:41.253688 (+ 295us) client_negotiation.cc:240] Negotiation successful
0810 04:41:41.253731 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":60,"threads_started":1}
I20260810 04:41:41.253793 4181 leader_election.cc:302] T 8d49e1ccae4546f59c658baa66310f40 P 5fc86b14002f4fd3ae81df14a166c562 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5fc86b14002f4fd3ae81df14a166c562, 84e7718d636e41efbd3a4b32db54e3f8; no voters:
I20260810 04:41:41.253944 4433 raft_consensus.cc:2781] T 8d49e1ccae4546f59c658baa66310f40 P 5fc86b14002f4fd3ae81df14a166c562 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 04:41:41.253979 4433 raft_consensus.cc:482] T 8d49e1ccae4546f59c658baa66310f40 P 5fc86b14002f4fd3ae81df14a166c562 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 04:41:41.254004 4433 raft_consensus.cc:3037] T 8d49e1ccae4546f59c658baa66310f40 P 5fc86b14002f4fd3ae81df14a166c562 [term 0 FOLLOWER]: Advancing to term 1
I20260810 04:41:41.254102 4055 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8d49e1ccae4546f59c658baa66310f40" candidate_uuid: "5fc86b14002f4fd3ae81df14a166c562" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "603becb479b7443f854a91ab3530293a" is_pre_election: true
I20260810 04:41:41.254226 4055 raft_consensus.cc:2454] T 8d49e1ccae4546f59c658baa66310f40 P 603becb479b7443f854a91ab3530293a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5fc86b14002f4fd3ae81df14a166c562 in term 0.
I20260810 04:41:41.254782 4433 raft_consensus.cc:504] T 8d49e1ccae4546f59c658baa66310f40 P 5fc86b14002f4fd3ae81df14a166c562 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84e7718d636e41efbd3a4b32db54e3f8" member_type: VOTER last_known_addr { host: "127.22.167.65" port: 40941 } } peers { permanent_uuid: "603becb479b7443f854a91ab3530293a" member_type: VOTER last_known_addr { host: "127.22.167.66" port: 34655 } } peers { permanent_uuid: "5fc86b14002f4fd3ae81df14a166c562" member_type: VOTER last_known_addr { host: "127.22.167.67" port: 33991 } }
I20260810 04:41:41.254912 4433 leader_election.cc:288] T 8d49e1ccae4546f59c658baa66310f40 P 5fc86b14002f4fd3ae81df14a166c562 [CANDIDATE]: Term 1 election: Requested vote from peers 84e7718d636e41efbd3a4b32db54e3f8 (127.22.167.65:40941), 603becb479b7443f854a91ab3530293a (127.22.167.66:34655)
I20260810 04:41:41.255079 4055 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8d49e1ccae4546f59c658baa66310f40" candidate_uuid: "5fc86b14002f4fd3ae81df14a166c562" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "603becb479b7443f854a91ab3530293a"
I20260810 04:41:41.255098 3848 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8d49e1ccae4546f59c658baa66310f40" candidate_uuid: "5fc86b14002f4fd3ae81df14a166c562" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84e7718d636e41efbd3a4b32db54e3f8"
I20260810 04:41:41.255152 4055 raft_consensus.cc:3037] T 8d49e1ccae4546f59c658baa66310f40 P 603becb479b7443f854a91ab3530293a [term 0 FOLLOWER]: Advancing to term 1
I20260810 04:41:41.255174 3848 raft_consensus.cc:3037] T 8d49e1ccae4546f59c658baa66310f40 P 84e7718d636e41efbd3a4b32db54e3f8 [term 0 FOLLOWER]: Advancing to term 1
I20260810 04:41:41.255746 3848 raft_consensus.cc:2454] T 8d49e1ccae4546f59c658baa66310f40 P 84e7718d636e41efbd3a4b32db54e3f8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5fc86b14002f4fd3ae81df14a166c562 in term 1.
I20260810 04:41:41.255784 4055 raft_consensus.cc:2454] T 8d49e1ccae4546f59c658baa66310f40 P 603becb479b7443f854a91ab3530293a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5fc86b14002f4fd3ae81df14a166c562 in term 1.
I20260810 04:41:41.255905 4181 leader_election.cc:302] T 8d49e1ccae4546f59c658baa66310f40 P 5fc86b14002f4fd3ae81df14a166c562 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5fc86b14002f4fd3ae81df14a166c562, 84e7718d636e41efbd3a4b32db54e3f8; no voters:
I20260810 04:41:41.256003 4433 raft_consensus.cc:2781] T 8d49e1ccae4546f59c658baa66310f40 P 5fc86b14002f4fd3ae81df14a166c562 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 04:41:41.256143 4433 raft_consensus.cc:686] T 8d49e1ccae4546f59c658baa66310f40 P 5fc86b14002f4fd3ae81df14a166c562 [term 1 LEADER]: Becoming Leader. State: Replica: 5fc86b14002f4fd3ae81df14a166c562, State: Running, Role: LEADER
I20260810 04:41:41.256239 4433 consensus_queue.cc:237] T 8d49e1ccae4546f59c658baa66310f40 P 5fc86b14002f4fd3ae81df14a166c562 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84e7718d636e41efbd3a4b32db54e3f8" member_type: VOTER last_known_addr { host: "127.22.167.65" port: 40941 } } peers { permanent_uuid: "603becb479b7443f854a91ab3530293a" member_type: VOTER last_known_addr { host: "127.22.167.66" port: 34655 } } peers { permanent_uuid: "5fc86b14002f4fd3ae81df14a166c562" member_type: VOTER last_known_addr { host: "127.22.167.67" port: 33991 } }
I20260810 04:41:41.256893 3596 catalog_manager.cc:5495] T 8d49e1ccae4546f59c658baa66310f40 P 5fc86b14002f4fd3ae81df14a166c562 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5fc86b14002f4fd3ae81df14a166c562 (127.22.167.67). New cstate: current_term: 1 leader_uuid: "5fc86b14002f4fd3ae81df14a166c562" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84e7718d636e41efbd3a4b32db54e3f8" member_type: VOTER last_known_addr { host: "127.22.167.65" port: 40941 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "603becb479b7443f854a91ab3530293a" member_type: VOTER last_known_addr { host: "127.22.167.66" port: 34655 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5fc86b14002f4fd3ae81df14a166c562" member_type: VOTER last_known_addr { host: "127.22.167.67" port: 33991 } health_report { overall_health: HEALTHY } } }
I20260810 04:41:41.274165 4413 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:41:41.272300 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56582 (local address 127.22.167.67:33991)
0810 04:41:41.272343 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0810 04:41:41.272344 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0810 04:41:41.272351 (+ 7us) server_negotiation.cc:407] Connection header received
0810 04:41:41.272377 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:41:41.272377 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:41:41.272394 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:41:41.272447 (+ 53us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 04:41:41.272532 (+ 85us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:41:41.272968 (+ 436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:41:41.273596 (+ 628us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:41:41.273768 (+ 172us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:41:41.273811 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:41:41.273899 (+ 88us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 04:41:41.273999 (+ 100us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 04:41:41.274046 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0810 04:41:41.274062 (+ 16us) server_negotiation.cc:299] Negotiation successful
0810 04:41:41.274077 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260810 04:41:41.278282 4501 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:41:41.276278 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.167.67:43191 (local address 127.22.167.66:34655)
0810 04:41:41.276322 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0810 04:41:41.276323 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0810 04:41:41.276440 (+ 117us) server_negotiation.cc:407] Connection header received
0810 04:41:41.276462 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:41:41.276463 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:41:41.276473 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:41:41.276519 (+ 46us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 04:41:41.276634 (+ 115us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:41:41.277082 (+ 448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:41:41.277747 (+ 665us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:41:41.277925 (+ 178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:41:41.277970 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:41:41.278086 (+ 116us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 04:41:41.278108 (+ 22us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 04:41:41.278179 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0810 04:41:41.278220 (+ 41us) server_negotiation.cc:299] Negotiation successful
0810 04:41:41.278235 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260810 04:41:41.278323 4576 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:41:41.276223 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.66:34655 (local address 127.22.167.67:43191)
0810 04:41:41.276367 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0810 04:41:41.276376 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0810 04:41:41.276430 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 04:41:41.276526 (+ 96us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 04:41:41.276527 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 04:41:41.276527 (+ 0us) client_negotiation.cc:189] Negotiated authn=TOKEN
0810 04:41:41.276577 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:41:41.276578 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:41:41.277141 (+ 563us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:41:41.277141 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:41:41.277692 (+ 551us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:41:41.277693 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:41:41.277976 (+ 283us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:41:41.277976 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:41:41.278019 (+ 43us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:41:41.278033 (+ 14us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0810 04:41:41.278182 (+ 149us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0810 04:41:41.278184 (+ 2us) client_negotiation.cc:769] Sending connection context
0810 04:41:41.278216 (+ 32us) client_negotiation.cc:240] Negotiation successful
0810 04:41:41.278245 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":69,"threads_started":1}
I20260810 04:41:41.280288 3848 raft_consensus.cc:1261] T 8d49e1ccae4546f59c658baa66310f40 P 84e7718d636e41efbd3a4b32db54e3f8 [term 1 FOLLOWER]: Refusing update from remote peer 5fc86b14002f4fd3ae81df14a166c562: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 04:41:41.280318 4055 raft_consensus.cc:1261] T 8d49e1ccae4546f59c658baa66310f40 P 603becb479b7443f854a91ab3530293a [term 1 FOLLOWER]: Refusing update from remote peer 5fc86b14002f4fd3ae81df14a166c562: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 04:41:41.280515 4433 consensus_queue.cc:1035] T 8d49e1ccae4546f59c658baa66310f40 P 5fc86b14002f4fd3ae81df14a166c562 [LEADER]: Connected to new peer: Peer: permanent_uuid: "603becb479b7443f854a91ab3530293a" member_type: VOTER last_known_addr { host: "127.22.167.66" port: 34655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 04:41:41.280637 4571 consensus_queue.cc:1035] T 8d49e1ccae4546f59c658baa66310f40 P 5fc86b14002f4fd3ae81df14a166c562 [LEADER]: Connected to new peer: Peer: permanent_uuid: "84e7718d636e41efbd3a4b32db54e3f8" member_type: VOTER last_known_addr { host: "127.22.167.65" port: 40941 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 04:41:41.286966 4413 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:41:41.285124 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.167.66:37219 (local address 127.22.167.67:33991)
0810 04:41:41.285159 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0810 04:41:41.285169 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 04:41:41.285175 (+ 6us) server_negotiation.cc:407] Connection header received
0810 04:41:41.285191 (+ 16us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:41:41.285191 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:41:41.285198 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:41:41.285248 (+ 50us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 04:41:41.285320 (+ 72us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:41:41.285727 (+ 407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:41:41.286404 (+ 677us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:41:41.286590 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:41:41.286630 (+ 40us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:41:41.286765 (+ 135us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 04:41:41.286797 (+ 32us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 04:41:41.286841 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0810 04:41:41.286890 (+ 49us) server_negotiation.cc:299] Negotiation successful
0810 04:41:41.286908 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15}
I20260810 04:41:41.286993 4520 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:41:41.284952 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.67:33991 (local address 127.22.167.66:37219)
0810 04:41:41.284991 (+ 39us) negotiation.cc:107] Waiting for socket to connect
0810 04:41:41.285001 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0810 04:41:41.285036 (+ 35us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 04:41:41.285252 (+ 216us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 04:41:41.285252 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 04:41:41.285253 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0810 04:41:41.285281 (+ 28us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:41:41.285282 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:41:41.285772 (+ 490us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:41:41.285773 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:41:41.286343 (+ 570us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:41:41.286345 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:41:41.286635 (+ 290us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:41:41.286636 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:41:41.286694 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:41:41.286700 (+ 6us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0810 04:41:41.286847 (+ 147us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0810 04:41:41.286848 (+ 1us) client_negotiation.cc:769] Sending connection context
0810 04:41:41.286889 (+ 41us) client_negotiation.cc:240] Negotiation successful
0810 04:41:41.286930 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":19}
I20260810 04:41:41.293601 3593 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52986:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 04:41:41.293695 3593 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52986:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 04:41:41.294286 3593 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b8bbdb416f794faeb39ac8593eebe708: Sending DeleteTablet for 3 replicas of tablet 8d49e1ccae4546f59c658baa66310f40
I20260810 04:41:41.294519 3815 tablet_service.cc:1514] Processing DeleteTablet for tablet 8d49e1ccae4546f59c658baa66310f40 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 04:41:41 UTC) from {username='oryx', principal='oryx/127.22.167.126@KRBTEST.COM'} at 127.0.0.1:50476
I20260810 04:41:41.294548 4267 tablet_service.cc:1514] Processing DeleteTablet for tablet 8d49e1ccae4546f59c658baa66310f40 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 04:41:41 UTC) from {username='oryx', principal='oryx/127.22.167.126@KRBTEST.COM'} at 127.0.0.1:56580
I20260810 04:41:41.294610 4526 tablet_replica.cc:330] T 8d49e1ccae4546f59c658baa66310f40 P 84e7718d636e41efbd3a4b32db54e3f8: stopping tablet replica
I20260810 04:41:41.294653 4526 raft_consensus.cc:2229] T 8d49e1ccae4546f59c658baa66310f40 P 84e7718d636e41efbd3a4b32db54e3f8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 04:41:41.294636 4524 tablet_replica.cc:330] T 8d49e1ccae4546f59c658baa66310f40 P 5fc86b14002f4fd3ae81df14a166c562: stopping tablet replica
I20260810 04:41:41.294683 4526 raft_consensus.cc:2258] T 8d49e1ccae4546f59c658baa66310f40 P 84e7718d636e41efbd3a4b32db54e3f8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 04:41:41.294699 4524 raft_consensus.cc:2229] T 8d49e1ccae4546f59c658baa66310f40 P 5fc86b14002f4fd3ae81df14a166c562 [term 1 LEADER]: Raft consensus shutting down.
I20260810 04:41:41.294771 4524 raft_consensus.cc:2258] T 8d49e1ccae4546f59c658baa66310f40 P 5fc86b14002f4fd3ae81df14a166c562 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 04:41:41.294780 23197 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskemv_AM/build/release/bin/kudu with pid 3703
I20260810 04:41:41.294880 4031 tablet_service.cc:1514] Processing DeleteTablet for tablet 8d49e1ccae4546f59c658baa66310f40 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 04:41:41 UTC) from {username='oryx', principal='oryx/127.22.167.126@KRBTEST.COM'} at 127.0.0.1:50370
I20260810 04:41:41.301079 4529 tablet_replica.cc:330] T 8d49e1ccae4546f59c658baa66310f40 P 603becb479b7443f854a91ab3530293a: stopping tablet replica
I20260810 04:41:41.301138 4529 raft_consensus.cc:2229] T 8d49e1ccae4546f59c658baa66310f40 P 603becb479b7443f854a91ab3530293a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 04:41:41.301179 4529 raft_consensus.cc:2258] T 8d49e1ccae4546f59c658baa66310f40 P 603becb479b7443f854a91ab3530293a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 04:41:41.302913 4524 ts_tablet_manager.cc:1905] T 8d49e1ccae4546f59c658baa66310f40 P 5fc86b14002f4fd3ae81df14a166c562: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 04:41:41.303277 4529 ts_tablet_manager.cc:1905] T 8d49e1ccae4546f59c658baa66310f40 P 603becb479b7443f854a91ab3530293a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 04:41:41.304399 4524 ts_tablet_manager.cc:1918] T 8d49e1ccae4546f59c658baa66310f40 P 5fc86b14002f4fd3ae81df14a166c562: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260810 04:41:41.304464 4524 log.cc:1196] T 8d49e1ccae4546f59c658baa66310f40 P 5fc86b14002f4fd3ae81df14a166c562: Deleting WAL directory at /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786336863388938-23197-0/minicluster-data/ts-2/wal/wals/8d49e1ccae4546f59c658baa66310f40
I20260810 04:41:41.304677 4524 ts_tablet_manager.cc:1939] T 8d49e1ccae4546f59c658baa66310f40 P 5fc86b14002f4fd3ae81df14a166c562: Deleting consensus metadata
I20260810 04:41:41.304713 4529 ts_tablet_manager.cc:1918] T 8d49e1ccae4546f59c658baa66310f40 P 603becb479b7443f854a91ab3530293a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260810 04:41:41.304756 4529 log.cc:1196] T 8d49e1ccae4546f59c658baa66310f40 P 603becb479b7443f854a91ab3530293a: Deleting WAL directory at /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786336863388938-23197-0/minicluster-data/ts-1/wal/wals/8d49e1ccae4546f59c658baa66310f40
I20260810 04:41:41.304944 4529 ts_tablet_manager.cc:1939] T 8d49e1ccae4546f59c658baa66310f40 P 603becb479b7443f854a91ab3530293a: Deleting consensus metadata
I20260810 04:41:41.305039 3571 catalog_manager.cc:4841] TS 5fc86b14002f4fd3ae81df14a166c562 (127.22.167.67:33991): tablet 8d49e1ccae4546f59c658baa66310f40 (table test-table [id=4a53590cd77c4a169e78e0dc82489f40]) successfully deleted
I20260810 04:41:41.305815 3571 catalog_manager.cc:4841] TS 603becb479b7443f854a91ab3530293a (127.22.167.66:34655): tablet 8d49e1ccae4546f59c658baa66310f40 (table test-table [id=4a53590cd77c4a169e78e0dc82489f40]) successfully deleted
W20260810 04:41:41.310475 3573 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.22.167.65:40941 (error 108)
I20260810 04:41:41.310801 23197 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskemv_AM/build/release/bin/kudu with pid 3931
I20260810 04:41:41.318436 4584 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:41:41.310876 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.65:40941 (local address 127.0.0.1:50504)
0810 04:41:41.318318 (+ 7442us) negotiation.cc:107] Waiting for socket to connect
0810 04:41:41.318389 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.167.65:40941: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":7416,"thread_start_us":62,"threads_started":1}
W20260810 04:41:41.318661 3573 catalog_manager.cc:4584] TS 84e7718d636e41efbd3a4b32db54e3f8 (127.22.167.65:40941): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 8d49e1ccae4546f59c658baa66310f40: Network error: Client connection negotiation failed: client connection to 127.22.167.65:40941: connect: Connection refused (error 111)
I20260810 04:41:41.322293 23197 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskemv_AM/build/release/bin/kudu with pid 4148
I20260810 04:41:41.330250 23197 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskemv_AM/build/release/bin/kudu with pid 3535
2026-08-10T04:41:41Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipal (3184 ms)
[ RUN ] SecurityITest.TestNonExistentPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786336863388938-23197-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 04:41:41 dist-test-slave-bqcl krb5kdc[4592](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 04:41:41 dist-test-slave-bqcl krb5kdc[4592](info): set up 2 sockets
Aug 10 04:41:41 dist-test-slave-bqcl krb5kdc[4592](info): commencing operation
krb5kdc: starting...
W20260810 04:41:42.490739 23197 mini_kdc.cc:121] Time spent starting KDC: real 1.134s user 0.001s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786336863388938-23197-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786336863388938-23197-0/krb5kdc/test-user.keytab.
Aug 10 04:41:42 dist-test-slave-bqcl krb5kdc[4592](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786336902, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T04:41:42Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T04:41:42Z Disabled control of system clock
WARNING: no policy specified for kudu/127.22.167.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.167.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.167.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786336863388938-23197-0/krb5kdc/kudu_127.22.167.126.keytab.
Entry for principal kudu/127.22.167.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786336863388938-23197-0/krb5kdc/kudu_127.22.167.126.keytab.
I20260810 04:41:42.606320 23197 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskemv_AM/build/release/bin/kudu
/tmp/dist-test-taskemv_AM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786336863388938-23197-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786336863388938-23197-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786336863388938-23197-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786336863388938-23197-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.167.126:39457
--webserver_interface=127.22.167.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786336863388938-23197-0/krb5kdc/kudu_127.22.167.126.keytab
--principal=kudu/127.22.167.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.167.84:35521
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.167.126:39457
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--principal=oryx with env {KRB5CCNAME=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786336863388938-23197-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786336863388938-23197-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786336863388938-23197-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 04:41:42.688115 4612 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 04:41:42.688323 4612 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 04:41:42.688345 4612 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 04:41:42.689431 4612 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 04:41:42.689476 4612 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 04:41:42.689510 4612 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 04:41:42.689538 4612 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 04:41:42.689563 4612 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 04:41:42.691093 4612 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.167.84:35521
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786336863388938-23197-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786336863388938-23197-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.22.167.126:39457
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786336863388938-23197-0/krb5kdc/kudu_127.22.167.126.keytab
--principal=oryx
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.167.126:39457
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786336863388938-23197-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.167.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4612
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786336863388938-23197-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 03:54:01 UTC on ccycloud.quasar-icijmt.root.comops.site
build id 4577009
I20260810 04:41:42.691326 4612 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 04:41:42.691546 4612 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 04:41:42.696095 4621 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 04:41:42.696157 4623 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 04:41:42.696095 4626 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 04:41:42.696508 4612 server_base.cc:1034] running on GCE node
I20260810 04:41:42.696754 4612 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 04:41:42.697027 4612 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 04:41:42.700305 4612 hybrid_clock.cc:648] HybridClock initialized: now 1786336902700279 us; error 38 us; skew 500 ppm
Runtime error: kinit failed: unable to login from keytab: Keytab contains no suitable keys for oryx@KRBTEST.COM
2026-08-10T04:41:42Z chronyd exiting
[ OK ] SecurityITest.TestNonExistentPrincipal (1377 ms)
[ RUN ] SecurityITest.TestMismatchingPrincipals
Loading random data
Initializing database '/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786336863388938-23197-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 04:41:42 dist-test-slave-bqcl krb5kdc[4634](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 04:41:42 dist-test-slave-bqcl krb5kdc[4634](info): set up 2 sockets
Aug 10 04:41:42 dist-test-slave-bqcl krb5kdc[4634](info): commencing operation
krb5kdc: starting...
W20260810 04:41:43.873006 23197 mini_kdc.cc:121] Time spent starting KDC: real 1.139s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786336863388938-23197-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786336863388938-23197-0/krb5kdc/test-user.keytab.
Aug 10 04:41:43 dist-test-slave-bqcl krb5kdc[4634](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786336903, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T04:41:43Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T04:41:43Z Disabled control of system clock
WARNING: no policy specified for kudu/127.22.167.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.167.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.167.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786336863388938-23197-0/krb5kdc/kudu_127.22.167.126.keytab.
Entry for principal kudu/127.22.167.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786336863388938-23197-0/krb5kdc/kudu_127.22.167.126.keytab.
I20260810 04:41:43.986647 23197 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskemv_AM/build/release/bin/kudu
/tmp/dist-test-taskemv_AM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786336863388938-23197-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786336863388938-23197-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786336863388938-23197-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786336863388938-23197-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.167.126:37819
--webserver_interface=127.22.167.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786336863388938-23197-0/krb5kdc/kudu_127.22.167.126.keytab
--principal=kudu/127.22.167.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.167.84:41851
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.167.126:37819
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786336863388938-23197-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786336863388938-23197-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786336863388938-23197-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 04:41:44.065775 4659 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 04:41:44.065975 4659 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 04:41:44.066006 4659 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 04:41:44.067133 4659 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 04:41:44.067176 4659 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 04:41:44.067196 4659 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 04:41:44.067211 4659 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 04:41:44.067227 4659 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 04:41:44.068650 4659 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.167.84:41851
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786336863388938-23197-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786336863388938-23197-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.22.167.126:37819
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786336863388938-23197-0/krb5kdc/kudu_127.22.167.126.keytab
--principal=kudu/127.22.167.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.167.126:37819
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786336863388938-23197-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.167.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4659
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786336863388938-23197-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 03:54:01 UTC on ccycloud.quasar-icijmt.root.comops.site
build id 4577009
I20260810 04:41:44.068899 4659 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 04:41:44.069098 4659 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 04:41:44.073602 4671 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 04:41:44.073679 4659 server_base.cc:1034] running on GCE node
W20260810 04:41:44.073609 4666 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 04:41:44.073621 4668 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 04:41:44.074003 4659 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 04:41:44.074311 4659 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 04:41:44.075448 4659 hybrid_clock.cc:648] HybridClock initialized: now 1786336904075437 us; error 27 us; skew 500 ppm
Aug 10 04:41:44 dist-test-slave-bqcl krb5kdc[4634](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786336904, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.167.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 04:41:44.080577 4659 init.cc:377] Logged in from keytab as kudu/127.22.167.126@KRBTEST.COM (short username kudu)
I20260810 04:41:44.081015 4659 webserver.cc:460] Webserver started at http://127.22.167.126:39245/ using document root <none> and password file <none>
I20260810 04:41:44.081239 4659 fs_manager.cc:359] Metadata directory not provided
I20260810 04:41:44.081290 4659 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 04:41:44.081418 4659 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 04:41:44.082501 4659 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786336863388938-23197-0/minicluster-data/master-0/data/instance:
uuid: "75b33e7bf0f44523b3341a1681bc0578"
format_stamp: "Formatted at 2026-08-10 04:41:44 on dist-test-slave-bqcl"
server_key: "1b66466479ff8daab3a4146feb831faf"
server_key_iv: "858d63308572e9f4656dd31b0c5dfefa"
server_key_version: "encryptionkey@0"
I20260810 04:41:44.082839 4659 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786336863388938-23197-0/minicluster-data/master-0/wal/instance:
uuid: "75b33e7bf0f44523b3341a1681bc0578"
format_stamp: "Formatted at 2026-08-10 04:41:44 on dist-test-slave-bqcl"
server_key: "1b66466479ff8daab3a4146feb831faf"
server_key_iv: "858d63308572e9f4656dd31b0c5dfefa"
server_key_version: "encryptionkey@0"
I20260810 04:41:44.084043 4659 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260810 04:41:44.084674 4683 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 04:41:44.084831 4659 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260810 04:41:44.084918 4659 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786336863388938-23197-0/minicluster-data/master-0/data,/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786336863388938-23197-0/minicluster-data/master-0/wal
uuid: "75b33e7bf0f44523b3341a1681bc0578"
format_stamp: "Formatted at 2026-08-10 04:41:44 on dist-test-slave-bqcl"
server_key: "1b66466479ff8daab3a4146feb831faf"
server_key_iv: "858d63308572e9f4656dd31b0c5dfefa"
server_key_version: "encryptionkey@0"
I20260810 04:41:44.084976 4659 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786336863388938-23197-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786336863388938-23197-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786336863388938-23197-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 04:41:44.116711 4659 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 04:41:44.118896 4659 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 04:41:44.119035 4659 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 04:41:44.122836 4659 rpc_server.cc:307] RPC server started. Bound to: 127.22.167.126:37819
I20260810 04:41:44.122874 4778 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.167.126:37819 every 8 connection(s)
I20260810 04:41:44.123199 4659 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786336863388938-23197-0/minicluster-data/master-0/data/info.pb
I20260810 04:41:44.124022 4779 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 04:41:44.126246 4779 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 75b33e7bf0f44523b3341a1681bc0578: Bootstrap starting.
I20260810 04:41:44.126782 4779 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 75b33e7bf0f44523b3341a1681bc0578: Neither blocks nor log segments found. Creating new log.
I20260810 04:41:44.127015 4779 log.cc:824] T 00000000000000000000000000000000 P 75b33e7bf0f44523b3341a1681bc0578: Log is configured to *not* fsync() on all Append() calls
I20260810 04:41:44.127619 4779 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 75b33e7bf0f44523b3341a1681bc0578: No bootstrap required, opened a new log
I20260810 04:41:44.129153 4779 raft_consensus.cc:348] T 00000000000000000000000000000000 P 75b33e7bf0f44523b3341a1681bc0578 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "75b33e7bf0f44523b3341a1681bc0578" member_type: VOTER last_known_addr { host: "127.22.167.126" port: 37819 } }
I20260810 04:41:44.129292 4779 raft_consensus.cc:374] T 00000000000000000000000000000000 P 75b33e7bf0f44523b3341a1681bc0578 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 04:41:44.129313 4779 raft_consensus.cc:729] T 00000000000000000000000000000000 P 75b33e7bf0f44523b3341a1681bc0578 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 75b33e7bf0f44523b3341a1681bc0578, State: Initialized, Role: FOLLOWER
I20260810 04:41:44.129419 4779 consensus_queue.cc:260] T 00000000000000000000000000000000 P 75b33e7bf0f44523b3341a1681bc0578 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "75b33e7bf0f44523b3341a1681bc0578" member_type: VOTER last_known_addr { host: "127.22.167.126" port: 37819 } }
I20260810 04:41:44.129494 4779 raft_consensus.cc:388] T 00000000000000000000000000000000 P 75b33e7bf0f44523b3341a1681bc0578 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 04:41:44.129522 4779 raft_consensus.cc:482] T 00000000000000000000000000000000 P 75b33e7bf0f44523b3341a1681bc0578 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 04:41:44.129554 4779 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 75b33e7bf0f44523b3341a1681bc0578 [term 0 FOLLOWER]: Advancing to term 1
I20260810 04:41:44.130007 4779 raft_consensus.cc:504] T 00000000000000000000000000000000 P 75b33e7bf0f44523b3341a1681bc0578 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "75b33e7bf0f44523b3341a1681bc0578" member_type: VOTER last_known_addr { host: "127.22.167.126" port: 37819 } }
I20260810 04:41:44.130089 4779 leader_election.cc:302] T 00000000000000000000000000000000 P 75b33e7bf0f44523b3341a1681bc0578 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 75b33e7bf0f44523b3341a1681bc0578; no voters:
I20260810 04:41:44.130240 4779 leader_election.cc:288] T 00000000000000000000000000000000 P 75b33e7bf0f44523b3341a1681bc0578 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 04:41:44.130293 4783 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 75b33e7bf0f44523b3341a1681bc0578 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 04:41:44.130465 4779 sys_catalog.cc:564] T 00000000000000000000000000000000 P 75b33e7bf0f44523b3341a1681bc0578 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 04:41:44.130650 4783 raft_consensus.cc:686] T 00000000000000000000000000000000 P 75b33e7bf0f44523b3341a1681bc0578 [term 1 LEADER]: Becoming Leader. State: Replica: 75b33e7bf0f44523b3341a1681bc0578, State: Running, Role: LEADER
I20260810 04:41:44.130772 4783 consensus_queue.cc:237] T 00000000000000000000000000000000 P 75b33e7bf0f44523b3341a1681bc0578 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "75b33e7bf0f44523b3341a1681bc0578" member_type: VOTER last_known_addr { host: "127.22.167.126" port: 37819 } }
I20260810 04:41:44.130833 23197 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskemv_AM/build/release/bin/kudu as pid 4659
I20260810 04:41:44.130946 23197 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786336863388938-23197-0/minicluster-data/master-0/wal/instance
I20260810 04:41:44.131161 23197 external_mini_cluster.cc:1428] Setting key 314c6c4e53d5a780998e3e45c1a93585
I20260810 04:41:44.131115 4784 sys_catalog.cc:455] T 00000000000000000000000000000000 P 75b33e7bf0f44523b3341a1681bc0578 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "75b33e7bf0f44523b3341a1681bc0578" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "75b33e7bf0f44523b3341a1681bc0578" member_type: VOTER last_known_addr { host: "127.22.167.126" port: 37819 } } }
I20260810 04:41:44.131199 4784 sys_catalog.cc:458] T 00000000000000000000000000000000 P 75b33e7bf0f44523b3341a1681bc0578 [sys.catalog]: This master's current role is: LEADER
I20260810 04:41:44.131359 4785 sys_catalog.cc:455] T 00000000000000000000000000000000 P 75b33e7bf0f44523b3341a1681bc0578 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 75b33e7bf0f44523b3341a1681bc0578. Latest consensus state: current_term: 1 leader_uuid: "75b33e7bf0f44523b3341a1681bc0578" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "75b33e7bf0f44523b3341a1681bc0578" member_type: VOTER last_known_addr { host: "127.22.167.126" port: 37819 } } }
I20260810 04:41:44.131422 4785 sys_catalog.cc:458] T 00000000000000000000000000000000 P 75b33e7bf0f44523b3341a1681bc0578 [sys.catalog]: This master's current role is: LEADER
I20260810 04:41:44.131721 4797 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 04:41:44.132323 4797 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 04:41:44.133736 4797 catalog_manager.cc:1347] Generated new cluster ID: 6409a70d916b4d9abf80e9cce1e75f79
I20260810 04:41:44.133790 4797 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 10 04:41:44 dist-test-slave-bqcl krb5kdc[4634](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786336903, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.167.126@KRBTEST.COM
I20260810 04:41:44.140228 4797 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 04:41:44.140872 4797 catalog_manager.cc:1504] Loading token signing keys...
I20260810 04:41:44.142822 4809 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:41:44.131878 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34618 (local address 127.22.167.126:37819)
0810 04:41:44.132116 (+ 238us) server_negotiation.cc:206] Beginning negotiation
0810 04:41:44.132126 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 04:41:44.132157 (+ 31us) server_negotiation.cc:407] Connection header received
0810 04:41:44.132271 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:41:44.132278 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:41:44.132333 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:41:44.132507 (+ 174us) server_negotiation.cc:226] Negotiated authn=SASL
0810 04:41:44.133258 (+ 751us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:41:44.134681 (+ 1423us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:41:44.135744 (+ 1063us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:41:44.136341 (+ 597us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:41:44.136442 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:41:44.139139 (+ 2697us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 04:41:44.139143 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 04:41:44.139143 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 04:41:44.139155 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0810 04:41:44.141193 (+ 2038us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 04:41:44.141568 (+ 375us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 04:41:44.141570 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 04:41:44.141570 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 04:41:44.141677 (+ 107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 04:41:44.141856 (+ 179us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 04:41:44.141857 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 04:41:44.141857 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 04:41:44.142094 (+ 237us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 04:41:44.142273 (+ 179us) server_negotiation.cc:1035] Waiting for connection context
0810 04:41:44.142359 (+ 86us) server_negotiation.cc:299] Negotiation successful
0810 04:41:44.142587 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":195,"thread_start_us":59,"threads_started":1}
I20260810 04:41:44.145589 4797 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 75b33e7bf0f44523b3341a1681bc0578: Generated new TSK 0
I20260810 04:41:44.145751 4797 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.22.167.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.167.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.167.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786336863388938-23197-0/krb5kdc/kudu_127.22.167.65.keytab.
Entry for principal kudu/127.22.167.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786336863388938-23197-0/krb5kdc/kudu_127.22.167.65.keytab.
I20260810 04:41:44.172734 23197 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskemv_AM/build/release/bin/kudu
/tmp/dist-test-taskemv_AM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786336863388938-23197-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786336863388938-23197-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786336863388938-23197-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786336863388938-23197-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.167.65:0
--local_ip_for_outbound_sockets=127.22.167.65
--webserver_interface=127.22.167.65
--webserver_port=0
--tserver_master_addrs=127.22.167.126:37819
--keytab_file=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786336863388938-23197-0/krb5kdc/kudu_127.22.167.65.keytab
--principal=kudu/127.22.167.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.167.84:41851
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786336863388938-23197-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786336863388938-23197-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786336863388938-23197-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 04:41:44.253149 4817 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 04:41:44.253360 4817 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 04:41:44.253381 4817 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 04:41:44.254523 4817 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 04:41:44.254565 4817 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 04:41:44.254590 4817 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.167.65
I20260810 04:41:44.256002 4817 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.167.84:41851
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786336863388938-23197-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786336863388938-23197-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786336863388938-23197-0/krb5kdc/kudu_127.22.167.65.keytab
--principal=kudu/127.22.167.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.167.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786336863388938-23197-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.22.167.65
--webserver_port=0
--tserver_master_addrs=127.22.167.126:37819
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4817
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.167.65
--log_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786336863388938-23197-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 03:54:01 UTC on ccycloud.quasar-icijmt.root.comops.site
build id 4577009
I20260810 04:41:44.256229 4817 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 04:41:44.256436 4817 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 04:41:44.261298 4831 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 04:41:44.261298 4822 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 04:41:44.261356 4825 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 04:41:44.261648 4817 server_base.cc:1034] running on GCE node
I20260810 04:41:44.261883 4817 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 04:41:44.262109 4817 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 04:41:44.263276 4817 hybrid_clock.cc:648] HybridClock initialized: now 1786336904263257 us; error 34 us; skew 500 ppm
Aug 10 04:41:44 dist-test-slave-bqcl krb5kdc[4634](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786336904, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.167.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 04:41:44.265723 4817 init.cc:377] Logged in from keytab as kudu/127.22.167.65@KRBTEST.COM (short username kudu)
I20260810 04:41:44.266117 4817 webserver.cc:460] Webserver started at http://127.22.167.65:39965/ using document root <none> and password file <none>
I20260810 04:41:44.266350 4817 fs_manager.cc:359] Metadata directory not provided
I20260810 04:41:44.266398 4817 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 04:41:44.266521 4817 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 04:41:44.267506 4817 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786336863388938-23197-0/minicluster-data/ts-0/data/instance:
uuid: "b8aa536af5144554a53fc0f1a744a64a"
format_stamp: "Formatted at 2026-08-10 04:41:44 on dist-test-slave-bqcl"
server_key: "0ae9c66349d57715f58025da7d4e1f78"
server_key_iv: "73bad8c3de589f49db7c18a7ff257ba9"
server_key_version: "encryptionkey@0"
I20260810 04:41:44.267845 4817 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786336863388938-23197-0/minicluster-data/ts-0/wal/instance:
uuid: "b8aa536af5144554a53fc0f1a744a64a"
format_stamp: "Formatted at 2026-08-10 04:41:44 on dist-test-slave-bqcl"
server_key: "0ae9c66349d57715f58025da7d4e1f78"
server_key_iv: "73bad8c3de589f49db7c18a7ff257ba9"
server_key_version: "encryptionkey@0"
I20260810 04:41:44.269040 4817 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260810 04:41:44.269663 4843 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 04:41:44.269826 4817 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260810 04:41:44.269907 4817 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786336863388938-23197-0/minicluster-data/ts-0/data,/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786336863388938-23197-0/minicluster-data/ts-0/wal
uuid: "b8aa536af5144554a53fc0f1a744a64a"
format_stamp: "Formatted at 2026-08-10 04:41:44 on dist-test-slave-bqcl"
server_key: "0ae9c66349d57715f58025da7d4e1f78"
server_key_iv: "73bad8c3de589f49db7c18a7ff257ba9"
server_key_version: "encryptionkey@0"
I20260810 04:41:44.269973 4817 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786336863388938-23197-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786336863388938-23197-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786336863388938-23197-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 04:41:44.289901 4817 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 04:41:44.291678 4817 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 04:41:44.291802 4817 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 04:41:44.292004 4817 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 04:41:44.292310 4817 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 04:41:44.292344 4817 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 04:41:44.292373 4817 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 04:41:44.292390 4817 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 04:41:44.297768 4817 rpc_server.cc:307] RPC server started. Bound to: 127.22.167.65:44213
I20260810 04:41:44.297832 5030 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.167.65:44213 every 8 connection(s)
I20260810 04:41:44.298105 4817 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786336863388938-23197-0/minicluster-data/ts-0/data/info.pb
Aug 10 04:41:44 dist-test-slave-bqcl krb5kdc[4634](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786336904, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.167.65@KRBTEST.COM for kudu/127.22.167.126@KRBTEST.COM
I20260810 04:41:44.306157 4809 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:41:44.298782 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.167.65:37863 (local address 127.22.167.126:37819)
0810 04:41:44.298864 (+ 82us) server_negotiation.cc:206] Beginning negotiation
0810 04:41:44.298866 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 04:41:44.299024 (+ 158us) server_negotiation.cc:407] Connection header received
0810 04:41:44.299495 (+ 471us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:41:44.299496 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:41:44.299506 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:41:44.299557 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0810 04:41:44.300433 (+ 876us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:41:44.300933 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:41:44.301840 (+ 907us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:41:44.302029 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:41:44.302074 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:41:44.304000 (+ 1926us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 04:41:44.304001 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 04:41:44.304002 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 04:41:44.304008 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0810 04:41:44.305094 (+ 1086us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 04:41:44.305440 (+ 346us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 04:41:44.305440 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 04:41:44.305441 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 04:41:44.305487 (+ 46us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 04:41:44.305657 (+ 170us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 04:41:44.305657 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 04:41:44.305657 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 04:41:44.305755 (+ 98us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 04:41:44.305800 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0810 04:41:44.306001 (+ 201us) server_negotiation.cc:299] Negotiation successful
0810 04:41:44.306061 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260810 04:41:44.306342 5033 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:41:44.298745 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.126:37819 (local address 127.22.167.65:37863)
0810 04:41:44.298910 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0810 04:41:44.298946 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0810 04:41:44.299411 (+ 465us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 04:41:44.299614 (+ 203us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 04:41:44.299617 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 04:41:44.300173 (+ 556us) client_negotiation.cc:189] Negotiated authn=SASL
0810 04:41:44.300361 (+ 188us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:41:44.300365 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:41:44.300993 (+ 628us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:41:44.300994 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:41:44.301777 (+ 783us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:41:44.301779 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:41:44.302082 (+ 303us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:41:44.302083 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:41:44.302211 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:41:44.302628 (+ 417us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 04:41:44.302636 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0810 04:41:44.303920 (+ 1284us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 04:41:44.305158 (+ 1238us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 04:41:44.305165 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 04:41:44.305167 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 04:41:44.305384 (+ 217us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 04:41:44.305528 (+ 144us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 04:41:44.305529 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 04:41:44.305529 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 04:41:44.305617 (+ 88us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 04:41:44.305795 (+ 178us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 04:41:44.305798 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 04:41:44.305907 (+ 109us) client_negotiation.cc:769] Sending connection context
0810 04:41:44.305984 (+ 77us) client_negotiation.cc:240] Negotiation successful
0810 04:41:44.306100 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":87,"threads_started":1}
I20260810 04:41:44.306864 5031 heartbeater.cc:347] Connected to a master server at 127.22.167.126:37819
I20260810 04:41:44.306947 5031 heartbeater.cc:464] Registering TS with master...
I20260810 04:41:44.307113 5031 heartbeater.cc:511] Master 127.22.167.126:37819 requested a full tablet report, sending...
I20260810 04:41:44.307556 23197 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskemv_AM/build/release/bin/kudu as pid 4817
I20260810 04:41:44.307642 4722 ts_manager.cc:194] Registered new tserver with Master: b8aa536af5144554a53fc0f1a744a64a (127.22.167.65:44213)
I20260810 04:41:44.307682 23197 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786336863388938-23197-0/minicluster-data/ts-0/wal/instance
I20260810 04:41:44.307889 23197 external_mini_cluster.cc:1428] Setting key 20c3ec4963ff5d3fdfaa0ff057643552
I20260810 04:41:44.308379 4722 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.167.65@KRBTEST.COM'} at 127.22.167.65:37863
WARNING: no policy specified for kudu/127.22.167.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.167.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.167.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786336863388938-23197-0/krb5kdc/kudu_127.22.167.66.keytab.
Entry for principal kudu/127.22.167.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786336863388938-23197-0/krb5kdc/kudu_127.22.167.66.keytab.
I20260810 04:41:44.330642 23197 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskemv_AM/build/release/bin/kudu
/tmp/dist-test-taskemv_AM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786336863388938-23197-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786336863388938-23197-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786336863388938-23197-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786336863388938-23197-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.167.66:0
--local_ip_for_outbound_sockets=127.22.167.66
--webserver_interface=127.22.167.66
--webserver_port=0
--tserver_master_addrs=127.22.167.126:37819
--keytab_file=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786336863388938-23197-0/krb5kdc/kudu_127.22.167.66.keytab
--principal=kudu/127.22.167.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.167.84:41851
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786336863388938-23197-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786336863388938-23197-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786336863388938-23197-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 04:41:44.410220 5041 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 04:41:44.410416 5041 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 04:41:44.410437 5041 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 04:41:44.411526 5041 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 04:41:44.411559 5041 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 04:41:44.411583 5041 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.167.66
I20260810 04:41:44.412941 5041 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.167.84:41851
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786336863388938-23197-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786336863388938-23197-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786336863388938-23197-0/krb5kdc/kudu_127.22.167.66.keytab
--principal=kudu/127.22.167.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.167.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786336863388938-23197-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.22.167.66
--webserver_port=0
--tserver_master_addrs=127.22.167.126:37819
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5041
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.167.66
--log_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786336863388938-23197-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 03:54:01 UTC on ccycloud.quasar-icijmt.root.comops.site
build id 4577009
I20260810 04:41:44.413118 5041 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 04:41:44.413288 5041 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 04:41:44.417928 5048 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 04:41:44.418004 5041 server_base.cc:1034] running on GCE node
W20260810 04:41:44.418015 5047 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 04:41:44.417927 5051 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 04:41:44.418350 5041 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 04:41:44.418588 5041 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 04:41:44.419721 5041 hybrid_clock.cc:648] HybridClock initialized: now 1786336904419710 us; error 31 us; skew 500 ppm
Aug 10 04:41:44 dist-test-slave-bqcl krb5kdc[4634](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786336904, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.167.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 04:41:44.422073 5041 init.cc:377] Logged in from keytab as kudu/127.22.167.66@KRBTEST.COM (short username kudu)
I20260810 04:41:44.422483 5041 webserver.cc:460] Webserver started at http://127.22.167.66:43373/ using document root <none> and password file <none>
I20260810 04:41:44.422688 5041 fs_manager.cc:359] Metadata directory not provided
I20260810 04:41:44.422734 5041 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 04:41:44.422856 5041 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 04:41:44.423802 5041 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786336863388938-23197-0/minicluster-data/ts-1/data/instance:
uuid: "cf0f39900c034704822cbd1fec7cbc39"
format_stamp: "Formatted at 2026-08-10 04:41:44 on dist-test-slave-bqcl"
server_key: "6ec9bce4caa595e5c7f372ebdde5825d"
server_key_iv: "e7d9304b55d9c887198bae6215e5d54a"
server_key_version: "encryptionkey@0"
I20260810 04:41:44.424113 5041 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786336863388938-23197-0/minicluster-data/ts-1/wal/instance:
uuid: "cf0f39900c034704822cbd1fec7cbc39"
format_stamp: "Formatted at 2026-08-10 04:41:44 on dist-test-slave-bqcl"
server_key: "6ec9bce4caa595e5c7f372ebdde5825d"
server_key_iv: "e7d9304b55d9c887198bae6215e5d54a"
server_key_version: "encryptionkey@0"
I20260810 04:41:44.425328 5041 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260810 04:41:44.425933 5067 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 04:41:44.426083 5041 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260810 04:41:44.426182 5041 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786336863388938-23197-0/minicluster-data/ts-1/data,/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786336863388938-23197-0/minicluster-data/ts-1/wal
uuid: "cf0f39900c034704822cbd1fec7cbc39"
format_stamp: "Formatted at 2026-08-10 04:41:44 on dist-test-slave-bqcl"
server_key: "6ec9bce4caa595e5c7f372ebdde5825d"
server_key_iv: "e7d9304b55d9c887198bae6215e5d54a"
server_key_version: "encryptionkey@0"
I20260810 04:41:44.426247 5041 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786336863388938-23197-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786336863388938-23197-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786336863388938-23197-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 04:41:44.438293 5041 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 04:41:44.440279 5041 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 04:41:44.440408 5041 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 04:41:44.440605 5041 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 04:41:44.440879 5041 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 04:41:44.440913 5041 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 04:41:44.440939 5041 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 04:41:44.440958 5041 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 04:41:44.446346 5041 rpc_server.cc:307] RPC server started. Bound to: 127.22.167.66:42063
I20260810 04:41:44.446384 5241 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.167.66:42063 every 8 connection(s)
I20260810 04:41:44.446652 5041 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786336863388938-23197-0/minicluster-data/ts-1/data/info.pb
Aug 10 04:41:44 dist-test-slave-bqcl krb5kdc[4634](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786336904, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.167.66@KRBTEST.COM for kudu/127.22.167.126@KRBTEST.COM
I20260810 04:41:44.454406 4809 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:41:44.447282 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.167.66:37211 (local address 127.22.167.126:37819)
0810 04:41:44.447352 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0810 04:41:44.447355 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 04:41:44.447486 (+ 131us) server_negotiation.cc:407] Connection header received
0810 04:41:44.447932 (+ 446us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:41:44.447933 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:41:44.447943 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:41:44.447981 (+ 38us) server_negotiation.cc:226] Negotiated authn=SASL
0810 04:41:44.448812 (+ 831us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:41:44.449290 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:41:44.450204 (+ 914us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:41:44.450385 (+ 181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:41:44.450432 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:41:44.452291 (+ 1859us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 04:41:44.452292 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 04:41:44.452292 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 04:41:44.452299 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 04:41:44.453335 (+ 1036us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 04:41:44.453653 (+ 318us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 04:41:44.453654 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 04:41:44.453654 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 04:41:44.453707 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 04:41:44.453874 (+ 167us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 04:41:44.453875 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 04:41:44.453875 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 04:41:44.453966 (+ 91us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 04:41:44.454010 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0810 04:41:44.454279 (+ 269us) server_negotiation.cc:299] Negotiation successful
0810 04:41:44.454341 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260810 04:41:44.454552 5245 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:41:44.447241 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.126:37819 (local address 127.22.167.66:37211)
0810 04:41:44.447379 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0810 04:41:44.447413 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0810 04:41:44.447847 (+ 434us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 04:41:44.448032 (+ 185us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 04:41:44.448035 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 04:41:44.448566 (+ 531us) client_negotiation.cc:189] Negotiated authn=SASL
0810 04:41:44.448744 (+ 178us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:41:44.448749 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:41:44.449348 (+ 599us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:41:44.449348 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:41:44.450142 (+ 794us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:41:44.450144 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:41:44.450440 (+ 296us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:41:44.450441 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:41:44.450535 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:41:44.450951 (+ 416us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 04:41:44.450958 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0810 04:41:44.452211 (+ 1253us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 04:41:44.453398 (+ 1187us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 04:41:44.453404 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 04:41:44.453406 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 04:41:44.453599 (+ 193us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 04:41:44.453749 (+ 150us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 04:41:44.453750 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 04:41:44.453750 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 04:41:44.453834 (+ 84us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 04:41:44.454006 (+ 172us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 04:41:44.454009 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 04:41:44.454111 (+ 102us) client_negotiation.cc:769] Sending connection context
0810 04:41:44.454253 (+ 142us) client_negotiation.cc:240] Negotiation successful
0810 04:41:44.454366 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":102,"thread_start_us":76,"threads_started":1}
I20260810 04:41:44.455025 5243 heartbeater.cc:347] Connected to a master server at 127.22.167.126:37819
I20260810 04:41:44.455124 5243 heartbeater.cc:464] Registering TS with master...
I20260810 04:41:44.455271 23197 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskemv_AM/build/release/bin/kudu as pid 5041
I20260810 04:41:44.455310 5243 heartbeater.cc:511] Master 127.22.167.126:37819 requested a full tablet report, sending...
I20260810 04:41:44.455389 23197 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786336863388938-23197-0/minicluster-data/ts-1/wal/instance
I20260810 04:41:44.455567 23197 external_mini_cluster.cc:1428] Setting key 44e396cee08fbfcfedd958c1f7cfa877
I20260810 04:41:44.455699 4722 ts_manager.cc:194] Registered new tserver with Master: cf0f39900c034704822cbd1fec7cbc39 (127.22.167.66:42063)
I20260810 04:41:44.456120 4722 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.167.66@KRBTEST.COM'} at 127.22.167.66:37211
WARNING: no policy specified for kudu/127.22.167.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.167.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.167.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786336863388938-23197-0/krb5kdc/kudu_127.22.167.67.keytab.
Entry for principal kudu/127.22.167.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786336863388938-23197-0/krb5kdc/kudu_127.22.167.67.keytab.
I20260810 04:41:44.480151 23197 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskemv_AM/build/release/bin/kudu
/tmp/dist-test-taskemv_AM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786336863388938-23197-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786336863388938-23197-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786336863388938-23197-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786336863388938-23197-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.167.67:0
--local_ip_for_outbound_sockets=127.22.167.67
--webserver_interface=127.22.167.67
--webserver_port=0
--tserver_master_addrs=127.22.167.126:37819
--keytab_file=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786336863388938-23197-0/krb5kdc/kudu_127.22.167.67.keytab
--principal=kudu/127.22.167.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.167.84:41851
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786336863388938-23197-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786336863388938-23197-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786336863388938-23197-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 04:41:44.559978 5248 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 04:41:44.560168 5248 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 04:41:44.560190 5248 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 04:41:44.561308 5248 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 04:41:44.561349 5248 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 04:41:44.561378 5248 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.167.67
I20260810 04:41:44.562804 5248 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.167.84:41851
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786336863388938-23197-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786336863388938-23197-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786336863388938-23197-0/krb5kdc/kudu_127.22.167.67.keytab
--principal=kudu/127.22.167.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.167.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786336863388938-23197-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.22.167.67
--webserver_port=0
--tserver_master_addrs=127.22.167.126:37819
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5248
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.167.67
--log_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786336863388938-23197-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 03:54:01 UTC on ccycloud.quasar-icijmt.root.comops.site
build id 4577009
I20260810 04:41:44.563017 5248 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 04:41:44.563220 5248 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 04:41:44.567870 5259 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 04:41:44.567926 5257 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 04:41:44.567871 5267 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 04:41:44.567970 5248 server_base.cc:1034] running on GCE node
I20260810 04:41:44.568243 5248 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 04:41:44.568455 5248 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 04:41:44.569579 5248 hybrid_clock.cc:648] HybridClock initialized: now 1786336904569570 us; error 27 us; skew 500 ppm
Aug 10 04:41:44 dist-test-slave-bqcl krb5kdc[4634](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786336904, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.167.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 04:41:44.571923 5248 init.cc:377] Logged in from keytab as kudu/127.22.167.67@KRBTEST.COM (short username kudu)
I20260810 04:41:44.572340 5248 webserver.cc:460] Webserver started at http://127.22.167.67:33817/ using document root <none> and password file <none>
I20260810 04:41:44.572512 5248 fs_manager.cc:359] Metadata directory not provided
I20260810 04:41:44.572551 5248 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 04:41:44.572681 5248 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 04:41:44.573740 5248 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786336863388938-23197-0/minicluster-data/ts-2/data/instance:
uuid: "63933b546fa543a096a1ab82a2479cbc"
format_stamp: "Formatted at 2026-08-10 04:41:44 on dist-test-slave-bqcl"
server_key: "1e31c928a0a081d514b25645de4119a9"
server_key_iv: "1f30699aed7d56a62c08d73a0ead9901"
server_key_version: "encryptionkey@0"
I20260810 04:41:44.574048 5248 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786336863388938-23197-0/minicluster-data/ts-2/wal/instance:
uuid: "63933b546fa543a096a1ab82a2479cbc"
format_stamp: "Formatted at 2026-08-10 04:41:44 on dist-test-slave-bqcl"
server_key: "1e31c928a0a081d514b25645de4119a9"
server_key_iv: "1f30699aed7d56a62c08d73a0ead9901"
server_key_version: "encryptionkey@0"
I20260810 04:41:44.575289 5248 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260810 04:41:44.575887 5280 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 04:41:44.576026 5248 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260810 04:41:44.576104 5248 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786336863388938-23197-0/minicluster-data/ts-2/data,/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786336863388938-23197-0/minicluster-data/ts-2/wal
uuid: "63933b546fa543a096a1ab82a2479cbc"
format_stamp: "Formatted at 2026-08-10 04:41:44 on dist-test-slave-bqcl"
server_key: "1e31c928a0a081d514b25645de4119a9"
server_key_iv: "1f30699aed7d56a62c08d73a0ead9901"
server_key_version: "encryptionkey@0"
I20260810 04:41:44.576175 5248 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786336863388938-23197-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786336863388938-23197-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786336863388938-23197-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 04:41:44.593151 5248 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 04:41:44.594831 5248 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 04:41:44.594954 5248 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 04:41:44.595151 5248 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 04:41:44.595436 5248 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 04:41:44.595468 5248 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 04:41:44.595489 5248 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 04:41:44.595511 5248 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 04:41:44.600924 5248 rpc_server.cc:307] RPC server started. Bound to: 127.22.167.67:35303
I20260810 04:41:44.600961 5482 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.167.67:35303 every 8 connection(s)
I20260810 04:41:44.601240 5248 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786336863388938-23197-0/minicluster-data/ts-2/data/info.pb
I20260810 04:41:44.604703 23197 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskemv_AM/build/release/bin/kudu as pid 5248
I20260810 04:41:44.604811 23197 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786336863388938-23197-0/minicluster-data/ts-2/wal/instance
I20260810 04:41:44.604996 23197 external_mini_cluster.cc:1428] Setting key 341be3028a8aabff3e987c6ff46b3383
Aug 10 04:41:44 dist-test-slave-bqcl krb5kdc[4634](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786336904, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.167.67@KRBTEST.COM for kudu/127.22.167.126@KRBTEST.COM
I20260810 04:41:44.609431 4809 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:41:44.601879 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.167.67:59739 (local address 127.22.167.126:37819)
0810 04:41:44.601937 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0810 04:41:44.601939 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 04:41:44.602098 (+ 159us) server_negotiation.cc:407] Connection header received
0810 04:41:44.602601 (+ 503us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:41:44.602602 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:41:44.602611 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:41:44.602652 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0810 04:41:44.603474 (+ 822us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:41:44.603952 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:41:44.605051 (+ 1099us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:41:44.605332 (+ 281us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:41:44.605397 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:41:44.607414 (+ 2017us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 04:41:44.607415 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 04:41:44.607416 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 04:41:44.607422 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0810 04:41:44.608454 (+ 1032us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 04:41:44.608781 (+ 327us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 04:41:44.608781 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 04:41:44.608782 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 04:41:44.608825 (+ 43us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 04:41:44.608993 (+ 168us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 04:41:44.608993 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 04:41:44.608993 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 04:41:44.609079 (+ 86us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 04:41:44.609122 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0810 04:41:44.609318 (+ 196us) server_negotiation.cc:299] Negotiation successful
0810 04:41:44.609370 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260810 04:41:44.609580 5487 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:41:44.601842 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.126:37819 (local address 127.22.167.67:59739)
0810 04:41:44.601984 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0810 04:41:44.602025 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0810 04:41:44.602504 (+ 479us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 04:41:44.602705 (+ 201us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 04:41:44.602708 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 04:41:44.603229 (+ 521us) client_negotiation.cc:189] Negotiated authn=SASL
0810 04:41:44.603411 (+ 182us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:41:44.603415 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:41:44.604009 (+ 594us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:41:44.604009 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:41:44.604963 (+ 954us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:41:44.604966 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:41:44.605414 (+ 448us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:41:44.605415 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:41:44.605557 (+ 142us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:41:44.606009 (+ 452us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 04:41:44.606018 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0810 04:41:44.607332 (+ 1314us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 04:41:44.608515 (+ 1183us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 04:41:44.608520 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 04:41:44.608522 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 04:41:44.608726 (+ 204us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 04:41:44.608867 (+ 141us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 04:41:44.608868 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 04:41:44.608868 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 04:41:44.608953 (+ 85us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 04:41:44.609118 (+ 165us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 04:41:44.609121 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 04:41:44.609226 (+ 105us) client_negotiation.cc:769] Sending connection context
0810 04:41:44.609299 (+ 73us) client_negotiation.cc:240] Negotiation successful
0810 04:41:44.609406 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":103,"thread_start_us":75,"threads_started":1}
I20260810 04:41:44.609975 5483 heartbeater.cc:347] Connected to a master server at 127.22.167.126:37819
I20260810 04:41:44.610054 5483 heartbeater.cc:464] Registering TS with master...
I20260810 04:41:44.610234 5483 heartbeater.cc:511] Master 127.22.167.126:37819 requested a full tablet report, sending...
I20260810 04:41:44.610562 4722 ts_manager.cc:194] Registered new tserver with Master: 63933b546fa543a096a1ab82a2479cbc (127.22.167.67:35303)
I20260810 04:41:44.610901 4722 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.167.67@KRBTEST.COM'} at 127.22.167.67:59739
I20260810 04:41:44.616504 23197 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.22.167.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.22.167.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.22.167.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786336863388938-23197-0/krb5kdc/oryx_127.22.167.65.keytab.
Entry for principal oryx/127.22.167.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786336863388938-23197-0/krb5kdc/oryx_127.22.167.65.keytab.
WARNING: no policy specified for oryx/127.22.167.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.22.167.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.22.167.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786336863388938-23197-0/krb5kdc/oryx_127.22.167.66.keytab.
Entry for principal oryx/127.22.167.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786336863388938-23197-0/krb5kdc/oryx_127.22.167.66.keytab.
WARNING: no policy specified for oryx/127.22.167.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.22.167.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.22.167.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786336863388938-23197-0/krb5kdc/oryx_127.22.167.67.keytab.
Entry for principal oryx/127.22.167.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786336863388938-23197-0/krb5kdc/oryx_127.22.167.67.keytab.
I20260810 04:41:44.680631 23197 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskemv_AM/build/release/bin/kudu with pid 4817
I20260810 04:41:44.694254 23197 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskemv_AM/build/release/bin/kudu with pid 5041
I20260810 04:41:44.703305 23197 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskemv_AM/build/release/bin/kudu with pid 5248
I20260810 04:41:44.709072 23197 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskemv_AM/build/release/bin/kudu with pid 4659
I20260810 04:41:44.719686 23197 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskemv_AM/build/release/bin/kudu
/tmp/dist-test-taskemv_AM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786336863388938-23197-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786336863388938-23197-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786336863388938-23197-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786336863388938-23197-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.167.126:37819
--webserver_interface=127.22.167.126
--webserver_port=39245
--keytab_file=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786336863388938-23197-0/krb5kdc/kudu_127.22.167.126.keytab
--principal=kudu/127.22.167.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.167.84:41851
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.167.126:37819
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786336863388938-23197-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786336863388938-23197-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786336863388938-23197-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 04:41:44.799397 5495 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 04:41:44.799610 5495 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 04:41:44.799641 5495 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 04:41:44.800746 5495 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 04:41:44.800788 5495 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 04:41:44.800807 5495 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 04:41:44.800824 5495 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 04:41:44.800839 5495 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 04:41:44.802300 5495 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.167.84:41851
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786336863388938-23197-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786336863388938-23197-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.22.167.126:37819
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786336863388938-23197-0/krb5kdc/kudu_127.22.167.126.keytab
--principal=kudu/127.22.167.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.167.126:37819
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786336863388938-23197-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.167.126
--webserver_port=39245
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5495
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786336863388938-23197-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 03:54:01 UTC on ccycloud.quasar-icijmt.root.comops.site
build id 4577009
I20260810 04:41:44.802542 5495 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 04:41:44.802743 5495 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 04:41:44.807291 5510 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 04:41:44.807319 5508 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 04:41:44.807297 5514 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 04:41:44.807502 5495 server_base.cc:1034] running on GCE node
I20260810 04:41:44.807783 5495 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 04:41:44.808028 5495 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 04:41:44.809156 5495 hybrid_clock.cc:648] HybridClock initialized: now 1786336904809142 us; error 30 us; skew 500 ppm
Aug 10 04:41:44 dist-test-slave-bqcl krb5kdc[4634](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786336904, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.167.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 04:41:44.811457 5495 init.cc:377] Logged in from keytab as kudu/127.22.167.126@KRBTEST.COM (short username kudu)
I20260810 04:41:44.811837 5495 webserver.cc:460] Webserver started at http://127.22.167.126:39245/ using document root <none> and password file <none>
I20260810 04:41:44.812017 5495 fs_manager.cc:359] Metadata directory not provided
I20260810 04:41:44.812075 5495 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 04:41:44.813311 5495 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260810 04:41:44.813915 5529 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 04:41:44.814076 5495 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260810 04:41:44.814193 5495 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786336863388938-23197-0/minicluster-data/master-0/data,/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786336863388938-23197-0/minicluster-data/master-0/wal
uuid: "75b33e7bf0f44523b3341a1681bc0578"
format_stamp: "Formatted at 2026-08-10 04:41:44 on dist-test-slave-bqcl"
server_key: "1b66466479ff8daab3a4146feb831faf"
server_key_iv: "858d63308572e9f4656dd31b0c5dfefa"
server_key_version: "encryptionkey@0"
I20260810 04:41:44.814478 5495 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786336863388938-23197-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786336863388938-23197-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786336863388938-23197-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 04:41:44.851047 5495 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 04:41:44.852779 5495 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 04:41:44.852890 5495 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 04:41:44.856630 5495 rpc_server.cc:307] RPC server started. Bound to: 127.22.167.126:37819
I20260810 04:41:44.856678 5611 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.167.126:37819 every 8 connection(s)
I20260810 04:41:44.857007 5495 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786336863388938-23197-0/minicluster-data/master-0/data/info.pb
I20260810 04:41:44.858431 5612 sys_catalog.cc:263] Verifying existing consensus state
I20260810 04:41:44.859073 5612 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 75b33e7bf0f44523b3341a1681bc0578: Bootstrap starting.
I20260810 04:41:44.864387 23197 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskemv_AM/build/release/bin/kudu as pid 5495
I20260810 04:41:44.864662 23197 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskemv_AM/build/release/bin/kudu
/tmp/dist-test-taskemv_AM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786336863388938-23197-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786336863388938-23197-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786336863388938-23197-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786336863388938-23197-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.167.65:44213
--local_ip_for_outbound_sockets=127.22.167.65
--tserver_master_addrs=127.22.167.126:37819
--webserver_port=39965
--webserver_interface=127.22.167.65
--keytab_file=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786336863388938-23197-0/krb5kdc/kudu_127.22.167.65.keytab
--principal=kudu/127.22.167.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.167.84:41851
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.22.167.65
--keytab_file=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786336863388938-23197-0/krb5kdc/oryx_127.22.167.65.keytab with env {KRB5CCNAME=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786336863388938-23197-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786336863388938-23197-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786336863388938-23197-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 04:41:44.867431 5612 log.cc:824] T 00000000000000000000000000000000 P 75b33e7bf0f44523b3341a1681bc0578: Log is configured to *not* fsync() on all Append() calls
I20260810 04:41:44.869055 5612 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 75b33e7bf0f44523b3341a1681bc0578: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260810 04:41:44.869297 5612 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 75b33e7bf0f44523b3341a1681bc0578: Bootstrap complete.
I20260810 04:41:44.871387 5612 raft_consensus.cc:348] T 00000000000000000000000000000000 P 75b33e7bf0f44523b3341a1681bc0578 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "75b33e7bf0f44523b3341a1681bc0578" member_type: VOTER last_known_addr { host: "127.22.167.126" port: 37819 } }
I20260810 04:41:44.871603 5612 raft_consensus.cc:729] T 00000000000000000000000000000000 P 75b33e7bf0f44523b3341a1681bc0578 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 75b33e7bf0f44523b3341a1681bc0578, State: Initialized, Role: FOLLOWER
I20260810 04:41:44.871768 5612 consensus_queue.cc:260] T 00000000000000000000000000000000 P 75b33e7bf0f44523b3341a1681bc0578 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "75b33e7bf0f44523b3341a1681bc0578" member_type: VOTER last_known_addr { host: "127.22.167.126" port: 37819 } }
I20260810 04:41:44.871841 5612 raft_consensus.cc:388] T 00000000000000000000000000000000 P 75b33e7bf0f44523b3341a1681bc0578 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 04:41:44.871891 5612 raft_consensus.cc:482] T 00000000000000000000000000000000 P 75b33e7bf0f44523b3341a1681bc0578 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 04:41:44.871937 5612 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 75b33e7bf0f44523b3341a1681bc0578 [term 1 FOLLOWER]: Advancing to term 2
I20260810 04:41:44.872850 5612 raft_consensus.cc:504] T 00000000000000000000000000000000 P 75b33e7bf0f44523b3341a1681bc0578 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "75b33e7bf0f44523b3341a1681bc0578" member_type: VOTER last_known_addr { host: "127.22.167.126" port: 37819 } }
I20260810 04:41:44.872946 5612 leader_election.cc:302] T 00000000000000000000000000000000 P 75b33e7bf0f44523b3341a1681bc0578 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 75b33e7bf0f44523b3341a1681bc0578; no voters:
I20260810 04:41:44.873097 5612 leader_election.cc:288] T 00000000000000000000000000000000 P 75b33e7bf0f44523b3341a1681bc0578 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260810 04:41:44.873143 5618 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 75b33e7bf0f44523b3341a1681bc0578 [term 2 FOLLOWER]: Leader election won for term 2
I20260810 04:41:44.873279 5618 raft_consensus.cc:686] T 00000000000000000000000000000000 P 75b33e7bf0f44523b3341a1681bc0578 [term 2 LEADER]: Becoming Leader. State: Replica: 75b33e7bf0f44523b3341a1681bc0578, State: Running, Role: LEADER
I20260810 04:41:44.873358 5612 sys_catalog.cc:564] T 00000000000000000000000000000000 P 75b33e7bf0f44523b3341a1681bc0578 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 04:41:44.873366 5618 consensus_queue.cc:237] T 00000000000000000000000000000000 P 75b33e7bf0f44523b3341a1681bc0578 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "75b33e7bf0f44523b3341a1681bc0578" member_type: VOTER last_known_addr { host: "127.22.167.126" port: 37819 } }
I20260810 04:41:44.873679 5621 sys_catalog.cc:455] T 00000000000000000000000000000000 P 75b33e7bf0f44523b3341a1681bc0578 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 75b33e7bf0f44523b3341a1681bc0578. Latest consensus state: current_term: 2 leader_uuid: "75b33e7bf0f44523b3341a1681bc0578" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "75b33e7bf0f44523b3341a1681bc0578" member_type: VOTER last_known_addr { host: "127.22.167.126" port: 37819 } } }
I20260810 04:41:44.873674 5620 sys_catalog.cc:455] T 00000000000000000000000000000000 P 75b33e7bf0f44523b3341a1681bc0578 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "75b33e7bf0f44523b3341a1681bc0578" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "75b33e7bf0f44523b3341a1681bc0578" member_type: VOTER last_known_addr { host: "127.22.167.126" port: 37819 } } }
I20260810 04:41:44.873780 5621 sys_catalog.cc:458] T 00000000000000000000000000000000 P 75b33e7bf0f44523b3341a1681bc0578 [sys.catalog]: This master's current role is: LEADER
I20260810 04:41:44.873802 5620 sys_catalog.cc:458] T 00000000000000000000000000000000 P 75b33e7bf0f44523b3341a1681bc0578 [sys.catalog]: This master's current role is: LEADER
I20260810 04:41:44.874032 5630 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 04:41:44.874631 5630 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 04:41:44.874984 5630 catalog_manager.cc:1259] Loaded cluster ID: 6409a70d916b4d9abf80e9cce1e75f79
I20260810 04:41:44.875025 5630 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 04:41:44.876228 5630 catalog_manager.cc:1504] Loading token signing keys...
I20260810 04:41:44.876394 5630 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 75b33e7bf0f44523b3341a1681bc0578: Loaded TSK: 0
I20260810 04:41:44.876605 5630 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260810 04:41:44.946341 5615 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 04:41:44.946532 5615 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 04:41:44.946553 5615 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 04:41:44.947705 5615 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 04:41:44.947747 5615 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 04:41:44.947772 5615 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.167.65
I20260810 04:41:44.949165 5615 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.167.84:41851
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786336863388938-23197-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786336863388938-23197-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786336863388938-23197-0/krb5kdc/oryx_127.22.167.65.keytab
--principal=oryx/127.22.167.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.167.65:44213
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786336863388938-23197-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.22.167.65
--webserver_port=39965
--tserver_master_addrs=127.22.167.126:37819
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5615
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.167.65
--log_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786336863388938-23197-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 03:54:01 UTC on ccycloud.quasar-icijmt.root.comops.site
build id 4577009
I20260810 04:41:44.949383 5615 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 04:41:44.949589 5615 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 04:41:44.954345 5657 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 04:41:44.954319 5668 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 04:41:44.954336 5655 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 04:41:44.954509 5615 server_base.cc:1034] running on GCE node
I20260810 04:41:44.954766 5615 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 04:41:44.955041 5615 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 04:41:44.956216 5615 hybrid_clock.cc:648] HybridClock initialized: now 1786336904956198 us; error 33 us; skew 500 ppm
Aug 10 04:41:44 dist-test-slave-bqcl krb5kdc[4634](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786336904, etypes {rep=17 tkt=17 ses=17}, oryx/127.22.167.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 04:41:44.959205 5615 init.cc:377] Logged in from keytab as oryx/127.22.167.65@KRBTEST.COM (short username oryx)
I20260810 04:41:44.959748 5615 webserver.cc:460] Webserver started at http://127.22.167.65:39965/ using document root <none> and password file <none>
I20260810 04:41:44.960004 5615 fs_manager.cc:359] Metadata directory not provided
I20260810 04:41:44.960078 5615 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 04:41:44.961709 5615 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.002s
I20260810 04:41:44.962361 5681 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 04:41:44.962536 5615 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260810 04:41:44.962630 5615 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786336863388938-23197-0/minicluster-data/ts-0/data,/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786336863388938-23197-0/minicluster-data/ts-0/wal
uuid: "b8aa536af5144554a53fc0f1a744a64a"
format_stamp: "Formatted at 2026-08-10 04:41:44 on dist-test-slave-bqcl"
server_key: "0ae9c66349d57715f58025da7d4e1f78"
server_key_iv: "73bad8c3de589f49db7c18a7ff257ba9"
server_key_version: "encryptionkey@0"
I20260810 04:41:44.963052 5615 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786336863388938-23197-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786336863388938-23197-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786336863388938-23197-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 04:41:44.987197 5615 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 04:41:44.989326 5615 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 04:41:44.989470 5615 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 04:41:44.989753 5615 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 04:41:44.990079 5615 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 04:41:44.990145 5615 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 04:41:44.990183 5615 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 04:41:44.990206 5615 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 04:41:44.997252 5615 rpc_server.cc:307] RPC server started. Bound to: 127.22.167.65:44213
I20260810 04:41:44.997287 5870 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.167.65:44213 every 8 connection(s)
I20260810 04:41:44.997722 5615 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786336863388938-23197-0/minicluster-data/ts-0/data/info.pb
I20260810 04:41:45.000315 23197 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskemv_AM/build/release/bin/kudu as pid 5615
I20260810 04:41:45.000633 23197 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskemv_AM/build/release/bin/kudu
/tmp/dist-test-taskemv_AM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786336863388938-23197-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786336863388938-23197-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786336863388938-23197-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786336863388938-23197-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.167.66:42063
--local_ip_for_outbound_sockets=127.22.167.66
--tserver_master_addrs=127.22.167.126:37819
--webserver_port=43373
--webserver_interface=127.22.167.66
--keytab_file=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786336863388938-23197-0/krb5kdc/kudu_127.22.167.66.keytab
--principal=kudu/127.22.167.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.167.84:41851
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.22.167.66
--keytab_file=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786336863388938-23197-0/krb5kdc/oryx_127.22.167.66.keytab with env {KRB5CCNAME=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786336863388938-23197-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786336863388938-23197-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786336863388938-23197-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 10 04:41:45 dist-test-slave-bqcl krb5kdc[4634](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.167.65@KRBTEST.COM for oryx/127.22.167.126@KRBTEST.COM, Server not found in Kerberos database
Aug 10 04:41:45 dist-test-slave-bqcl krb5kdc[4634](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.167.65@KRBTEST.COM for oryx/127.22.167.126@KRBTEST.COM, Server not found in Kerberos database
I20260810 04:41:45.008567 5875 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:41:44.998466 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.126:37819 (local address 127.22.167.65:39747)
0810 04:41:44.998633 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0810 04:41:44.998687 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0810 04:41:44.999143 (+ 456us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 04:41:45.001083 (+ 1940us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 04:41:45.001088 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 04:41:45.001842 (+ 754us) client_negotiation.cc:189] Negotiated authn=SASL
0810 04:41:45.002118 (+ 276us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:41:45.002148 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:41:45.003652 (+ 1504us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:41:45.003653 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:41:45.004812 (+ 1159us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:41:45.004815 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:41:45.005423 (+ 608us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:41:45.005424 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:41:45.005556 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:41:45.006068 (+ 512us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 04:41:45.006075 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0810 04:41:45.008407 (+ 2332us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.167.126:37819: Server oryx/127.22.167.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":81,"threads_started":1}
W20260810 04:41:45.008775 5879 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 04:41:44.998541 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.167.65:39747 (local address 127.22.167.126:37819)
0810 04:41:45.000529 (+ 1988us) server_negotiation.cc:206] Beginning negotiation
0810 04:41:45.000540 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0810 04:41:45.000583 (+ 43us) server_negotiation.cc:407] Connection header received
0810 04:41:45.000727 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:41:45.000735 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:41:45.000798 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:41:45.000929 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0810 04:41:45.002282 (+ 1353us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:41:45.003534 (+ 1252us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:41:45.004899 (+ 1365us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:41:45.005317 (+ 418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:41:45.005420 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:41:45.008582 (+ 3162us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.167.65:39747: BlockingRecv error: recv got EOF from 127.22.167.65:39747 (error 108)
Metrics: {"server-negotiator.queue_time_us":1923,"thread_start_us":88,"threads_started":1}
W20260810 04:41:45.008791 5873 heartbeater.cc:644] Failed to heartbeat to 127.22.167.126:37819 (0 consecutive failures): Not authorized: Failed to ping master at 127.22.167.126:37819: Client connection negotiation failed: client connection to 127.22.167.126:37819: Server oryx/127.22.167.126@KRBTEST.COM not found in Kerberos database
Aug 10 04:41:45 dist-test-slave-bqcl krb5kdc[4634](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.167.65@KRBTEST.COM for oryx/127.22.167.126@KRBTEST.COM, Server not found in Kerberos database
Aug 10 04:41:45 dist-test-slave-bqcl krb5kdc[4634](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.167.65@KRBTEST.COM for oryx/127.22.167.126@KRBTEST.COM, Server not found in Kerberos database
I20260810 04:41:45.013342 5875 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:41:45.009055 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.126:37819 (local address 127.22.167.65:49587)
0810 04:41:45.009107 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0810 04:41:45.009119 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0810 04:41:45.009164 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 04:41:45.009295 (+ 131us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 04:41:45.009296 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 04:41:45.009406 (+ 110us) client_negotiation.cc:189] Negotiated authn=SASL
0810 04:41:45.009453 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:41:45.009454 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:41:45.010312 (+ 858us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:41:45.010312 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:41:45.010802 (+ 490us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:41:45.010803 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:41:45.011225 (+ 422us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:41:45.011225 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:41:45.011268 (+ 43us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:41:45.011671 (+ 403us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 04:41:45.011674 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0810 04:41:45.013298 (+ 1624us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.167.126:37819: Server oryx/127.22.167.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":32}
W20260810 04:41:45.013458 5879 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 04:41:45.009106 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.167.65:49587 (local address 127.22.167.126:37819)
0810 04:41:45.009160 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0810 04:41:45.009162 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 04:41:45.009172 (+ 10us) server_negotiation.cc:407] Connection header received
0810 04:41:45.009218 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:41:45.009219 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:41:45.009232 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:41:45.009294 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0810 04:41:45.009513 (+ 219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:41:45.010237 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:41:45.010877 (+ 640us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:41:45.011158 (+ 281us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:41:45.011232 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:41:45.013399 (+ 2167us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.167.65:49587: BlockingRecv error: recv got EOF from 127.22.167.65:49587 (error 108)
Metrics: {"server-negotiator.queue_time_us":32}
Aug 10 04:41:45 dist-test-slave-bqcl krb5kdc[4634](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.167.65@KRBTEST.COM for oryx/127.22.167.126@KRBTEST.COM, Server not found in Kerberos database
Aug 10 04:41:45 dist-test-slave-bqcl krb5kdc[4634](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.167.65@KRBTEST.COM for oryx/127.22.167.126@KRBTEST.COM, Server not found in Kerberos database
I20260810 04:41:45.017916 5875 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:41:45.013640 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.126:37819 (local address 127.22.167.65:58931)
0810 04:41:45.013742 (+ 102us) negotiation.cc:107] Waiting for socket to connect
0810 04:41:45.013750 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0810 04:41:45.013787 (+ 37us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 04:41:45.013895 (+ 108us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 04:41:45.013896 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 04:41:45.013993 (+ 97us) client_negotiation.cc:189] Negotiated authn=SASL
0810 04:41:45.014025 (+ 32us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:41:45.014026 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:41:45.014892 (+ 866us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:41:45.014893 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:41:45.015368 (+ 475us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:41:45.015370 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:41:45.015889 (+ 519us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:41:45.015890 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:41:45.015946 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:41:45.016302 (+ 356us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 04:41:45.016306 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0810 04:41:45.017873 (+ 1567us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.167.126:37819: Server oryx/127.22.167.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":90}
W20260810 04:41:45.018003 5879 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 04:41:45.013683 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.167.65:58931 (local address 127.22.167.126:37819)
0810 04:41:45.013715 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0810 04:41:45.013717 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 04:41:45.013779 (+ 62us) server_negotiation.cc:407] Connection header received
0810 04:41:45.013823 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:41:45.013824 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:41:45.013834 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:41:45.013893 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0810 04:41:45.014075 (+ 182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:41:45.014815 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:41:45.015499 (+ 684us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:41:45.015801 (+ 302us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:41:45.015861 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:41:45.017948 (+ 2087us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.167.65:58931: BlockingRecv error: recv got EOF from 127.22.167.65:58931 (error 108)
Metrics: {"server-negotiator.queue_time_us":15}
W20260810 04:41:45.018181 5873 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260810 04:41:45.085278 5880 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 04:41:45.085470 5880 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 04:41:45.085491 5880 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 04:41:45.086615 5880 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 04:41:45.086656 5880 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 04:41:45.086680 5880 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.167.66
I20260810 04:41:45.088025 5880 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.167.84:41851
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786336863388938-23197-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786336863388938-23197-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786336863388938-23197-0/krb5kdc/oryx_127.22.167.66.keytab
--principal=oryx/127.22.167.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.167.66:42063
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786336863388938-23197-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.22.167.66
--webserver_port=43373
--tserver_master_addrs=127.22.167.126:37819
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5880
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.167.66
--log_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786336863388938-23197-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 03:54:01 UTC on ccycloud.quasar-icijmt.root.comops.site
build id 4577009
I20260810 04:41:45.088207 5880 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 04:41:45.088384 5880 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 04:41:45.093005 5897 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 04:41:45.093032 5892 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 04:41:45.093129 5880 server_base.cc:1034] running on GCE node
W20260810 04:41:45.093004 5894 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 04:41:45.093398 5880 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 04:41:45.093621 5880 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 04:41:45.094766 5880 hybrid_clock.cc:648] HybridClock initialized: now 1786336905094752 us; error 27 us; skew 500 ppm
Aug 10 04:41:45 dist-test-slave-bqcl krb5kdc[4634](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786336905, etypes {rep=17 tkt=17 ses=17}, oryx/127.22.167.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 04:41:45.097054 5880 init.cc:377] Logged in from keytab as oryx/127.22.167.66@KRBTEST.COM (short username oryx)
I20260810 04:41:45.097448 5880 webserver.cc:460] Webserver started at http://127.22.167.66:43373/ using document root <none> and password file <none>
I20260810 04:41:45.097637 5880 fs_manager.cc:359] Metadata directory not provided
I20260810 04:41:45.097707 5880 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 04:41:45.098981 5880 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260810 04:41:45.099471 5908 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 04:41:45.099618 5880 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260810 04:41:45.099707 5880 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786336863388938-23197-0/minicluster-data/ts-1/data,/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786336863388938-23197-0/minicluster-data/ts-1/wal
uuid: "cf0f39900c034704822cbd1fec7cbc39"
format_stamp: "Formatted at 2026-08-10 04:41:44 on dist-test-slave-bqcl"
server_key: "6ec9bce4caa595e5c7f372ebdde5825d"
server_key_iv: "e7d9304b55d9c887198bae6215e5d54a"
server_key_version: "encryptionkey@0"
I20260810 04:41:45.100008 5880 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786336863388938-23197-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786336863388938-23197-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786336863388938-23197-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 04:41:45.109411 5880 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 04:41:45.111032 5880 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 04:41:45.111155 5880 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 04:41:45.111344 5880 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 04:41:45.111608 5880 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 04:41:45.111641 5880 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 04:41:45.111677 5880 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 04:41:45.111702 5880 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 04:41:45.117246 6092 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.167.66:42063 every 8 connection(s)
I20260810 04:41:45.117295 5880 rpc_server.cc:307] RPC server started. Bound to: 127.22.167.66:42063
I20260810 04:41:45.117640 5880 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786336863388938-23197-0/minicluster-data/ts-1/data/info.pb
Aug 10 04:41:45 dist-test-slave-bqcl krb5kdc[4634](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.167.66@KRBTEST.COM for oryx/127.22.167.126@KRBTEST.COM, Server not found in Kerberos database
Aug 10 04:41:45 dist-test-slave-bqcl krb5kdc[4634](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.167.66@KRBTEST.COM for oryx/127.22.167.126@KRBTEST.COM, Server not found in Kerberos database
W20260810 04:41:45.124007 5879 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 04:41:45.118298 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.167.66:38909 (local address 127.22.167.126:37819)
0810 04:41:45.118338 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0810 04:41:45.118341 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 04:41:45.118522 (+ 181us) server_negotiation.cc:407] Connection header received
0810 04:41:45.118968 (+ 446us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:41:45.118970 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:41:45.118983 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:41:45.119027 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0810 04:41:45.119866 (+ 839us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:41:45.120349 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:41:45.121236 (+ 887us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:41:45.121406 (+ 170us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:41:45.121449 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:41:45.123960 (+ 2511us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.167.66:38909: BlockingRecv error: recv got EOF from 127.22.167.66:38909 (error 108)
Metrics: {"server-negotiator.queue_time_us":17}
I20260810 04:41:45.124012 6096 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:41:45.118283 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.126:37819 (local address 127.22.167.66:38909)
0810 04:41:45.118426 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0810 04:41:45.118459 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0810 04:41:45.118894 (+ 435us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 04:41:45.119083 (+ 189us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 04:41:45.119086 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 04:41:45.119615 (+ 529us) client_negotiation.cc:189] Negotiated authn=SASL
0810 04:41:45.119801 (+ 186us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:41:45.119806 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:41:45.120407 (+ 601us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:41:45.120407 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:41:45.121178 (+ 771us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:41:45.121180 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:41:45.121456 (+ 276us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:41:45.121457 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:41:45.121545 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:41:45.121925 (+ 380us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 04:41:45.121932 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0810 04:41:45.123880 (+ 1948us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.167.126:37819: Server oryx/127.22.167.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":106,"thread_start_us":79,"threads_started":1}
W20260810 04:41:45.124215 6093 heartbeater.cc:644] Failed to heartbeat to 127.22.167.126:37819 (0 consecutive failures): Not authorized: Failed to ping master at 127.22.167.126:37819: Client connection negotiation failed: client connection to 127.22.167.126:37819: Server oryx/127.22.167.126@KRBTEST.COM not found in Kerberos database
Aug 10 04:41:45 dist-test-slave-bqcl krb5kdc[4634](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.167.66@KRBTEST.COM for oryx/127.22.167.126@KRBTEST.COM, Server not found in Kerberos database
I20260810 04:41:45.126931 23197 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskemv_AM/build/release/bin/kudu as pid 5880
I20260810 04:41:45.127182 23197 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskemv_AM/build/release/bin/kudu
/tmp/dist-test-taskemv_AM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786336863388938-23197-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786336863388938-23197-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786336863388938-23197-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786336863388938-23197-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.167.67:35303
--local_ip_for_outbound_sockets=127.22.167.67
--tserver_master_addrs=127.22.167.126:37819
--webserver_port=33817
--webserver_interface=127.22.167.67
--keytab_file=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786336863388938-23197-0/krb5kdc/kudu_127.22.167.67.keytab
--principal=kudu/127.22.167.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.167.84:41851
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.22.167.67
--keytab_file=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786336863388938-23197-0/krb5kdc/oryx_127.22.167.67.keytab with env {KRB5CCNAME=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786336863388938-23197-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786336863388938-23197-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786336863388938-23197-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 10 04:41:45 dist-test-slave-bqcl krb5kdc[4634](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.167.66@KRBTEST.COM for oryx/127.22.167.126@KRBTEST.COM, Server not found in Kerberos database
I20260810 04:41:45.127888 6096 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:41:45.124402 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.126:37819 (local address 127.22.167.66:37731)
0810 04:41:45.124442 (+ 40us) negotiation.cc:107] Waiting for socket to connect
0810 04:41:45.124452 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0810 04:41:45.124491 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 04:41:45.124612 (+ 121us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 04:41:45.124613 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 04:41:45.124717 (+ 104us) client_negotiation.cc:189] Negotiated authn=SASL
0810 04:41:45.124758 (+ 41us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:41:45.124759 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:41:45.125290 (+ 531us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:41:45.125290 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:41:45.125750 (+ 460us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:41:45.125751 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:41:45.126000 (+ 249us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:41:45.126001 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:41:45.126041 (+ 40us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:41:45.126352 (+ 311us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 04:41:45.126355 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0810 04:41:45.127834 (+ 1479us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.167.126:37819: Server oryx/127.22.167.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":24}
W20260810 04:41:45.127955 5879 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 04:41:45.124461 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.167.66:37731 (local address 127.22.167.126:37819)
0810 04:41:45.124505 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0810 04:41:45.124507 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 04:41:45.124516 (+ 9us) server_negotiation.cc:407] Connection header received
0810 04:41:45.124534 (+ 18us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:41:45.124535 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:41:45.124542 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:41:45.124583 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0810 04:41:45.124807 (+ 224us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:41:45.125242 (+ 435us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:41:45.125798 (+ 556us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:41:45.125959 (+ 161us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:41:45.125997 (+ 38us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:41:45.127905 (+ 1908us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.167.66:37731: BlockingRecv error: recv got EOF from 127.22.167.66:37731 (error 108)
Metrics: {"server-negotiator.queue_time_us":23}
Aug 10 04:41:45 dist-test-slave-bqcl krb5kdc[4634](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.167.66@KRBTEST.COM for oryx/127.22.167.126@KRBTEST.COM, Server not found in Kerberos database
Aug 10 04:41:45 dist-test-slave-bqcl krb5kdc[4634](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.167.66@KRBTEST.COM for oryx/127.22.167.126@KRBTEST.COM, Server not found in Kerberos database
I20260810 04:41:45.131982 6096 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:41:45.128203 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.126:37819 (local address 127.22.167.66:50551)
0810 04:41:45.128240 (+ 37us) negotiation.cc:107] Waiting for socket to connect
0810 04:41:45.128249 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0810 04:41:45.128281 (+ 32us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 04:41:45.128391 (+ 110us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 04:41:45.128392 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 04:41:45.128501 (+ 109us) client_negotiation.cc:189] Negotiated authn=SASL
0810 04:41:45.128524 (+ 23us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:41:45.128525 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:41:45.129295 (+ 770us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:41:45.129296 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:41:45.129751 (+ 455us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:41:45.129752 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:41:45.130145 (+ 393us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:41:45.130146 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:41:45.130200 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:41:45.130493 (+ 293us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 04:41:45.130498 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0810 04:41:45.131929 (+ 1431us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.167.126:37819: Server oryx/127.22.167.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":24}
W20260810 04:41:45.132045 5879 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 04:41:45.128256 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.167.66:50551 (local address 127.22.167.126:37819)
0810 04:41:45.128304 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0810 04:41:45.128306 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 04:41:45.128314 (+ 8us) server_negotiation.cc:407] Connection header received
0810 04:41:45.128331 (+ 17us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:41:45.128332 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:41:45.128339 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:41:45.128382 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0810 04:41:45.128584 (+ 202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:41:45.129208 (+ 624us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:41:45.129819 (+ 611us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:41:45.130056 (+ 237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:41:45.130115 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:41:45.132004 (+ 1889us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.167.66:50551: BlockingRecv error: recv got EOF from 127.22.167.66:50551 (error 108)
Metrics: {"server-negotiator.queue_time_us":28}
W20260810 04:41:45.132148 6093 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260810 04:41:45.209569 6098 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 04:41:45.209759 6098 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 04:41:45.209780 6098 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 04:41:45.210911 6098 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 04:41:45.210949 6098 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 04:41:45.210978 6098 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.167.67
I20260810 04:41:45.212364 6098 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.167.84:41851
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786336863388938-23197-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786336863388938-23197-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786336863388938-23197-0/krb5kdc/oryx_127.22.167.67.keytab
--principal=oryx/127.22.167.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.167.67:35303
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786336863388938-23197-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.22.167.67
--webserver_port=33817
--tserver_master_addrs=127.22.167.126:37819
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6098
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.167.67
--log_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786336863388938-23197-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 03:54:01 UTC on ccycloud.quasar-icijmt.root.comops.site
build id 4577009
I20260810 04:41:45.212574 6098 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 04:41:45.212776 6098 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 04:41:45.217459 6106 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 04:41:45.217504 6109 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 04:41:45.217542 6098 server_base.cc:1034] running on GCE node
W20260810 04:41:45.217489 6104 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 04:41:45.217886 6098 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 04:41:45.218145 6098 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 04:41:45.219336 6098 hybrid_clock.cc:648] HybridClock initialized: now 1786336905219320 us; error 32 us; skew 500 ppm
Aug 10 04:41:45 dist-test-slave-bqcl krb5kdc[4634](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786336905, etypes {rep=17 tkt=17 ses=17}, oryx/127.22.167.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 04:41:45.221645 6098 init.cc:377] Logged in from keytab as oryx/127.22.167.67@KRBTEST.COM (short username oryx)
I20260810 04:41:45.222044 6098 webserver.cc:460] Webserver started at http://127.22.167.67:33817/ using document root <none> and password file <none>
I20260810 04:41:45.222286 6098 fs_manager.cc:359] Metadata directory not provided
I20260810 04:41:45.222354 6098 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 04:41:45.223613 6098 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260810 04:41:45.224099 6121 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 04:41:45.224246 6098 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260810 04:41:45.224323 6098 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786336863388938-23197-0/minicluster-data/ts-2/data,/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786336863388938-23197-0/minicluster-data/ts-2/wal
uuid: "63933b546fa543a096a1ab82a2479cbc"
format_stamp: "Formatted at 2026-08-10 04:41:44 on dist-test-slave-bqcl"
server_key: "1e31c928a0a081d514b25645de4119a9"
server_key_iv: "1f30699aed7d56a62c08d73a0ead9901"
server_key_version: "encryptionkey@0"
I20260810 04:41:45.224622 6098 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786336863388938-23197-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786336863388938-23197-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786336863388938-23197-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 04:41:45.248971 6098 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 04:41:45.250732 6098 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 04:41:45.250867 6098 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 04:41:45.251077 6098 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 04:41:45.251371 6098 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 04:41:45.251405 6098 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 04:41:45.251433 6098 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 04:41:45.251453 6098 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 04:41:45.256696 6098 rpc_server.cc:307] RPC server started. Bound to: 127.22.167.67:35303
I20260810 04:41:45.256760 6325 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.167.67:35303 every 8 connection(s)
I20260810 04:41:45.257054 6098 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786336863388938-23197-0/minicluster-data/ts-2/data/info.pb
Aug 10 04:41:45 dist-test-slave-bqcl krb5kdc[4634](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.167.67@KRBTEST.COM for oryx/127.22.167.126@KRBTEST.COM, Server not found in Kerberos database
Aug 10 04:41:45 dist-test-slave-bqcl krb5kdc[4634](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.167.67@KRBTEST.COM for oryx/127.22.167.126@KRBTEST.COM, Server not found in Kerberos database
I20260810 04:41:45.263218 23197 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskemv_AM/build/release/bin/kudu as pid 6098
W20260810 04:41:45.263679 5879 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 04:41:45.257757 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.167.67:58571 (local address 127.22.167.126:37819)
0810 04:41:45.257816 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0810 04:41:45.257817 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0810 04:41:45.257975 (+ 158us) server_negotiation.cc:407] Connection header received
0810 04:41:45.258454 (+ 479us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:41:45.258455 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:41:45.258464 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:41:45.258516 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0810 04:41:45.259366 (+ 850us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:41:45.259847 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:41:45.260755 (+ 908us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:41:45.260927 (+ 172us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:41:45.260968 (+ 41us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:41:45.263615 (+ 2647us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.167.67:58571: BlockingRecv error: recv got EOF from 127.22.167.67:58571 (error 108)
Metrics: {"server-negotiator.queue_time_us":30}
I20260810 04:41:45.263711 6330 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:41:45.257714 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.126:37819 (local address 127.22.167.67:58571)
0810 04:41:45.257861 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0810 04:41:45.257901 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0810 04:41:45.258378 (+ 477us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 04:41:45.258571 (+ 193us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 04:41:45.258575 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 04:41:45.259107 (+ 532us) client_negotiation.cc:189] Negotiated authn=SASL
0810 04:41:45.259301 (+ 194us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:41:45.259305 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:41:45.259908 (+ 603us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:41:45.259909 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:41:45.260696 (+ 787us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:41:45.260699 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:41:45.260978 (+ 279us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:41:45.260979 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:41:45.261075 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:41:45.261472 (+ 397us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 04:41:45.261480 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0810 04:41:45.263520 (+ 2040us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.167.126:37819: Server oryx/127.22.167.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":76,"threads_started":1}
W20260810 04:41:45.263898 6326 heartbeater.cc:644] Failed to heartbeat to 127.22.167.126:37819 (0 consecutive failures): Not authorized: Failed to ping master at 127.22.167.126:37819: Client connection negotiation failed: client connection to 127.22.167.126:37819: Server oryx/127.22.167.126@KRBTEST.COM not found in Kerberos database
Aug 10 04:41:45 dist-test-slave-bqcl krb5kdc[4634](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.167.67@KRBTEST.COM for oryx/127.22.167.126@KRBTEST.COM, Server not found in Kerberos database
Aug 10 04:41:45 dist-test-slave-bqcl krb5kdc[4634](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.167.67@KRBTEST.COM for oryx/127.22.167.126@KRBTEST.COM, Server not found in Kerberos database
I20260810 04:41:45.268381 6330 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:41:45.264117 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.126:37819 (local address 127.22.167.67:51633)
0810 04:41:45.264161 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0810 04:41:45.264170 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0810 04:41:45.264212 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 04:41:45.264374 (+ 162us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 04:41:45.264374 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 04:41:45.264501 (+ 127us) client_negotiation.cc:189] Negotiated authn=SASL
0810 04:41:45.264550 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:41:45.264551 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:41:45.265231 (+ 680us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:41:45.265240 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:41:45.265835 (+ 595us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:41:45.265837 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:41:45.266167 (+ 330us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:41:45.266168 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:41:45.266240 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:41:45.266618 (+ 378us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 04:41:45.266622 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0810 04:41:45.268320 (+ 1698us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.167.126:37819: Server oryx/127.22.167.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":30}
W20260810 04:41:45.268476 6334 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 04:41:45.264168 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.167.67:51633 (local address 127.22.167.126:37819)
0810 04:41:45.264275 (+ 107us) server_negotiation.cc:206] Beginning negotiation
0810 04:41:45.264276 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0810 04:41:45.264283 (+ 7us) server_negotiation.cc:407] Connection header received
0810 04:41:45.264297 (+ 14us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:41:45.264297 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:41:45.264306 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:41:45.264364 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0810 04:41:45.264620 (+ 256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:41:45.265158 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:41:45.265908 (+ 750us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:41:45.266089 (+ 181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:41:45.266152 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:41:45.268408 (+ 2256us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.167.67:51633: BlockingRecv error: recv got EOF from 127.22.167.67:51633 (error 108)
Metrics: {"server-negotiator.queue_time_us":92,"thread_start_us":65,"threads_started":1}
I20260810 04:41:45.270870 5879 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:41:45.263746 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34630 (local address 127.22.167.126:37819)
0810 04:41:45.263874 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0810 04:41:45.263875 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0810 04:41:45.264028 (+ 153us) server_negotiation.cc:407] Connection header received
0810 04:41:45.264242 (+ 214us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:41:45.264243 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:41:45.264253 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:41:45.264315 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0810 04:41:45.264967 (+ 652us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:41:45.265487 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:41:45.266460 (+ 973us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:41:45.266695 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:41:45.266744 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:41:45.267866 (+ 1122us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 04:41:45.267870 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 04:41:45.267870 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 04:41:45.267882 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0810 04:41:45.269516 (+ 1634us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 04:41:45.269870 (+ 354us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 04:41:45.269873 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 04:41:45.269874 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 04:41:45.269972 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 04:41:45.270160 (+ 188us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 04:41:45.270161 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 04:41:45.270161 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 04:41:45.270416 (+ 255us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 04:41:45.270544 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0810 04:41:45.270666 (+ 122us) server_negotiation.cc:299] Negotiation successful
0810 04:41:45.270786 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":109}
Aug 10 04:41:45 dist-test-slave-bqcl krb5kdc[4634](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.167.67@KRBTEST.COM for oryx/127.22.167.126@KRBTEST.COM, Server not found in Kerberos database
Aug 10 04:41:45 dist-test-slave-bqcl krb5kdc[4634](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.167.67@KRBTEST.COM for oryx/127.22.167.126@KRBTEST.COM, Server not found in Kerberos database
I20260810 04:41:45.272632 6330 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:41:45.268768 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.126:37819 (local address 127.22.167.67:46261)
0810 04:41:45.268856 (+ 88us) negotiation.cc:107] Waiting for socket to connect
0810 04:41:45.268864 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0810 04:41:45.268905 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 04:41:45.269000 (+ 95us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 04:41:45.269000 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 04:41:45.269117 (+ 117us) client_negotiation.cc:189] Negotiated authn=SASL
0810 04:41:45.269139 (+ 22us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:41:45.269140 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:41:45.269754 (+ 614us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:41:45.269755 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:41:45.270284 (+ 529us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:41:45.270286 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:41:45.270690 (+ 404us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:41:45.270691 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:41:45.270743 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:41:45.271080 (+ 337us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 04:41:45.271085 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0810 04:41:45.272592 (+ 1507us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.167.126:37819: Server oryx/127.22.167.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":75}
W20260810 04:41:45.272706 6334 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 04:41:45.268819 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.167.67:46261 (local address 127.22.167.126:37819)
0810 04:41:45.268855 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0810 04:41:45.268857 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 04:41:45.268894 (+ 37us) server_negotiation.cc:407] Connection header received
0810 04:41:45.268946 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:41:45.268947 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:41:45.268956 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:41:45.268998 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0810 04:41:45.269197 (+ 199us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:41:45.269678 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:41:45.270351 (+ 673us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:41:45.270614 (+ 263us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:41:45.270673 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:41:45.272662 (+ 1989us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.167.67:46261: BlockingRecv error: recv got EOF from 127.22.167.67:46261 (error 108)
Metrics: {"server-negotiator.queue_time_us":17}
W20260810 04:41:45.272766 6326 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
Aug 10 04:41:46 dist-test-slave-bqcl krb5kdc[4634](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.167.65@KRBTEST.COM for oryx/127.22.167.126@KRBTEST.COM, Server not found in Kerberos database
Aug 10 04:41:46 dist-test-slave-bqcl krb5kdc[4634](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.167.65@KRBTEST.COM for oryx/127.22.167.126@KRBTEST.COM, Server not found in Kerberos database
I20260810 04:41:46.022691 6336 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:41:46.018495 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.126:37819 (local address 127.22.167.65:55169)
0810 04:41:46.018633 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0810 04:41:46.018645 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0810 04:41:46.018711 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 04:41:46.018822 (+ 111us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 04:41:46.018823 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 04:41:46.018976 (+ 153us) client_negotiation.cc:189] Negotiated authn=SASL
0810 04:41:46.019025 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:41:46.019027 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:41:46.019607 (+ 580us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:41:46.019607 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:41:46.020142 (+ 535us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:41:46.020143 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:41:46.020425 (+ 282us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:41:46.020426 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:41:46.020462 (+ 36us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:41:46.020893 (+ 431us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 04:41:46.020898 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0810 04:41:46.022617 (+ 1719us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.167.126:37819: Server oryx/127.22.167.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":65,"threads_started":1}
W20260810 04:41:46.022778 6337 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 04:41:46.018556 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.167.65:55169 (local address 127.22.167.126:37819)
0810 04:41:46.018669 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0810 04:41:46.018671 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 04:41:46.018694 (+ 23us) server_negotiation.cc:407] Connection header received
0810 04:41:46.018752 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:41:46.018752 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:41:46.018761 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:41:46.018813 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0810 04:41:46.019068 (+ 255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:41:46.019557 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:41:46.020199 (+ 642us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:41:46.020378 (+ 179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:41:46.020420 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:41:46.022709 (+ 2289us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.167.65:55169: BlockingRecv error: recv got EOF from 127.22.167.65:55169 (error 108)
Metrics: {"server-negotiator.queue_time_us":93,"thread_start_us":61,"threads_started":1}
Aug 10 04:41:46 dist-test-slave-bqcl krb5kdc[4634](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.167.66@KRBTEST.COM for oryx/127.22.167.126@KRBTEST.COM, Server not found in Kerberos database
Aug 10 04:41:46 dist-test-slave-bqcl krb5kdc[4634](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.167.66@KRBTEST.COM for oryx/127.22.167.126@KRBTEST.COM, Server not found in Kerberos database
I20260810 04:41:46.136752 6339 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:41:46.132469 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.126:37819 (local address 127.22.167.66:49295)
0810 04:41:46.132611 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0810 04:41:46.132622 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0810 04:41:46.132683 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 04:41:46.132810 (+ 127us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 04:41:46.132810 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 04:41:46.132971 (+ 161us) client_negotiation.cc:189] Negotiated authn=SASL
0810 04:41:46.133017 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:41:46.133019 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:41:46.133610 (+ 591us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:41:46.133611 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:41:46.134207 (+ 596us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:41:46.134209 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:41:46.134504 (+ 295us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:41:46.134505 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:41:46.134541 (+ 36us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:41:46.134985 (+ 444us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 04:41:46.134990 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0810 04:41:46.136668 (+ 1678us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.167.126:37819: Server oryx/127.22.167.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":70,"threads_started":1}
W20260810 04:41:46.136837 6337 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 04:41:46.132530 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.167.66:49295 (local address 127.22.167.126:37819)
0810 04:41:46.132589 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0810 04:41:46.132591 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 04:41:46.132672 (+ 81us) server_negotiation.cc:407] Connection header received
0810 04:41:46.132733 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:41:46.132734 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:41:46.132744 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:41:46.132796 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0810 04:41:46.133065 (+ 269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:41:46.133558 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:41:46.134268 (+ 710us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:41:46.134454 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:41:46.134502 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:41:46.136784 (+ 2282us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.167.66:49295: BlockingRecv error: recv got EOF from 127.22.167.66:49295 (error 108)
Metrics: {"server-negotiator.queue_time_us":31}
Aug 10 04:41:46 dist-test-slave-bqcl krb5kdc[4634](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.167.67@KRBTEST.COM for oryx/127.22.167.126@KRBTEST.COM, Server not found in Kerberos database
Aug 10 04:41:46 dist-test-slave-bqcl krb5kdc[4634](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.167.67@KRBTEST.COM for oryx/127.22.167.126@KRBTEST.COM, Server not found in Kerberos database
I20260810 04:41:46.277527 6340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:41:46.273094 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.126:37819 (local address 127.22.167.67:40925)
0810 04:41:46.273221 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0810 04:41:46.273232 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0810 04:41:46.273301 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 04:41:46.273426 (+ 125us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 04:41:46.273427 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 04:41:46.273601 (+ 174us) client_negotiation.cc:189] Negotiated authn=SASL
0810 04:41:46.273647 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:41:46.273649 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:41:46.274350 (+ 701us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:41:46.274350 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:41:46.274883 (+ 533us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:41:46.274885 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:41:46.275262 (+ 377us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:41:46.275263 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:41:46.275320 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:41:46.275764 (+ 444us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 04:41:46.275769 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0810 04:41:46.277464 (+ 1695us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.167.126:37819: Server oryx/127.22.167.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":67,"threads_started":1}
W20260810 04:41:46.277618 6337 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 04:41:46.273157 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.167.67:40925 (local address 127.22.167.126:37819)
0810 04:41:46.273216 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0810 04:41:46.273218 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 04:41:46.273290 (+ 72us) server_negotiation.cc:407] Connection header received
0810 04:41:46.273359 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:41:46.273360 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:41:46.273373 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:41:46.273415 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0810 04:41:46.273719 (+ 304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:41:46.274282 (+ 563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:41:46.274960 (+ 678us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:41:46.275191 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:41:46.275241 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:41:46.277558 (+ 2317us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.167.67:40925: BlockingRecv error: recv got EOF from 127.22.167.67:40925 (error 108)
Metrics: {"server-negotiator.queue_time_us":31}
Aug 10 04:41:47 dist-test-slave-bqcl krb5kdc[4634](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.167.65@KRBTEST.COM for oryx/127.22.167.126@KRBTEST.COM, Server not found in Kerberos database
Aug 10 04:41:47 dist-test-slave-bqcl krb5kdc[4634](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.167.65@KRBTEST.COM for oryx/127.22.167.126@KRBTEST.COM, Server not found in Kerberos database
I20260810 04:41:47.027590 6341 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:41:47.023097 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.126:37819 (local address 127.22.167.65:37859)
0810 04:41:47.023220 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0810 04:41:47.023230 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0810 04:41:47.023280 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 04:41:47.023413 (+ 133us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 04:41:47.023414 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 04:41:47.023566 (+ 152us) client_negotiation.cc:189] Negotiated authn=SASL
0810 04:41:47.023615 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:41:47.023617 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:41:47.024222 (+ 605us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:41:47.024222 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:41:47.024770 (+ 548us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:41:47.024772 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:41:47.025061 (+ 289us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:41:47.025061 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:41:47.025097 (+ 36us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:41:47.025507 (+ 410us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 04:41:47.025511 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0810 04:41:47.027532 (+ 2021us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.167.126:37819: Server oryx/127.22.167.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":104,"thread_start_us":71,"threads_started":1}
W20260810 04:41:47.027705 6343 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 04:41:47.023170 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.167.65:37859 (local address 127.22.167.126:37819)
0810 04:41:47.023310 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0810 04:41:47.023311 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0810 04:41:47.023319 (+ 8us) server_negotiation.cc:407] Connection header received
0810 04:41:47.023339 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:41:47.023339 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:41:47.023349 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:41:47.023403 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0810 04:41:47.023673 (+ 270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:41:47.024161 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:41:47.024827 (+ 666us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:41:47.025014 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:41:47.025058 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:41:47.027626 (+ 2568us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.167.65:37859: BlockingRecv error: recv got EOF from 127.22.167.65:37859 (error 108)
Metrics: {"server-negotiator.queue_time_us":121,"thread_start_us":78,"threads_started":1}
Aug 10 04:41:47 dist-test-slave-bqcl krb5kdc[4634](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.167.66@KRBTEST.COM for oryx/127.22.167.126@KRBTEST.COM, Server not found in Kerberos database
Aug 10 04:41:47 dist-test-slave-bqcl krb5kdc[4634](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.167.66@KRBTEST.COM for oryx/127.22.167.126@KRBTEST.COM, Server not found in Kerberos database
I20260810 04:41:47.141151 6344 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:41:47.137175 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.126:37819 (local address 127.22.167.66:48367)
0810 04:41:47.137304 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0810 04:41:47.137315 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0810 04:41:47.137370 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 04:41:47.137485 (+ 115us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 04:41:47.137485 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 04:41:47.137643 (+ 158us) client_negotiation.cc:189] Negotiated authn=SASL
0810 04:41:47.137691 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:41:47.137693 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:41:47.138291 (+ 598us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:41:47.138292 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:41:47.138829 (+ 537us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:41:47.138831 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:41:47.139096 (+ 265us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:41:47.139096 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:41:47.139132 (+ 36us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:41:47.139522 (+ 390us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 04:41:47.139527 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0810 04:41:47.141085 (+ 1558us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.167.126:37819: Server oryx/127.22.167.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":63,"threads_started":1}
W20260810 04:41:47.141242 6343 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 04:41:47.137237 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.167.66:48367 (local address 127.22.167.126:37819)
0810 04:41:47.137291 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0810 04:41:47.137293 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 04:41:47.137370 (+ 77us) server_negotiation.cc:407] Connection header received
0810 04:41:47.137421 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:41:47.137421 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:41:47.137432 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:41:47.137470 (+ 38us) server_negotiation.cc:226] Negotiated authn=SASL
0810 04:41:47.137748 (+ 278us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:41:47.138243 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:41:47.138881 (+ 638us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:41:47.139052 (+ 171us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:41:47.139093 (+ 41us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:41:47.141183 (+ 2090us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.167.66:48367: BlockingRecv error: recv got EOF from 127.22.167.66:48367 (error 108)
Metrics: {"server-negotiator.queue_time_us":29}
Aug 10 04:41:47 dist-test-slave-bqcl krb5kdc[4634](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.167.67@KRBTEST.COM for oryx/127.22.167.126@KRBTEST.COM, Server not found in Kerberos database
Aug 10 04:41:47 dist-test-slave-bqcl krb5kdc[4634](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.167.67@KRBTEST.COM for oryx/127.22.167.126@KRBTEST.COM, Server not found in Kerberos database
I20260810 04:41:47.282156 6345 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:41:47.277925 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.126:37819 (local address 127.22.167.67:52851)
0810 04:41:47.278053 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0810 04:41:47.278063 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0810 04:41:47.278110 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 04:41:47.278264 (+ 154us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 04:41:47.278265 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 04:41:47.278416 (+ 151us) client_negotiation.cc:189] Negotiated authn=SASL
0810 04:41:47.278466 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:41:47.278468 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:41:47.279055 (+ 587us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:41:47.279055 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:41:47.279577 (+ 522us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:41:47.279578 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:41:47.279846 (+ 268us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:41:47.279847 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:41:47.279883 (+ 36us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:41:47.280274 (+ 391us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 04:41:47.280278 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0810 04:41:47.282056 (+ 1778us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.167.126:37819: Server oryx/127.22.167.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":78,"threads_started":1}
W20260810 04:41:47.282196 6343 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 04:41:47.278034 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.167.67:52851 (local address 127.22.167.126:37819)
0810 04:41:47.278094 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0810 04:41:47.278096 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 04:41:47.278104 (+ 8us) server_negotiation.cc:407] Connection header received
0810 04:41:47.278193 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:41:47.278194 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:41:47.278208 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:41:47.278253 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0810 04:41:47.278515 (+ 262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:41:47.279004 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:41:47.279628 (+ 624us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:41:47.279803 (+ 175us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:41:47.279843 (+ 40us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:41:47.282144 (+ 2301us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.167.67:52851: BlockingRecv error: recv got EOF from 127.22.167.67:52851 (error 108)
Metrics: {"server-negotiator.queue_time_us":37}
Aug 10 04:41:48 dist-test-slave-bqcl krb5kdc[4634](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.167.65@KRBTEST.COM for oryx/127.22.167.126@KRBTEST.COM, Server not found in Kerberos database
Aug 10 04:41:48 dist-test-slave-bqcl krb5kdc[4634](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.167.65@KRBTEST.COM for oryx/127.22.167.126@KRBTEST.COM, Server not found in Kerberos database
I20260810 04:41:48.032371 6347 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:41:48.028011 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.126:37819 (local address 127.22.167.65:46341)
0810 04:41:48.028145 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0810 04:41:48.028155 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0810 04:41:48.028207 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 04:41:48.028363 (+ 156us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 04:41:48.028365 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 04:41:48.028542 (+ 177us) client_negotiation.cc:189] Negotiated authn=SASL
0810 04:41:48.028599 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:41:48.028601 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:41:48.029224 (+ 623us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:41:48.029225 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:41:48.029783 (+ 558us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:41:48.029785 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:41:48.030069 (+ 284us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:41:48.030070 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:41:48.030109 (+ 39us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:41:48.030535 (+ 426us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 04:41:48.030540 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0810 04:41:48.032298 (+ 1758us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.167.126:37819: Server oryx/127.22.167.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":74,"threads_started":1}
W20260810 04:41:48.032455 6351 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 04:41:48.028052 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.167.65:46341 (local address 127.22.167.126:37819)
0810 04:41:48.028191 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0810 04:41:48.028193 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 04:41:48.028201 (+ 8us) server_negotiation.cc:407] Connection header received
0810 04:41:48.028273 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:41:48.028274 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:41:48.028284 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:41:48.028346 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0810 04:41:48.028675 (+ 329us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:41:48.029163 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:41:48.029843 (+ 680us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:41:48.030020 (+ 177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:41:48.030060 (+ 40us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:41:48.032387 (+ 2327us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.167.65:46341: BlockingRecv error: recv got EOF from 127.22.167.65:46341 (error 108)
Metrics: {"server-negotiator.queue_time_us":117,"thread_start_us":80,"threads_started":1}
Aug 10 04:41:48 dist-test-slave-bqcl krb5kdc[4634](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.167.66@KRBTEST.COM for oryx/127.22.167.126@KRBTEST.COM, Server not found in Kerberos database
Aug 10 04:41:48 dist-test-slave-bqcl krb5kdc[4634](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.167.66@KRBTEST.COM for oryx/127.22.167.126@KRBTEST.COM, Server not found in Kerberos database
I20260810 04:41:48.145732 6353 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:41:48.141585 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.126:37819 (local address 127.22.167.66:34049)
0810 04:41:48.141725 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0810 04:41:48.141734 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0810 04:41:48.141791 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 04:41:48.141908 (+ 117us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 04:41:48.141909 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 04:41:48.142069 (+ 160us) client_negotiation.cc:189] Negotiated authn=SASL
0810 04:41:48.142144 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:41:48.142146 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:41:48.142745 (+ 599us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:41:48.142746 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:41:48.143274 (+ 528us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:41:48.143276 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:41:48.143539 (+ 263us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:41:48.143540 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:41:48.143576 (+ 36us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:41:48.143983 (+ 407us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 04:41:48.143988 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0810 04:41:48.145669 (+ 1681us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.167.126:37819: Server oryx/127.22.167.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":73,"threads_started":1}
W20260810 04:41:48.145830 6351 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 04:41:48.141644 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.167.66:34049 (local address 127.22.167.126:37819)
0810 04:41:48.141709 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0810 04:41:48.141711 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 04:41:48.141786 (+ 75us) server_negotiation.cc:407] Connection header received
0810 04:41:48.141846 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:41:48.141846 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:41:48.141857 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:41:48.141898 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0810 04:41:48.142208 (+ 310us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:41:48.142690 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:41:48.143325 (+ 635us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:41:48.143494 (+ 169us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:41:48.143537 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:41:48.145770 (+ 2233us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.167.66:34049: BlockingRecv error: recv got EOF from 127.22.167.66:34049 (error 108)
Metrics: {"server-negotiator.queue_time_us":34}
Aug 10 04:41:48 dist-test-slave-bqcl krb5kdc[4634](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.167.67@KRBTEST.COM for oryx/127.22.167.126@KRBTEST.COM, Server not found in Kerberos database
Aug 10 04:41:48 dist-test-slave-bqcl krb5kdc[4634](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.167.67@KRBTEST.COM for oryx/127.22.167.126@KRBTEST.COM, Server not found in Kerberos database
I20260810 04:41:48.286940 6354 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:41:48.282614 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.126:37819 (local address 127.22.167.67:33343)
0810 04:41:48.282767 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0810 04:41:48.282778 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0810 04:41:48.282835 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 04:41:48.282950 (+ 115us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 04:41:48.282951 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 04:41:48.283110 (+ 159us) client_negotiation.cc:189] Negotiated authn=SASL
0810 04:41:48.283159 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:41:48.283161 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:41:48.283744 (+ 583us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:41:48.283744 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:41:48.284279 (+ 535us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:41:48.284281 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:41:48.284554 (+ 273us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:41:48.284555 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:41:48.284600 (+ 45us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:41:48.285001 (+ 401us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 04:41:48.285005 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0810 04:41:48.286865 (+ 1860us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.167.126:37819: Server oryx/127.22.167.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":95,"threads_started":1}
W20260810 04:41:48.286970 6351 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 04:41:48.282686 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.167.67:33343 (local address 127.22.167.126:37819)
0810 04:41:48.282750 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0810 04:41:48.282752 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 04:41:48.282830 (+ 78us) server_negotiation.cc:407] Connection header received
0810 04:41:48.282889 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:41:48.282890 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:41:48.282900 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:41:48.282940 (+ 40us) server_negotiation.cc:226] Negotiated authn=SASL
0810 04:41:48.283207 (+ 267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:41:48.283688 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:41:48.284330 (+ 642us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:41:48.284496 (+ 166us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:41:48.284538 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:41:48.286922 (+ 2384us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.167.67:33343: BlockingRecv error: recv got EOF from 127.22.167.67:33343 (error 108)
Metrics: {"server-negotiator.queue_time_us":39}
Aug 10 04:41:49 dist-test-slave-bqcl krb5kdc[4634](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.167.65@KRBTEST.COM for oryx/127.22.167.126@KRBTEST.COM, Server not found in Kerberos database
Aug 10 04:41:49 dist-test-slave-bqcl krb5kdc[4634](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.167.65@KRBTEST.COM for oryx/127.22.167.126@KRBTEST.COM, Server not found in Kerberos database
I20260810 04:41:49.036996 6355 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:41:49.032790 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.126:37819 (local address 127.22.167.65:33845)
0810 04:41:49.032923 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0810 04:41:49.032933 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0810 04:41:49.032991 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 04:41:49.033100 (+ 109us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 04:41:49.033101 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 04:41:49.033262 (+ 161us) client_negotiation.cc:189] Negotiated authn=SASL
0810 04:41:49.033314 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:41:49.033315 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:41:49.033924 (+ 609us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:41:49.033925 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:41:49.034502 (+ 577us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:41:49.034504 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:41:49.034793 (+ 289us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:41:49.034794 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:41:49.034830 (+ 36us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:41:49.035292 (+ 462us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 04:41:49.035297 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0810 04:41:49.036938 (+ 1641us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.167.126:37819: Server oryx/127.22.167.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":70,"threads_started":1}
W20260810 04:41:49.037089 6356 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 04:41:49.032833 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.167.65:33845 (local address 127.22.167.126:37819)
0810 04:41:49.032936 (+ 103us) server_negotiation.cc:206] Beginning negotiation
0810 04:41:49.032938 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 04:41:49.032975 (+ 37us) server_negotiation.cc:407] Connection header received
0810 04:41:49.033037 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:41:49.033038 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:41:49.033048 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:41:49.033090 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0810 04:41:49.033361 (+ 271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:41:49.033864 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:41:49.034563 (+ 699us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:41:49.034748 (+ 185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:41:49.034790 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:41:49.037021 (+ 2231us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.167.65:33845: BlockingRecv error: recv got EOF from 127.22.167.65:33845 (error 108)
Metrics: {"server-negotiator.queue_time_us":81,"thread_start_us":51,"threads_started":1}
Aug 10 04:41:49 dist-test-slave-bqcl krb5kdc[4634](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.167.66@KRBTEST.COM for oryx/127.22.167.126@KRBTEST.COM, Server not found in Kerberos database
Aug 10 04:41:49 dist-test-slave-bqcl krb5kdc[4634](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.167.66@KRBTEST.COM for oryx/127.22.167.126@KRBTEST.COM, Server not found in Kerberos database
I20260810 04:41:49.153496 6357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:41:49.146233 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.126:37819 (local address 127.22.167.66:40761)
0810 04:41:49.146397 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0810 04:41:49.146409 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0810 04:41:49.146481 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 04:41:49.146601 (+ 120us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 04:41:49.146601 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 04:41:49.146793 (+ 192us) client_negotiation.cc:189] Negotiated authn=SASL
0810 04:41:49.146840 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:41:49.146842 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:41:49.147496 (+ 654us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:41:49.147498 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:41:49.148029 (+ 531us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:41:49.148031 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:41:49.148362 (+ 331us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:41:49.148363 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:41:49.148441 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:41:49.151810 (+ 3369us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 04:41:49.151815 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0810 04:41:49.153438 (+ 1623us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.167.126:37819: Server oryx/127.22.167.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":88,"threads_started":1}
W20260810 04:41:49.153596 6356 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 04:41:49.146321 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.167.66:40761 (local address 127.22.167.126:37819)
0810 04:41:49.146392 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0810 04:41:49.146395 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 04:41:49.146461 (+ 66us) server_negotiation.cc:407] Connection header received
0810 04:41:49.146534 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:41:49.146535 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:41:49.146548 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:41:49.146595 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0810 04:41:49.146913 (+ 318us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:41:49.147420 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:41:49.148105 (+ 685us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:41:49.148303 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:41:49.148357 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:41:49.153536 (+ 5179us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.167.66:40761: BlockingRecv error: recv got EOF from 127.22.167.66:40761 (error 108)
Metrics: {"server-negotiator.queue_time_us":39}
Aug 10 04:41:49 dist-test-slave-bqcl krb5kdc[4634](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.167.67@KRBTEST.COM for oryx/127.22.167.126@KRBTEST.COM, Server not found in Kerberos database
Aug 10 04:41:49 dist-test-slave-bqcl krb5kdc[4634](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.167.67@KRBTEST.COM for oryx/127.22.167.126@KRBTEST.COM, Server not found in Kerberos database
I20260810 04:41:49.296597 6359 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:41:49.287353 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.126:37819 (local address 127.22.167.67:44021)
0810 04:41:49.287494 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0810 04:41:49.287505 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0810 04:41:49.287572 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 04:41:49.287708 (+ 136us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 04:41:49.287709 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 04:41:49.287885 (+ 176us) client_negotiation.cc:189] Negotiated authn=SASL
0810 04:41:49.287930 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:41:49.287932 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:41:49.288566 (+ 634us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:41:49.288567 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:41:49.289100 (+ 533us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:41:49.289102 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:41:49.289418 (+ 316us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:41:49.289419 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:41:49.289473 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:41:49.294862 (+ 5389us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 04:41:49.294866 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0810 04:41:49.296522 (+ 1656us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.167.126:37819: Server oryx/127.22.167.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":71,"threads_started":1}
W20260810 04:41:49.296689 6356 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 04:41:49.287410 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.167.67:44021 (local address 127.22.167.126:37819)
0810 04:41:49.287458 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0810 04:41:49.287460 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 04:41:49.287561 (+ 101us) server_negotiation.cc:407] Connection header received
0810 04:41:49.287630 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:41:49.287631 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:41:49.287647 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:41:49.287695 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0810 04:41:49.287997 (+ 302us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:41:49.288493 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:41:49.289179 (+ 686us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:41:49.289358 (+ 179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:41:49.289413 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:41:49.296625 (+ 7212us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.167.67:44021: BlockingRecv error: recv got EOF from 127.22.167.67:44021 (error 108)
Metrics: {"server-negotiator.queue_time_us":20}
Aug 10 04:41:50 dist-test-slave-bqcl krb5kdc[4634](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.167.65@KRBTEST.COM for oryx/127.22.167.126@KRBTEST.COM, Server not found in Kerberos database
Aug 10 04:41:50 dist-test-slave-bqcl krb5kdc[4634](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.167.65@KRBTEST.COM for oryx/127.22.167.126@KRBTEST.COM, Server not found in Kerberos database
I20260810 04:41:50.041564 6362 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:41:50.037446 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.126:37819 (local address 127.22.167.65:49905)
0810 04:41:50.037569 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0810 04:41:50.037578 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0810 04:41:50.037628 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 04:41:50.037748 (+ 120us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 04:41:50.037748 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 04:41:50.037910 (+ 162us) client_negotiation.cc:189] Negotiated authn=SASL
0810 04:41:50.037960 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:41:50.037961 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:41:50.038568 (+ 607us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:41:50.038568 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:41:50.039092 (+ 524us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:41:50.039093 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:41:50.039367 (+ 274us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:41:50.039367 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:41:50.039402 (+ 35us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:41:50.039854 (+ 452us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 04:41:50.039858 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0810 04:41:50.041504 (+ 1646us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.167.126:37819: Server oryx/127.22.167.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":103,"thread_start_us":65,"threads_started":1}
W20260810 04:41:50.041674 6363 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 04:41:50.037500 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.167.65:49905 (local address 127.22.167.126:37819)
0810 04:41:50.037624 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0810 04:41:50.037626 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 04:41:50.037635 (+ 9us) server_negotiation.cc:407] Connection header received
0810 04:41:50.037660 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:41:50.037660 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:41:50.037670 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:41:50.037729 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0810 04:41:50.038010 (+ 281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:41:50.038520 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:41:50.039144 (+ 624us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:41:50.039322 (+ 178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:41:50.039364 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:41:50.041601 (+ 2237us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.167.65:49905: BlockingRecv error: recv got EOF from 127.22.167.65:49905 (error 108)
Metrics: {"server-negotiator.queue_time_us":104,"thread_start_us":64,"threads_started":1}
Aug 10 04:41:50 dist-test-slave-bqcl krb5kdc[4634](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.167.66@KRBTEST.COM for oryx/127.22.167.126@KRBTEST.COM, Server not found in Kerberos database
Aug 10 04:41:50 dist-test-slave-bqcl krb5kdc[4634](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.167.66@KRBTEST.COM for oryx/127.22.167.126@KRBTEST.COM, Server not found in Kerberos database
I20260810 04:41:50.158444 6364 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:41:50.153937 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.126:37819 (local address 127.22.167.66:44403)
0810 04:41:50.154102 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0810 04:41:50.154115 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0810 04:41:50.154218 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 04:41:50.154355 (+ 137us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 04:41:50.154356 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 04:41:50.154527 (+ 171us) client_negotiation.cc:189] Negotiated authn=SASL
0810 04:41:50.154574 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:41:50.154576 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:41:50.155223 (+ 647us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:41:50.155224 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:41:50.155812 (+ 588us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:41:50.155815 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:41:50.156140 (+ 325us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:41:50.156141 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:41:50.156207 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:41:50.156629 (+ 422us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 04:41:50.156634 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0810 04:41:50.158384 (+ 1750us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.167.126:37819: Server oryx/127.22.167.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":75,"threads_started":1}
W20260810 04:41:50.158540 6363 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 04:41:50.154065 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.167.66:44403 (local address 127.22.167.126:37819)
0810 04:41:50.154147 (+ 82us) server_negotiation.cc:206] Beginning negotiation
0810 04:41:50.154149 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 04:41:50.154208 (+ 59us) server_negotiation.cc:407] Connection header received
0810 04:41:50.154278 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:41:50.154278 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:41:50.154292 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:41:50.154334 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0810 04:41:50.154644 (+ 310us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:41:50.155149 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:41:50.155887 (+ 738us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:41:50.156080 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:41:50.156135 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:41:50.158475 (+ 2340us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.167.66:44403: BlockingRecv error: recv got EOF from 127.22.167.66:44403 (error 108)
Metrics: {"server-negotiator.queue_time_us":36}
Aug 10 04:41:50 dist-test-slave-bqcl krb5kdc[4634](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.167.67@KRBTEST.COM for oryx/127.22.167.126@KRBTEST.COM, Server not found in Kerberos database
Aug 10 04:41:50 dist-test-slave-bqcl krb5kdc[4634](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.167.67@KRBTEST.COM for oryx/127.22.167.126@KRBTEST.COM, Server not found in Kerberos database
I20260810 04:41:50.301399 6365 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:41:50.297112 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.126:37819 (local address 127.22.167.67:41859)
0810 04:41:50.297262 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0810 04:41:50.297274 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0810 04:41:50.297343 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 04:41:50.297478 (+ 135us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 04:41:50.297479 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 04:41:50.297646 (+ 167us) client_negotiation.cc:189] Negotiated authn=SASL
0810 04:41:50.297692 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:41:50.297694 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:41:50.298378 (+ 684us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:41:50.298379 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:41:50.298901 (+ 522us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:41:50.298903 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:41:50.299238 (+ 335us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:41:50.299239 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:41:50.299295 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:41:50.299747 (+ 452us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 04:41:50.299751 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0810 04:41:50.301320 (+ 1569us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.167.126:37819: Server oryx/127.22.167.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":91,"threads_started":1}
W20260810 04:41:50.301481 6363 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 04:41:50.297153 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.167.67:41859 (local address 127.22.167.126:37819)
0810 04:41:50.297219 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0810 04:41:50.297221 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 04:41:50.297346 (+ 125us) server_negotiation.cc:407] Connection header received
0810 04:41:50.297394 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:41:50.297394 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:41:50.297409 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:41:50.297455 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0810 04:41:50.297768 (+ 313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:41:50.298312 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:41:50.298978 (+ 666us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:41:50.299180 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:41:50.299233 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:41:50.301425 (+ 2192us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.167.67:41859: BlockingRecv error: recv got EOF from 127.22.167.67:41859 (error 108)
Metrics: {"server-negotiator.queue_time_us":42}
Aug 10 04:41:51 dist-test-slave-bqcl krb5kdc[4634](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.167.65@KRBTEST.COM for oryx/127.22.167.126@KRBTEST.COM, Server not found in Kerberos database
Aug 10 04:41:51 dist-test-slave-bqcl krb5kdc[4634](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.167.65@KRBTEST.COM for oryx/127.22.167.126@KRBTEST.COM, Server not found in Kerberos database
I20260810 04:41:51.046401 6366 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:41:51.042009 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.126:37819 (local address 127.22.167.65:46691)
0810 04:41:51.042150 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0810 04:41:51.042162 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0810 04:41:51.042225 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 04:41:51.042352 (+ 127us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 04:41:51.042353 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 04:41:51.042517 (+ 164us) client_negotiation.cc:189] Negotiated authn=SASL
0810 04:41:51.042582 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:41:51.042583 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:41:51.043212 (+ 629us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:41:51.043212 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:41:51.043738 (+ 526us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:41:51.043738 (+ 0us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:41:51.044039 (+ 301us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:41:51.044040 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:41:51.044089 (+ 49us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:41:51.044504 (+ 415us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 04:41:51.044508 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0810 04:41:51.046321 (+ 1813us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.167.126:37819: Server oryx/127.22.167.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":106,"thread_start_us":59,"threads_started":1}
W20260810 04:41:51.046490 6367 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 04:41:51.042065 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.167.65:46691 (local address 127.22.167.126:37819)
0810 04:41:51.042194 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0810 04:41:51.042196 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 04:41:51.042212 (+ 16us) server_negotiation.cc:407] Connection header received
0810 04:41:51.042268 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:41:51.042269 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:41:51.042280 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:41:51.042335 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0810 04:41:51.042635 (+ 300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:41:51.043148 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:41:51.043794 (+ 646us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:41:51.043987 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:41:51.044035 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:41:51.046428 (+ 2393us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.167.65:46691: BlockingRecv error: recv got EOF from 127.22.167.65:46691 (error 108)
Metrics: {"server-negotiator.queue_time_us":109,"thread_start_us":81,"threads_started":1}
Aug 10 04:41:51 dist-test-slave-bqcl krb5kdc[4634](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.167.66@KRBTEST.COM for oryx/127.22.167.126@KRBTEST.COM, Server not found in Kerberos database
Aug 10 04:41:51 dist-test-slave-bqcl krb5kdc[4634](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.167.66@KRBTEST.COM for oryx/127.22.167.126@KRBTEST.COM, Server not found in Kerberos database
I20260810 04:41:51.163126 6368 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:41:51.158884 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.126:37819 (local address 127.22.167.66:33885)
0810 04:41:51.159022 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0810 04:41:51.159035 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0810 04:41:51.159097 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 04:41:51.159215 (+ 118us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 04:41:51.159216 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 04:41:51.159372 (+ 156us) client_negotiation.cc:189] Negotiated authn=SASL
0810 04:41:51.159421 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:41:51.159422 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:41:51.160009 (+ 587us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:41:51.160009 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:41:51.160550 (+ 541us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:41:51.160552 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:41:51.160825 (+ 273us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:41:51.160826 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:41:51.160859 (+ 33us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:41:51.161265 (+ 406us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 04:41:51.161269 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0810 04:41:51.163051 (+ 1782us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.167.126:37819: Server oryx/127.22.167.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":70,"threads_started":1}
W20260810 04:41:51.163208 6367 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 04:41:51.158969 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.167.66:33885 (local address 127.22.167.126:37819)
0810 04:41:51.159028 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0810 04:41:51.159030 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 04:41:51.159081 (+ 51us) server_negotiation.cc:407] Connection header received
0810 04:41:51.159149 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:41:51.159150 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:41:51.159160 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:41:51.159200 (+ 40us) server_negotiation.cc:226] Negotiated authn=SASL
0810 04:41:51.159469 (+ 269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:41:51.159959 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:41:51.160601 (+ 642us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:41:51.160781 (+ 180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:41:51.160822 (+ 41us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:41:51.163145 (+ 2323us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.167.66:33885: BlockingRecv error: recv got EOF from 127.22.167.66:33885 (error 108)
Metrics: {"server-negotiator.queue_time_us":36}
Aug 10 04:41:51 dist-test-slave-bqcl krb5kdc[4634](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.167.67@KRBTEST.COM for oryx/127.22.167.126@KRBTEST.COM, Server not found in Kerberos database
Aug 10 04:41:51 dist-test-slave-bqcl krb5kdc[4634](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.167.67@KRBTEST.COM for oryx/127.22.167.126@KRBTEST.COM, Server not found in Kerberos database
I20260810 04:41:51.306028 6370 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:41:51.301802 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.126:37819 (local address 127.22.167.67:35935)
0810 04:41:51.301924 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0810 04:41:51.301934 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0810 04:41:51.301990 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 04:41:51.302147 (+ 157us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 04:41:51.302148 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 04:41:51.302313 (+ 165us) client_negotiation.cc:189] Negotiated authn=SASL
0810 04:41:51.302376 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:41:51.302378 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:41:51.302978 (+ 600us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:41:51.302979 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:41:51.303545 (+ 566us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:41:51.303548 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:41:51.303874 (+ 326us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:41:51.303874 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:41:51.303925 (+ 51us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:41:51.304325 (+ 400us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 04:41:51.304329 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0810 04:41:51.305970 (+ 1641us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.167.126:37819: Server oryx/127.22.167.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":103,"thread_start_us":59,"threads_started":1}
W20260810 04:41:51.306143 6367 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 04:41:51.301854 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.167.67:35935 (local address 127.22.167.126:37819)
0810 04:41:51.301898 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0810 04:41:51.301900 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 04:41:51.301978 (+ 78us) server_negotiation.cc:407] Connection header received
0810 04:41:51.302039 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:41:51.302039 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:41:51.302051 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:41:51.302088 (+ 37us) server_negotiation.cc:226] Negotiated authn=SASL
0810 04:41:51.302434 (+ 346us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:41:51.302923 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:41:51.303619 (+ 696us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:41:51.303818 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:41:51.303870 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:41:51.306060 (+ 2190us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.167.67:35935: BlockingRecv error: recv got EOF from 127.22.167.67:35935 (error 108)
Metrics: {"server-negotiator.queue_time_us":20}
Aug 10 04:41:52 dist-test-slave-bqcl krb5kdc[4634](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.167.65@KRBTEST.COM for oryx/127.22.167.126@KRBTEST.COM, Server not found in Kerberos database
Aug 10 04:41:52 dist-test-slave-bqcl krb5kdc[4634](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.167.65@KRBTEST.COM for oryx/127.22.167.126@KRBTEST.COM, Server not found in Kerberos database
I20260810 04:41:52.051055 6373 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:41:52.046845 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.126:37819 (local address 127.22.167.65:41647)
0810 04:41:52.046971 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0810 04:41:52.046982 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0810 04:41:52.047033 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 04:41:52.047151 (+ 118us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 04:41:52.047152 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 04:41:52.047309 (+ 157us) client_negotiation.cc:189] Negotiated authn=SASL
0810 04:41:52.047357 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:41:52.047358 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:41:52.047959 (+ 601us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:41:52.047959 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:41:52.048472 (+ 513us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:41:52.048473 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:41:52.048753 (+ 280us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:41:52.048754 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:41:52.048788 (+ 34us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:41:52.049179 (+ 391us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 04:41:52.049183 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0810 04:41:52.050995 (+ 1812us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.167.126:37819: Server oryx/127.22.167.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":70,"threads_started":1}
W20260810 04:41:52.051167 6374 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 04:41:52.046922 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.167.65:41647 (local address 127.22.167.126:37819)
0810 04:41:52.047034 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0810 04:41:52.047036 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 04:41:52.047064 (+ 28us) server_negotiation.cc:407] Connection header received
0810 04:41:52.047083 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:41:52.047084 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:41:52.047094 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:41:52.047141 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0810 04:41:52.047408 (+ 267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:41:52.047908 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:41:52.048523 (+ 615us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:41:52.048708 (+ 185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:41:52.048751 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:41:52.051088 (+ 2337us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.167.65:41647: BlockingRecv error: recv got EOF from 127.22.167.65:41647 (error 108)
Metrics: {"server-negotiator.queue_time_us":91,"thread_start_us":51,"threads_started":1}
Aug 10 04:41:52 dist-test-slave-bqcl krb5kdc[4634](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.167.66@KRBTEST.COM for oryx/127.22.167.126@KRBTEST.COM, Server not found in Kerberos database
Aug 10 04:41:52 dist-test-slave-bqcl krb5kdc[4634](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.167.66@KRBTEST.COM for oryx/127.22.167.126@KRBTEST.COM, Server not found in Kerberos database
I20260810 04:41:52.167886 6375 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:41:52.163551 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.126:37819 (local address 127.22.167.66:43841)
0810 04:41:52.163676 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0810 04:41:52.163686 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0810 04:41:52.163743 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 04:41:52.163858 (+ 115us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 04:41:52.163859 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 04:41:52.164012 (+ 153us) client_negotiation.cc:189] Negotiated authn=SASL
0810 04:41:52.164061 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:41:52.164062 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:41:52.164645 (+ 583us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:41:52.164645 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:41:52.165158 (+ 513us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:41:52.165159 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:41:52.165451 (+ 292us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:41:52.165452 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:41:52.165487 (+ 35us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:41:52.165890 (+ 403us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 04:41:52.165894 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0810 04:41:52.167826 (+ 1932us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.167.126:37819: Server oryx/127.22.167.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":67,"threads_started":1}
W20260810 04:41:52.167980 6374 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 04:41:52.163602 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.167.66:43841 (local address 127.22.167.126:37819)
0810 04:41:52.163667 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0810 04:41:52.163670 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 04:41:52.163738 (+ 68us) server_negotiation.cc:407] Connection header received
0810 04:41:52.163797 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:41:52.163798 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:41:52.163808 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:41:52.163849 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0810 04:41:52.164110 (+ 261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:41:52.164593 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:41:52.165220 (+ 627us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:41:52.165404 (+ 184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:41:52.165448 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:41:52.167920 (+ 2472us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.167.66:43841: BlockingRecv error: recv got EOF from 127.22.167.66:43841 (error 108)
Metrics: {"server-negotiator.queue_time_us":39}
Aug 10 04:41:52 dist-test-slave-bqcl krb5kdc[4634](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.167.67@KRBTEST.COM for oryx/127.22.167.126@KRBTEST.COM, Server not found in Kerberos database
Aug 10 04:41:52 dist-test-slave-bqcl krb5kdc[4634](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.167.67@KRBTEST.COM for oryx/127.22.167.126@KRBTEST.COM, Server not found in Kerberos database
I20260810 04:41:52.310698 6376 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:41:52.306451 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.126:37819 (local address 127.22.167.67:46263)
0810 04:41:52.306581 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0810 04:41:52.306592 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0810 04:41:52.306653 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 04:41:52.306779 (+ 126us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 04:41:52.306779 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 04:41:52.306941 (+ 162us) client_negotiation.cc:189] Negotiated authn=SASL
0810 04:41:52.306991 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:41:52.306993 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:41:52.307631 (+ 638us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:41:52.307631 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:41:52.308139 (+ 508us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:41:52.308140 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:41:52.308421 (+ 281us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:41:52.308421 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:41:52.308458 (+ 37us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:41:52.308858 (+ 400us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 04:41:52.308862 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0810 04:41:52.310637 (+ 1775us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.167.126:37819: Server oryx/127.22.167.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":67,"threads_started":1}
W20260810 04:41:52.310786 6374 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 04:41:52.306533 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.167.67:46263 (local address 127.22.167.126:37819)
0810 04:41:52.306609 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0810 04:41:52.306611 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 04:41:52.306642 (+ 31us) server_negotiation.cc:407] Connection header received
0810 04:41:52.306694 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:41:52.306695 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:41:52.306707 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:41:52.306767 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0810 04:41:52.307060 (+ 293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:41:52.307572 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:41:52.308205 (+ 633us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:41:52.308375 (+ 170us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:41:52.308416 (+ 41us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:41:52.310723 (+ 2307us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.167.67:46263: BlockingRecv error: recv got EOF from 127.22.167.67:46263 (error 108)
Metrics: {"server-negotiator.queue_time_us":53}
Aug 10 04:41:53 dist-test-slave-bqcl krb5kdc[4634](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.167.65@KRBTEST.COM for oryx/127.22.167.126@KRBTEST.COM, Server not found in Kerberos database
Aug 10 04:41:53 dist-test-slave-bqcl krb5kdc[4634](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.167.65@KRBTEST.COM for oryx/127.22.167.126@KRBTEST.COM, Server not found in Kerberos database
I20260810 04:41:53.055979 6378 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:41:53.051525 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.126:37819 (local address 127.22.167.65:55679)
0810 04:41:53.051660 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0810 04:41:53.051673 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0810 04:41:53.051723 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 04:41:53.051887 (+ 164us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 04:41:53.051888 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 04:41:53.052051 (+ 163us) client_negotiation.cc:189] Negotiated authn=SASL
0810 04:41:53.052101 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:41:53.052103 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:41:53.052880 (+ 777us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:41:53.052881 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:41:53.053421 (+ 540us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:41:53.053422 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:41:53.053721 (+ 299us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:41:53.053722 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:41:53.053758 (+ 36us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:41:53.054205 (+ 447us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 04:41:53.054209 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0810 04:41:53.055916 (+ 1707us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.167.126:37819: Server oryx/127.22.167.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":65,"threads_started":1}
W20260810 04:41:53.056087 6381 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 04:41:53.051655 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.167.65:55679 (local address 127.22.167.126:37819)
0810 04:41:53.051781 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0810 04:41:53.051782 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0810 04:41:53.051790 (+ 8us) server_negotiation.cc:407] Connection header received
0810 04:41:53.051809 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:41:53.051810 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:41:53.051821 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:41:53.051876 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0810 04:41:53.052153 (+ 277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:41:53.052802 (+ 649us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:41:53.053481 (+ 679us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:41:53.053671 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:41:53.053723 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:41:53.056016 (+ 2293us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.167.65:55679: BlockingRecv error: recv got EOF from 127.22.167.65:55679 (error 108)
Metrics: {"server-negotiator.queue_time_us":108,"thread_start_us":66,"threads_started":1}
Aug 10 04:41:53 dist-test-slave-bqcl krb5kdc[4634](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.167.66@KRBTEST.COM for oryx/127.22.167.126@KRBTEST.COM, Server not found in Kerberos database
Aug 10 04:41:53 dist-test-slave-bqcl krb5kdc[4634](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.167.66@KRBTEST.COM for oryx/127.22.167.126@KRBTEST.COM, Server not found in Kerberos database
I20260810 04:41:53.173146 6382 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:41:53.168353 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.126:37819 (local address 127.22.167.66:39915)
0810 04:41:53.168523 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0810 04:41:53.168535 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0810 04:41:53.168620 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 04:41:53.168765 (+ 145us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 04:41:53.168766 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 04:41:53.168989 (+ 223us) client_negotiation.cc:189] Negotiated authn=SASL
0810 04:41:53.169056 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:41:53.169059 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:41:53.169945 (+ 886us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:41:53.169946 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:41:53.170516 (+ 570us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:41:53.170519 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:41:53.170896 (+ 377us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:41:53.170897 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:41:53.170955 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:41:53.171368 (+ 413us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 04:41:53.171373 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0810 04:41:53.173085 (+ 1712us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.167.126:37819: Server oryx/127.22.167.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":151,"thread_start_us":89,"threads_started":1}
W20260810 04:41:53.173266 6381 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 04:41:53.168448 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.167.66:39915 (local address 127.22.167.126:37819)
0810 04:41:53.168515 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0810 04:41:53.168525 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 04:41:53.168599 (+ 74us) server_negotiation.cc:407] Connection header received
0810 04:41:53.168675 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:41:53.168676 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:41:53.168690 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:41:53.168737 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0810 04:41:53.169195 (+ 458us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:41:53.169867 (+ 672us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:41:53.170621 (+ 754us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:41:53.170830 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:41:53.170887 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:41:53.173195 (+ 2308us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.167.66:39915: BlockingRecv error: recv got EOF from 127.22.167.66:39915 (error 108)
Metrics: {"server-negotiator.queue_time_us":38}
Aug 10 04:41:53 dist-test-slave-bqcl krb5kdc[4634](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.167.67@KRBTEST.COM for oryx/127.22.167.126@KRBTEST.COM, Server not found in Kerberos database
Aug 10 04:41:53 dist-test-slave-bqcl krb5kdc[4634](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.167.67@KRBTEST.COM for oryx/127.22.167.126@KRBTEST.COM, Server not found in Kerberos database
I20260810 04:41:53.315415 6384 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:41:53.311126 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.126:37819 (local address 127.22.167.67:57231)
0810 04:41:53.311279 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0810 04:41:53.311290 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0810 04:41:53.311357 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 04:41:53.311493 (+ 136us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 04:41:53.311494 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 04:41:53.311670 (+ 176us) client_negotiation.cc:189] Negotiated authn=SASL
0810 04:41:53.311718 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:41:53.311720 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:41:53.312363 (+ 643us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:41:53.312364 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:41:53.312888 (+ 524us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:41:53.312890 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:41:53.313219 (+ 329us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:41:53.313220 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:41:53.313272 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:41:53.313681 (+ 409us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 04:41:53.313685 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0810 04:41:53.315354 (+ 1669us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.167.126:37819: Server oryx/127.22.167.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":90,"threads_started":1}
W20260810 04:41:53.315510 6381 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 04:41:53.311189 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.167.67:57231 (local address 127.22.167.126:37819)
0810 04:41:53.311244 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0810 04:41:53.311247 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 04:41:53.311350 (+ 103us) server_negotiation.cc:407] Connection header received
0810 04:41:53.311419 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:41:53.311420 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:41:53.311434 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:41:53.311482 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0810 04:41:53.311791 (+ 309us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:41:53.312289 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:41:53.312968 (+ 679us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:41:53.313155 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:41:53.313212 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:41:53.315448 (+ 2236us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.167.67:57231: BlockingRecv error: recv got EOF from 127.22.167.67:57231 (error 108)
Metrics: {"server-negotiator.queue_time_us":28}
Aug 10 04:41:54 dist-test-slave-bqcl krb5kdc[4634](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.167.65@KRBTEST.COM for oryx/127.22.167.126@KRBTEST.COM, Server not found in Kerberos database
Aug 10 04:41:54 dist-test-slave-bqcl krb5kdc[4634](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.167.65@KRBTEST.COM for oryx/127.22.167.126@KRBTEST.COM, Server not found in Kerberos database
I20260810 04:41:54.061589 6385 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:41:54.056448 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.126:37819 (local address 127.22.167.65:33673)
0810 04:41:54.056625 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0810 04:41:54.056636 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0810 04:41:54.056703 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 04:41:54.056833 (+ 130us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 04:41:54.056834 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 04:41:54.057040 (+ 206us) client_negotiation.cc:189] Negotiated authn=SASL
0810 04:41:54.057096 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:41:54.057098 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:41:54.058036 (+ 938us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:41:54.058037 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:41:54.058778 (+ 741us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:41:54.058781 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:41:54.059133 (+ 352us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:41:54.059134 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:41:54.059173 (+ 39us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:41:54.059624 (+ 451us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 04:41:54.059627 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0810 04:41:54.061525 (+ 1898us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.167.126:37819: Server oryx/127.22.167.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":157,"thread_start_us":116,"threads_started":1}
W20260810 04:41:54.061687 6386 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 04:41:54.056521 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.167.65:33673 (local address 127.22.167.126:37819)
0810 04:41:54.056651 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0810 04:41:54.056654 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 04:41:54.056701 (+ 47us) server_negotiation.cc:407] Connection header received
0810 04:41:54.056746 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:41:54.056747 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:41:54.056758 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:41:54.056832 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0810 04:41:54.057187 (+ 355us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:41:54.057932 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:41:54.058850 (+ 918us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:41:54.059068 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:41:54.059130 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:41:54.061619 (+ 2489us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.167.65:33673: BlockingRecv error: recv got EOF from 127.22.167.65:33673 (error 108)
Metrics: {"server-negotiator.queue_time_us":108,"thread_start_us":69,"threads_started":1}
Aug 10 04:41:54 dist-test-slave-bqcl krb5kdc[4634](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.167.66@KRBTEST.COM for oryx/127.22.167.126@KRBTEST.COM, Server not found in Kerberos database
Aug 10 04:41:54 dist-test-slave-bqcl krb5kdc[4634](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.167.66@KRBTEST.COM for oryx/127.22.167.126@KRBTEST.COM, Server not found in Kerberos database
I20260810 04:41:54.180642 6387 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:41:54.173552 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.126:37819 (local address 127.22.167.66:58281)
0810 04:41:54.173666 (+ 114us) negotiation.cc:107] Waiting for socket to connect
0810 04:41:54.173675 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0810 04:41:54.173729 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 04:41:54.173842 (+ 113us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 04:41:54.173843 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 04:41:54.174097 (+ 254us) client_negotiation.cc:189] Negotiated authn=SASL
0810 04:41:54.174177 (+ 80us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:41:54.174180 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:41:54.174924 (+ 744us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:41:54.174925 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:41:54.175538 (+ 613us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:41:54.175542 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:41:54.175874 (+ 332us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:41:54.175875 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:41:54.175937 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:41:54.178852 (+ 2915us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 04:41:54.178857 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0810 04:41:54.180562 (+ 1705us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.167.126:37819: Server oryx/127.22.167.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":97,"thread_start_us":58,"threads_started":1}
W20260810 04:41:54.180732 6386 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 04:41:54.173610 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.167.66:58281 (local address 127.22.167.126:37819)
0810 04:41:54.173669 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0810 04:41:54.173671 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 04:41:54.173722 (+ 51us) server_negotiation.cc:407] Connection header received
0810 04:41:54.173771 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:41:54.173772 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:41:54.173782 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:41:54.173833 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0810 04:41:54.174232 (+ 399us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:41:54.174852 (+ 620us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:41:54.175601 (+ 749us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:41:54.175792 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:41:54.175840 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:41:54.180679 (+ 4839us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.167.66:58281: BlockingRecv error: recv got EOF from 127.22.167.66:58281 (error 108)
Metrics: {"server-negotiator.queue_time_us":26}
Aug 10 04:41:54 dist-test-slave-bqcl krb5kdc[4634](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.167.67@KRBTEST.COM for oryx/127.22.167.126@KRBTEST.COM, Server not found in Kerberos database
Aug 10 04:41:54 dist-test-slave-bqcl krb5kdc[4634](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.167.67@KRBTEST.COM for oryx/127.22.167.126@KRBTEST.COM, Server not found in Kerberos database
I20260810 04:41:54.320326 6390 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:41:54.315848 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.126:37819 (local address 127.22.167.67:52417)
0810 04:41:54.316020 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0810 04:41:54.316032 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0810 04:41:54.316096 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 04:41:54.316228 (+ 132us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 04:41:54.316228 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 04:41:54.316400 (+ 172us) client_negotiation.cc:189] Negotiated authn=SASL
0810 04:41:54.316448 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:41:54.316449 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:41:54.317127 (+ 678us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:41:54.317128 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:41:54.317651 (+ 523us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:41:54.317653 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:41:54.318003 (+ 350us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:41:54.318004 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:41:54.318064 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:41:54.318521 (+ 457us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 04:41:54.318526 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0810 04:41:54.320250 (+ 1724us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.167.126:37819: Server oryx/127.22.167.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":155,"thread_start_us":107,"threads_started":1}
W20260810 04:41:54.320400 6386 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 04:41:54.315924 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.167.67:52417 (local address 127.22.167.126:37819)
0810 04:41:54.315983 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0810 04:41:54.315985 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 04:41:54.316090 (+ 105us) server_negotiation.cc:407] Connection header received
0810 04:41:54.316147 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:41:54.316148 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:41:54.316168 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:41:54.316214 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0810 04:41:54.316521 (+ 307us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:41:54.317038 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:41:54.317753 (+ 715us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:41:54.317943 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:41:54.317999 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:41:54.320348 (+ 2349us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.167.67:52417: BlockingRecv error: recv got EOF from 127.22.167.67:52417 (error 108)
Metrics: {"server-negotiator.queue_time_us":30}
Aug 10 04:41:55 dist-test-slave-bqcl krb5kdc[4634](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.167.65@KRBTEST.COM for oryx/127.22.167.126@KRBTEST.COM, Server not found in Kerberos database
Aug 10 04:41:55 dist-test-slave-bqcl krb5kdc[4634](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.167.65@KRBTEST.COM for oryx/127.22.167.126@KRBTEST.COM, Server not found in Kerberos database
I20260810 04:41:55.066341 6391 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:41:55.062047 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.126:37819 (local address 127.22.167.65:59783)
0810 04:41:55.062188 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0810 04:41:55.062198 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0810 04:41:55.062250 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 04:41:55.062384 (+ 134us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 04:41:55.062384 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 04:41:55.062549 (+ 165us) client_negotiation.cc:189] Negotiated authn=SASL
0810 04:41:55.062600 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:41:55.062601 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:41:55.063367 (+ 766us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:41:55.063368 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:41:55.063893 (+ 525us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:41:55.063894 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:41:55.064183 (+ 289us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:41:55.064184 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:41:55.064219 (+ 35us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:41:55.064612 (+ 393us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 04:41:55.064616 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0810 04:41:55.066280 (+ 1664us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.167.126:37819: Server oryx/127.22.167.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":90,"threads_started":1}
W20260810 04:41:55.066447 6392 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 04:41:55.062113 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.167.65:59783 (local address 127.22.167.126:37819)
0810 04:41:55.062277 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0810 04:41:55.062279 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 04:41:55.062286 (+ 7us) server_negotiation.cc:407] Connection header received
0810 04:41:55.062306 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:41:55.062307 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:41:55.062317 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:41:55.062373 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0810 04:41:55.062652 (+ 279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:41:55.063315 (+ 663us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:41:55.063949 (+ 634us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:41:55.064131 (+ 182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:41:55.064173 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:41:55.066369 (+ 2196us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.167.65:59783: BlockingRecv error: recv got EOF from 127.22.167.65:59783 (error 108)
Metrics: {"server-negotiator.queue_time_us":144,"thread_start_us":101,"threads_started":1}
Aug 10 04:41:55 dist-test-slave-bqcl krb5kdc[4634](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.167.66@KRBTEST.COM for oryx/127.22.167.126@KRBTEST.COM, Server not found in Kerberos database
Aug 10 04:41:55 dist-test-slave-bqcl krb5kdc[4634](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.167.66@KRBTEST.COM for oryx/127.22.167.126@KRBTEST.COM, Server not found in Kerberos database
I20260810 04:41:55.185380 6393 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:41:55.181097 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.126:37819 (local address 127.22.167.66:50397)
0810 04:41:55.181274 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0810 04:41:55.181284 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0810 04:41:55.181337 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 04:41:55.181448 (+ 111us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 04:41:55.181449 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 04:41:55.181613 (+ 164us) client_negotiation.cc:189] Negotiated authn=SASL
0810 04:41:55.181661 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:41:55.181663 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:41:55.182303 (+ 640us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:41:55.182304 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:41:55.182836 (+ 532us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:41:55.182838 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:41:55.183100 (+ 262us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:41:55.183101 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:41:55.183134 (+ 33us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:41:55.183581 (+ 447us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 04:41:55.183585 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0810 04:41:55.185286 (+ 1701us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.167.126:37819: Server oryx/127.22.167.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":158,"thread_start_us":93,"threads_started":1}
W20260810 04:41:55.185438 6392 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 04:41:55.181174 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.167.66:50397 (local address 127.22.167.126:37819)
0810 04:41:55.181227 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0810 04:41:55.181228 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0810 04:41:55.181333 (+ 105us) server_negotiation.cc:407] Connection header received
0810 04:41:55.181379 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:41:55.181380 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:41:55.181391 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:41:55.181436 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0810 04:41:55.181709 (+ 273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:41:55.182240 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:41:55.182885 (+ 645us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:41:55.183056 (+ 171us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:41:55.183099 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:41:55.185387 (+ 2288us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.167.66:50397: BlockingRecv error: recv got EOF from 127.22.167.66:50397 (error 108)
Metrics: {"server-negotiator.queue_time_us":29}
Aug 10 04:41:55 dist-test-slave-bqcl krb5kdc[4634](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.167.67@KRBTEST.COM for oryx/127.22.167.126@KRBTEST.COM, Server not found in Kerberos database
Aug 10 04:41:55 dist-test-slave-bqcl krb5kdc[4634](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.167.67@KRBTEST.COM for oryx/127.22.167.126@KRBTEST.COM, Server not found in Kerberos database
I20260810 04:41:55.324774 6394 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:41:55.320753 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.126:37819 (local address 127.22.167.67:44033)
0810 04:41:55.320895 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0810 04:41:55.320905 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0810 04:41:55.320961 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 04:41:55.321067 (+ 106us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 04:41:55.321067 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 04:41:55.321232 (+ 165us) client_negotiation.cc:189] Negotiated authn=SASL
0810 04:41:55.321281 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:41:55.321283 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:41:55.321874 (+ 591us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:41:55.321875 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:41:55.322412 (+ 537us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:41:55.322414 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:41:55.322688 (+ 274us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:41:55.322688 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:41:55.322723 (+ 35us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:41:55.323120 (+ 397us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 04:41:55.323124 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0810 04:41:55.324715 (+ 1591us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.167.126:37819: Server oryx/127.22.167.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":84,"threads_started":1}
W20260810 04:41:55.324865 6392 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 04:41:55.320863 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.167.67:44033 (local address 127.22.167.126:37819)
0810 04:41:55.320907 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0810 04:41:55.320909 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 04:41:55.320955 (+ 46us) server_negotiation.cc:407] Connection header received
0810 04:41:55.320998 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:41:55.320999 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:41:55.321011 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:41:55.321056 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0810 04:41:55.321331 (+ 275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:41:55.321823 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:41:55.322468 (+ 645us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:41:55.322640 (+ 172us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:41:55.322679 (+ 39us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:41:55.324800 (+ 2121us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.167.67:44033: BlockingRecv error: recv got EOF from 127.22.167.67:44033 (error 108)
Metrics: {"server-negotiator.queue_time_us":20}
Aug 10 04:41:56 dist-test-slave-bqcl krb5kdc[4634](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.167.65@KRBTEST.COM for oryx/127.22.167.126@KRBTEST.COM, Server not found in Kerberos database
Aug 10 04:41:56 dist-test-slave-bqcl krb5kdc[4634](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.167.65@KRBTEST.COM for oryx/127.22.167.126@KRBTEST.COM, Server not found in Kerberos database
I20260810 04:41:56.071290 6397 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:41:56.066793 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.126:37819 (local address 127.22.167.65:48331)
0810 04:41:56.066969 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0810 04:41:56.066983 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0810 04:41:56.067046 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 04:41:56.067181 (+ 135us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 04:41:56.067182 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 04:41:56.067377 (+ 195us) client_negotiation.cc:189] Negotiated authn=SASL
0810 04:41:56.067425 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:41:56.067427 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:41:56.068092 (+ 665us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:41:56.068093 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:41:56.068642 (+ 549us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:41:56.068644 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:41:56.068995 (+ 351us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:41:56.068997 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:41:56.069053 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:41:56.069466 (+ 413us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 04:41:56.069470 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0810 04:41:56.071197 (+ 1727us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.167.126:37819: Server oryx/127.22.167.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":153,"thread_start_us":79,"threads_started":1}
W20260810 04:41:56.071378 6400 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 04:41:56.066860 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.167.65:48331 (local address 127.22.167.126:37819)
0810 04:41:56.066988 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0810 04:41:56.066990 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 04:41:56.067028 (+ 38us) server_negotiation.cc:407] Connection header received
0810 04:41:56.067097 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:41:56.067098 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:41:56.067108 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:41:56.067170 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0810 04:41:56.067505 (+ 335us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:41:56.068022 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:41:56.068720 (+ 698us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:41:56.068935 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:41:56.068989 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:41:56.071315 (+ 2326us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.167.65:48331: BlockingRecv error: recv got EOF from 127.22.167.65:48331 (error 108)
Metrics: {"server-negotiator.queue_time_us":108,"thread_start_us":71,"threads_started":1}
Aug 10 04:41:56 dist-test-slave-bqcl krb5kdc[4634](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.167.66@KRBTEST.COM for oryx/127.22.167.126@KRBTEST.COM, Server not found in Kerberos database
Aug 10 04:41:56 dist-test-slave-bqcl krb5kdc[4634](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.167.66@KRBTEST.COM for oryx/127.22.167.126@KRBTEST.COM, Server not found in Kerberos database
I20260810 04:41:56.190047 6402 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:41:56.185842 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.126:37819 (local address 127.22.167.66:45571)
0810 04:41:56.185964 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0810 04:41:56.185975 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0810 04:41:56.186030 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 04:41:56.186154 (+ 124us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 04:41:56.186155 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 04:41:56.186330 (+ 175us) client_negotiation.cc:189] Negotiated authn=SASL
0810 04:41:56.186378 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:41:56.186379 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:41:56.186971 (+ 592us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:41:56.186971 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:41:56.187493 (+ 522us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:41:56.187495 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:41:56.187768 (+ 273us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:41:56.187768 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:41:56.187802 (+ 34us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:41:56.188254 (+ 452us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 04:41:56.188259 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0810 04:41:56.189975 (+ 1716us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.167.126:37819: Server oryx/127.22.167.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":102,"thread_start_us":72,"threads_started":1}
W20260810 04:41:56.190145 6400 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 04:41:56.185906 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.167.66:45571 (local address 127.22.167.126:37819)
0810 04:41:56.185963 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0810 04:41:56.185965 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 04:41:56.186023 (+ 58us) server_negotiation.cc:407] Connection header received
0810 04:41:56.186083 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:41:56.186084 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:41:56.186095 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:41:56.186150 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0810 04:41:56.186426 (+ 276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:41:56.186920 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:41:56.187546 (+ 626us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:41:56.187723 (+ 177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:41:56.187764 (+ 41us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:41:56.190057 (+ 2293us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.167.66:45571: BlockingRecv error: recv got EOF from 127.22.167.66:45571 (error 108)
Metrics: {"server-negotiator.queue_time_us":33}
Aug 10 04:41:56 dist-test-slave-bqcl krb5kdc[4634](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.167.67@KRBTEST.COM for oryx/127.22.167.126@KRBTEST.COM, Server not found in Kerberos database
Aug 10 04:41:56 dist-test-slave-bqcl krb5kdc[4634](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.167.67@KRBTEST.COM for oryx/127.22.167.126@KRBTEST.COM, Server not found in Kerberos database
I20260810 04:41:56.329360 6403 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:41:56.325212 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.126:37819 (local address 127.22.167.67:47127)
0810 04:41:56.325371 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0810 04:41:56.325382 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0810 04:41:56.325438 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 04:41:56.325553 (+ 115us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 04:41:56.325554 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 04:41:56.325727 (+ 173us) client_negotiation.cc:189] Negotiated authn=SASL
0810 04:41:56.325775 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:41:56.325777 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:41:56.326405 (+ 628us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:41:56.326405 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:41:56.326957 (+ 552us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:41:56.326959 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:41:56.327251 (+ 292us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:41:56.327252 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:41:56.327288 (+ 36us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:41:56.327707 (+ 419us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 04:41:56.327712 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0810 04:41:56.329287 (+ 1575us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.167.126:37819: Server oryx/127.22.167.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":87,"threads_started":1}
W20260810 04:41:56.329424 6400 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 04:41:56.325282 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.167.67:47127 (local address 127.22.167.126:37819)
0810 04:41:56.325341 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0810 04:41:56.325343 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 04:41:56.325433 (+ 90us) server_negotiation.cc:407] Connection header received
0810 04:41:56.325476 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:41:56.325477 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:41:56.325490 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:41:56.325538 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0810 04:41:56.325825 (+ 287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:41:56.326353 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:41:56.327013 (+ 660us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:41:56.327201 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:41:56.327249 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:41:56.329373 (+ 2124us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.167.67:47127: BlockingRecv error: recv got EOF from 127.22.167.67:47127 (error 108)
Metrics: {"server-negotiator.queue_time_us":33}
Aug 10 04:41:57 dist-test-slave-bqcl krb5kdc[4634](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.167.65@KRBTEST.COM for oryx/127.22.167.126@KRBTEST.COM, Server not found in Kerberos database
Aug 10 04:41:57 dist-test-slave-bqcl krb5kdc[4634](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.167.65@KRBTEST.COM for oryx/127.22.167.126@KRBTEST.COM, Server not found in Kerberos database
I20260810 04:41:57.075968 6404 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:41:57.071712 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.126:37819 (local address 127.22.167.65:50089)
0810 04:41:57.071847 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0810 04:41:57.071856 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0810 04:41:57.071907 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 04:41:57.072024 (+ 117us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 04:41:57.072024 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 04:41:57.072206 (+ 182us) client_negotiation.cc:189] Negotiated authn=SASL
0810 04:41:57.072255 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:41:57.072257 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:41:57.072858 (+ 601us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:41:57.072858 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:41:57.073439 (+ 581us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:41:57.073441 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:41:57.073750 (+ 309us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:41:57.073750 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:41:57.073787 (+ 37us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:41:57.074281 (+ 494us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 04:41:57.074286 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0810 04:41:57.075910 (+ 1624us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.167.126:37819: Server oryx/127.22.167.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":71,"threads_started":1}
W20260810 04:41:57.076077 6406 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 04:41:57.071789 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.167.65:50089 (local address 127.22.167.126:37819)
0810 04:41:57.071892 (+ 103us) server_negotiation.cc:206] Beginning negotiation
0810 04:41:57.071894 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 04:41:57.071902 (+ 8us) server_negotiation.cc:407] Connection header received
0810 04:41:57.071949 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:41:57.071950 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:41:57.071960 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:41:57.072014 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0810 04:41:57.072305 (+ 291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:41:57.072807 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:41:57.073498 (+ 691us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:41:57.073698 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:41:57.073745 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:41:57.076008 (+ 2263us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.167.65:50089: BlockingRecv error: recv got EOF from 127.22.167.65:50089 (error 108)
Metrics: {"server-negotiator.queue_time_us":82,"thread_start_us":52,"threads_started":1}
Aug 10 04:41:57 dist-test-slave-bqcl krb5kdc[4634](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.167.66@KRBTEST.COM for oryx/127.22.167.126@KRBTEST.COM, Server not found in Kerberos database
Aug 10 04:41:57 dist-test-slave-bqcl krb5kdc[4634](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.167.66@KRBTEST.COM for oryx/127.22.167.126@KRBTEST.COM, Server not found in Kerberos database
I20260810 04:41:57.194675 6407 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:41:57.190486 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.126:37819 (local address 127.22.167.66:34207)
0810 04:41:57.190634 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0810 04:41:57.190643 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0810 04:41:57.190699 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 04:41:57.190824 (+ 125us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 04:41:57.190824 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 04:41:57.190992 (+ 168us) client_negotiation.cc:189] Negotiated authn=SASL
0810 04:41:57.191039 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:41:57.191040 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:41:57.191650 (+ 610us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:41:57.191651 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:41:57.192180 (+ 529us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:41:57.192181 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:41:57.192478 (+ 297us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:41:57.192479 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:41:57.192514 (+ 35us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:41:57.192923 (+ 409us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 04:41:57.192927 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0810 04:41:57.194607 (+ 1680us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.167.126:37819: Server oryx/127.22.167.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":71,"threads_started":1}
W20260810 04:41:57.194779 6406 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 04:41:57.190548 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.167.66:34207 (local address 127.22.167.126:37819)
0810 04:41:57.190605 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0810 04:41:57.190607 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 04:41:57.190693 (+ 86us) server_negotiation.cc:407] Connection header received
0810 04:41:57.190756 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:41:57.190757 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:41:57.190770 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:41:57.190811 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0810 04:41:57.191086 (+ 275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:41:57.191583 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:41:57.192245 (+ 662us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:41:57.192428 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:41:57.192472 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:41:57.194717 (+ 2245us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.167.66:34207: BlockingRecv error: recv got EOF from 127.22.167.66:34207 (error 108)
Metrics: {"server-negotiator.queue_time_us":30}
Aug 10 04:41:57 dist-test-slave-bqcl krb5kdc[4634](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.167.67@KRBTEST.COM for oryx/127.22.167.126@KRBTEST.COM, Server not found in Kerberos database
Aug 10 04:41:57 dist-test-slave-bqcl krb5kdc[4634](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.167.67@KRBTEST.COM for oryx/127.22.167.126@KRBTEST.COM, Server not found in Kerberos database
I20260810 04:41:57.334111 6409 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:41:57.329766 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.126:37819 (local address 127.22.167.67:47643)
0810 04:41:57.329902 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0810 04:41:57.329915 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0810 04:41:57.329990 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 04:41:57.330118 (+ 128us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 04:41:57.330137 (+ 19us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 04:41:57.330322 (+ 185us) client_negotiation.cc:189] Negotiated authn=SASL
0810 04:41:57.330370 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:41:57.330372 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:41:57.331002 (+ 630us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:41:57.331003 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:41:57.331528 (+ 525us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:41:57.331529 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:41:57.331852 (+ 323us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:41:57.331853 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:41:57.331907 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:41:57.332357 (+ 450us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 04:41:57.332361 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0810 04:41:57.334032 (+ 1671us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.167.126:37819: Server oryx/127.22.167.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":72,"threads_started":1}
W20260810 04:41:57.334226 6406 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 04:41:57.329841 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.167.67:47643 (local address 127.22.167.126:37819)
0810 04:41:57.329906 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0810 04:41:57.329909 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 04:41:57.329981 (+ 72us) server_negotiation.cc:407] Connection header received
0810 04:41:57.330040 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:41:57.330041 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:41:57.330056 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:41:57.330103 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0810 04:41:57.330445 (+ 342us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:41:57.330931 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:41:57.331604 (+ 673us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:41:57.331792 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:41:57.331847 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:41:57.334149 (+ 2302us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.167.67:47643: BlockingRecv error: recv got EOF from 127.22.167.67:47643 (error 108)
Metrics: {"server-negotiator.queue_time_us":37}
Aug 10 04:41:58 dist-test-slave-bqcl krb5kdc[4634](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.167.65@KRBTEST.COM for oryx/127.22.167.126@KRBTEST.COM, Server not found in Kerberos database
Aug 10 04:41:58 dist-test-slave-bqcl krb5kdc[4634](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.167.65@KRBTEST.COM for oryx/127.22.167.126@KRBTEST.COM, Server not found in Kerberos database
I20260810 04:41:58.080699 6411 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:41:58.076427 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.126:37819 (local address 127.22.167.65:43543)
0810 04:41:58.076569 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0810 04:41:58.076581 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0810 04:41:58.076644 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 04:41:58.076775 (+ 131us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 04:41:58.076776 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 04:41:58.076943 (+ 167us) client_negotiation.cc:189] Negotiated authn=SASL
0810 04:41:58.076992 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:41:58.076993 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:41:58.077586 (+ 593us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:41:58.077586 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:41:58.078151 (+ 565us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:41:58.078152 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:41:58.078445 (+ 293us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:41:58.078445 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:41:58.078488 (+ 43us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:41:58.078942 (+ 454us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 04:41:58.078946 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0810 04:41:58.080625 (+ 1679us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.167.126:37819: Server oryx/127.22.167.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":71,"threads_started":1}
W20260810 04:41:58.080791 6413 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 04:41:58.076495 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.167.65:43543 (local address 127.22.167.126:37819)
0810 04:41:58.076592 (+ 97us) server_negotiation.cc:206] Beginning negotiation
0810 04:41:58.076594 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 04:41:58.076627 (+ 33us) server_negotiation.cc:407] Connection header received
0810 04:41:58.076691 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:41:58.076692 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:41:58.076708 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:41:58.076763 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0810 04:41:58.077039 (+ 276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:41:58.077534 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:41:58.078217 (+ 683us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:41:58.078397 (+ 180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:41:58.078440 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:41:58.080718 (+ 2278us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.167.65:43543: BlockingRecv error: recv got EOF from 127.22.167.65:43543 (error 108)
Metrics: {"server-negotiator.queue_time_us":78,"thread_start_us":50,"threads_started":1}
Aug 10 04:41:58 dist-test-slave-bqcl krb5kdc[4634](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.167.66@KRBTEST.COM for oryx/127.22.167.126@KRBTEST.COM, Server not found in Kerberos database
Aug 10 04:41:58 dist-test-slave-bqcl krb5kdc[4634](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.167.66@KRBTEST.COM for oryx/127.22.167.126@KRBTEST.COM, Server not found in Kerberos database
I20260810 04:41:58.199319 6415 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:41:58.195114 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.126:37819 (local address 127.22.167.66:47531)
0810 04:41:58.195277 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0810 04:41:58.195289 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0810 04:41:58.195352 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 04:41:58.195467 (+ 115us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 04:41:58.195467 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 04:41:58.195632 (+ 165us) client_negotiation.cc:189] Negotiated authn=SASL
0810 04:41:58.195681 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:41:58.195683 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:41:58.196267 (+ 584us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:41:58.196267 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:41:58.196805 (+ 538us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:41:58.196806 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:41:58.197075 (+ 269us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:41:58.197076 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:41:58.197111 (+ 35us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:41:58.197505 (+ 394us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 04:41:58.197509 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0810 04:41:58.199230 (+ 1721us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.167.126:37819: Server oryx/127.22.167.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":84,"threads_started":1}
W20260810 04:41:58.199391 6413 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 04:41:58.195233 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.167.66:47531 (local address 127.22.167.126:37819)
0810 04:41:58.195296 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0810 04:41:58.195298 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 04:41:58.195335 (+ 37us) server_negotiation.cc:407] Connection header received
0810 04:41:58.195400 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:41:58.195401 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:41:58.195413 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:41:58.195455 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0810 04:41:58.195728 (+ 273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:41:58.196216 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:41:58.196856 (+ 640us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:41:58.197029 (+ 173us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:41:58.197072 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:41:58.199338 (+ 2266us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.167.66:47531: BlockingRecv error: recv got EOF from 127.22.167.66:47531 (error 108)
Metrics: {"server-negotiator.queue_time_us":38}
Aug 10 04:41:58 dist-test-slave-bqcl krb5kdc[4634](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.167.67@KRBTEST.COM for oryx/127.22.167.126@KRBTEST.COM, Server not found in Kerberos database
Aug 10 04:41:58 dist-test-slave-bqcl krb5kdc[4634](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.167.67@KRBTEST.COM for oryx/127.22.167.126@KRBTEST.COM, Server not found in Kerberos database
I20260810 04:41:58.339095 6416 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:41:58.334623 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.126:37819 (local address 127.22.167.67:54271)
0810 04:41:58.334769 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0810 04:41:58.334780 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0810 04:41:58.334843 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 04:41:58.334967 (+ 124us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 04:41:58.334968 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 04:41:58.335143 (+ 175us) client_negotiation.cc:189] Negotiated authn=SASL
0810 04:41:58.335187 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:41:58.335189 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:41:58.335818 (+ 629us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:41:58.335819 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:41:58.336352 (+ 533us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:41:58.336354 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:41:58.336669 (+ 315us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:41:58.336670 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:41:58.336734 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:41:58.337131 (+ 397us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 04:41:58.337134 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0810 04:41:58.339000 (+ 1866us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.167.126:37819: Server oryx/127.22.167.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":64,"threads_started":1}
W20260810 04:41:58.339164 6413 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 04:41:58.334675 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.167.67:54271 (local address 127.22.167.126:37819)
0810 04:41:58.334730 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0810 04:41:58.334732 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 04:41:58.334836 (+ 104us) server_negotiation.cc:407] Connection header received
0810 04:41:58.334893 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:41:58.334894 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:41:58.334908 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:41:58.334955 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0810 04:41:58.335261 (+ 306us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:41:58.335748 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:41:58.336427 (+ 679us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:41:58.336611 (+ 184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:41:58.336665 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:41:58.339099 (+ 2434us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.167.67:54271: BlockingRecv error: recv got EOF from 127.22.167.67:54271 (error 108)
Metrics: {"server-negotiator.queue_time_us":28}
Aug 10 04:41:59 dist-test-slave-bqcl krb5kdc[4634](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.167.65@KRBTEST.COM for oryx/127.22.167.126@KRBTEST.COM, Server not found in Kerberos database
Aug 10 04:41:59 dist-test-slave-bqcl krb5kdc[4634](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.167.65@KRBTEST.COM for oryx/127.22.167.126@KRBTEST.COM, Server not found in Kerberos database
I20260810 04:41:59.085448 6420 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:41:59.081111 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.126:37819 (local address 127.22.167.65:35935)
0810 04:41:59.081265 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0810 04:41:59.081278 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0810 04:41:59.081336 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 04:41:59.081459 (+ 123us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 04:41:59.081460 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 04:41:59.081660 (+ 200us) client_negotiation.cc:189] Negotiated authn=SASL
0810 04:41:59.081715 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:41:59.081717 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:41:59.082436 (+ 719us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:41:59.082436 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:41:59.082968 (+ 532us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:41:59.082969 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:41:59.083251 (+ 282us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:41:59.083251 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:41:59.083288 (+ 37us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:41:59.083708 (+ 420us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 04:41:59.083712 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0810 04:41:59.085387 (+ 1675us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.167.126:37819: Server oryx/127.22.167.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":69,"threads_started":1}
W20260810 04:41:59.085561 6421 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 04:41:59.081169 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.167.65:35935 (local address 127.22.167.126:37819)
0810 04:41:59.081335 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0810 04:41:59.081337 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 04:41:59.081346 (+ 9us) server_negotiation.cc:407] Connection header received
0810 04:41:59.081369 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:41:59.081370 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:41:59.081381 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:41:59.081437 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0810 04:41:59.081794 (+ 357us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:41:59.082377 (+ 583us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:41:59.083024 (+ 647us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:41:59.083205 (+ 181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:41:59.083246 (+ 41us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:41:59.085483 (+ 2237us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.167.65:35935: BlockingRecv error: recv got EOF from 127.22.167.65:35935 (error 108)
Metrics: {"server-negotiator.queue_time_us":145,"thread_start_us":95,"threads_started":1}
Aug 10 04:41:59 dist-test-slave-bqcl krb5kdc[4634](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.167.66@KRBTEST.COM for oryx/127.22.167.126@KRBTEST.COM, Server not found in Kerberos database
Aug 10 04:41:59 dist-test-slave-bqcl krb5kdc[4634](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.167.66@KRBTEST.COM for oryx/127.22.167.126@KRBTEST.COM, Server not found in Kerberos database
I20260810 04:41:59.206746 6422 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:41:59.199768 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.126:37819 (local address 127.22.167.66:40617)
0810 04:41:59.199897 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0810 04:41:59.199907 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0810 04:41:59.199970 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 04:41:59.200138 (+ 168us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 04:41:59.200139 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 04:41:59.200313 (+ 174us) client_negotiation.cc:189] Negotiated authn=SASL
0810 04:41:59.200364 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:41:59.200365 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:41:59.200994 (+ 629us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:41:59.200995 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:41:59.201526 (+ 531us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:41:59.201527 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:41:59.201812 (+ 285us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:41:59.201812 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:41:59.201849 (+ 37us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:41:59.204936 (+ 3087us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 04:41:59.204940 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0810 04:41:59.206684 (+ 1744us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.167.126:37819: Server oryx/127.22.167.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":60,"threads_started":1}
W20260810 04:41:59.206851 6421 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 04:41:59.199842 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.167.66:40617 (local address 127.22.167.126:37819)
0810 04:41:59.199917 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0810 04:41:59.199919 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 04:41:59.199954 (+ 35us) server_negotiation.cc:407] Connection header received
0810 04:41:59.200014 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:41:59.200014 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:41:59.200026 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:41:59.200071 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0810 04:41:59.200428 (+ 357us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:41:59.200920 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:41:59.201584 (+ 664us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:41:59.201764 (+ 180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:41:59.201807 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:41:59.206787 (+ 4980us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.167.66:40617: BlockingRecv error: recv got EOF from 127.22.167.66:40617 (error 108)
Metrics: {"server-negotiator.queue_time_us":44}
Aug 10 04:41:59 dist-test-slave-bqcl krb5kdc[4634](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.167.67@KRBTEST.COM for oryx/127.22.167.126@KRBTEST.COM, Server not found in Kerberos database
Aug 10 04:41:59 dist-test-slave-bqcl krb5kdc[4634](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.167.67@KRBTEST.COM for oryx/127.22.167.126@KRBTEST.COM, Server not found in Kerberos database
I20260810 04:41:59.343884 6425 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:41:59.339543 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.126:37819 (local address 127.22.167.67:49879)
0810 04:41:59.339686 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0810 04:41:59.339697 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0810 04:41:59.339757 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 04:41:59.339880 (+ 123us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 04:41:59.339881 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 04:41:59.340048 (+ 167us) client_negotiation.cc:189] Negotiated authn=SASL
0810 04:41:59.340097 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:41:59.340098 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:41:59.340685 (+ 587us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:41:59.340685 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:41:59.341203 (+ 518us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:41:59.341204 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:41:59.341513 (+ 309us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:41:59.341514 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:41:59.341592 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:41:59.342087 (+ 495us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 04:41:59.342092 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0810 04:41:59.343805 (+ 1713us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.167.126:37819: Server oryx/127.22.167.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":81,"threads_started":1}
W20260810 04:41:59.343962 6421 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 04:41:59.339588 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.167.67:49879 (local address 127.22.167.126:37819)
0810 04:41:59.339633 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0810 04:41:59.339636 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 04:41:59.339755 (+ 119us) server_negotiation.cc:407] Connection header received
0810 04:41:59.339810 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:41:59.339811 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:41:59.339824 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:41:59.339863 (+ 39us) server_negotiation.cc:226] Negotiated authn=SASL
0810 04:41:59.340149 (+ 286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:41:59.340630 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:41:59.341261 (+ 631us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:41:59.341429 (+ 168us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:41:59.341471 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:41:59.343909 (+ 2438us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.167.67:49879: BlockingRecv error: recv got EOF from 127.22.167.67:49879 (error 108)
Metrics: {"server-negotiator.queue_time_us":21}
Aug 10 04:42:00 dist-test-slave-bqcl krb5kdc[4634](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.167.65@KRBTEST.COM for oryx/127.22.167.126@KRBTEST.COM, Server not found in Kerberos database
Aug 10 04:42:00 dist-test-slave-bqcl krb5kdc[4634](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.167.65@KRBTEST.COM for oryx/127.22.167.126@KRBTEST.COM, Server not found in Kerberos database
I20260810 04:42:00.090095 6426 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:42:00.085911 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.126:37819 (local address 127.22.167.65:35531)
0810 04:42:00.086027 (+ 116us) negotiation.cc:107] Waiting for socket to connect
0810 04:42:00.086037 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0810 04:42:00.086091 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 04:42:00.086237 (+ 146us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 04:42:00.086238 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 04:42:00.086398 (+ 160us) client_negotiation.cc:189] Negotiated authn=SASL
0810 04:42:00.086449 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:42:00.086450 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:42:00.087058 (+ 608us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:42:00.087059 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:42:00.087572 (+ 513us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:42:00.087573 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:42:00.087854 (+ 281us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:42:00.087855 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:42:00.087891 (+ 36us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:42:00.088277 (+ 386us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 04:42:00.088281 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0810 04:42:00.090034 (+ 1753us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.167.126:37819: Server oryx/127.22.167.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":99,"thread_start_us":70,"threads_started":1}
W20260810 04:42:00.090230 6429 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 04:42:00.085980 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.167.65:35531 (local address 127.22.167.126:37819)
0810 04:42:00.086087 (+ 107us) server_negotiation.cc:206] Beginning negotiation
0810 04:42:00.086089 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 04:42:00.086099 (+ 10us) server_negotiation.cc:407] Connection header received
0810 04:42:00.086158 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:42:00.086159 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:42:00.086171 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:42:00.086231 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0810 04:42:00.086502 (+ 271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:42:00.087006 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:42:00.087623 (+ 617us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:42:00.087807 (+ 184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:42:00.087850 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:42:00.090149 (+ 2299us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.167.65:35531: BlockingRecv error: recv got EOF from 127.22.167.65:35531 (error 108)
Metrics: {"server-negotiator.queue_time_us":81,"thread_start_us":55,"threads_started":1}
Aug 10 04:42:00 dist-test-slave-bqcl krb5kdc[4634](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.167.66@KRBTEST.COM for oryx/127.22.167.126@KRBTEST.COM, Server not found in Kerberos database
Aug 10 04:42:00 dist-test-slave-bqcl krb5kdc[4634](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.167.66@KRBTEST.COM for oryx/127.22.167.126@KRBTEST.COM, Server not found in Kerberos database
I20260810 04:42:00.211722 6431 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:42:00.207242 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.126:37819 (local address 127.22.167.66:46565)
0810 04:42:00.207373 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0810 04:42:00.207386 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0810 04:42:00.207454 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 04:42:00.207578 (+ 124us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 04:42:00.207579 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 04:42:00.207772 (+ 193us) client_negotiation.cc:189] Negotiated authn=SASL
0810 04:42:00.207821 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:42:00.207823 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:42:00.208470 (+ 647us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:42:00.208471 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:42:00.209013 (+ 542us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:42:00.209015 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:42:00.209360 (+ 345us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:42:00.209361 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:42:00.209414 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:42:00.209853 (+ 439us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 04:42:00.209857 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0810 04:42:00.211635 (+ 1778us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.167.126:37819: Server oryx/127.22.167.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":72,"threads_started":1}
W20260810 04:42:00.211791 6429 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 04:42:00.207337 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.167.66:46565 (local address 127.22.167.126:37819)
0810 04:42:00.207397 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0810 04:42:00.207399 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 04:42:00.207435 (+ 36us) server_negotiation.cc:407] Connection header received
0810 04:42:00.207512 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:42:00.207513 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:42:00.207525 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:42:00.207573 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0810 04:42:00.207893 (+ 320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:42:00.208400 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:42:00.209088 (+ 688us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:42:00.209295 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:42:00.209351 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:42:00.211729 (+ 2378us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.167.66:46565: BlockingRecv error: recv got EOF from 127.22.167.66:46565 (error 108)
Metrics: {"server-negotiator.queue_time_us":37}
I20260810 04:42:00.269176 23197 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskemv_AM/build/release/bin/kudu with pid 5615
I20260810 04:42:00.281198 23197 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskemv_AM/build/release/bin/kudu with pid 5880
I20260810 04:42:00.296262 23197 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskemv_AM/build/release/bin/kudu with pid 6098
I20260810 04:42:00.304708 23197 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskemv_AM/build/release/bin/kudu with pid 5495
2026-08-10T04:42:00Z chronyd exiting
[ OK ] SecurityITest.TestMismatchingPrincipals (17598 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationInsecureCluster
2026-08-10T04:42:00Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T04:42:00Z Disabled control of system clock
I20260810 04:42:00.340592 23197 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskemv_AM/build/release/bin/kudu
/tmp/dist-test-taskemv_AM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786336863388938-23197-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786336863388938-23197-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786336863388938-23197-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786336863388938-23197-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.167.126:39377
--webserver_interface=127.22.167.126
--webserver_port=0
--builtin_ntp_servers=127.22.167.84:41771
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.167.126:39377
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260810 04:42:00.420706 6437 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 04:42:00.420909 6437 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 04:42:00.420928 6437 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 04:42:00.422055 6437 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 04:42:00.422089 6437 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 04:42:00.422101 6437 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 04:42:00.422111 6437 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 04:42:00.422149 6437 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 04:42:00.423480 6437 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.167.84:41771
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786336863388938-23197-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786336863388938-23197-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.22.167.126:39377
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.167.126:39377
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786336863388938-23197-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.167.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6437
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786336863388938-23197-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 03:54:01 UTC on ccycloud.quasar-icijmt.root.comops.site
build id 4577009
I20260810 04:42:00.423678 6437 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 04:42:00.423906 6437 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 04:42:00.428916 6450 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 04:42:00.428944 6437 server_base.cc:1034] running on GCE node
W20260810 04:42:00.428947 6445 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 04:42:00.428917 6447 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 04:42:00.429450 6437 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 04:42:00.429729 6437 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 04:42:00.430866 6437 hybrid_clock.cc:648] HybridClock initialized: now 1786336920430849 us; error 32 us; skew 500 ppm
I20260810 04:42:00.432054 6437 webserver.cc:460] Webserver started at http://127.22.167.126:35123/ using document root <none> and password file <none>
I20260810 04:42:00.432276 6437 fs_manager.cc:359] Metadata directory not provided
I20260810 04:42:00.432320 6437 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 04:42:00.432443 6437 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 04:42:00.433485 6437 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786336863388938-23197-0/minicluster-data/master-0/data/instance:
uuid: "c83f376ffcf047df95345791e10d5ed4"
format_stamp: "Formatted at 2026-08-10 04:42:00 on dist-test-slave-bqcl"
server_key: "0579260fbf09bed9f10f57f7238d785c"
server_key_iv: "c3350e23438b9f750d978002c62622d3"
server_key_version: "encryptionkey@0"
I20260810 04:42:00.433861 6437 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786336863388938-23197-0/minicluster-data/master-0/wal/instance:
uuid: "c83f376ffcf047df95345791e10d5ed4"
format_stamp: "Formatted at 2026-08-10 04:42:00 on dist-test-slave-bqcl"
server_key: "0579260fbf09bed9f10f57f7238d785c"
server_key_iv: "c3350e23438b9f750d978002c62622d3"
server_key_version: "encryptionkey@0"
I20260810 04:42:00.435155 6437 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260810 04:42:00.435815 6465 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 04:42:00.435958 6437 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260810 04:42:00.436043 6437 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786336863388938-23197-0/minicluster-data/master-0/data,/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786336863388938-23197-0/minicluster-data/master-0/wal
uuid: "c83f376ffcf047df95345791e10d5ed4"
format_stamp: "Formatted at 2026-08-10 04:42:00 on dist-test-slave-bqcl"
server_key: "0579260fbf09bed9f10f57f7238d785c"
server_key_iv: "c3350e23438b9f750d978002c62622d3"
server_key_version: "encryptionkey@0"
I20260810 04:42:00.436105 6437 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786336863388938-23197-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786336863388938-23197-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786336863388938-23197-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 04:42:00.450058 6437 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 04:42:00.450337 6437 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 04:42:00.450441 6437 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 04:42:00.454234 6437 rpc_server.cc:307] RPC server started. Bound to: 127.22.167.126:39377
I20260810 04:42:00.454326 6550 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.167.126:39377 every 8 connection(s)
I20260810 04:42:00.454622 6437 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786336863388938-23197-0/minicluster-data/master-0/data/info.pb
I20260810 04:42:00.454833 23197 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskemv_AM/build/release/bin/kudu as pid 6437
I20260810 04:42:00.454936 23197 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786336863388938-23197-0/minicluster-data/master-0/wal/instance
I20260810 04:42:00.455137 23197 external_mini_cluster.cc:1428] Setting key 2f530c25952394f3db257ddd09a75276
I20260810 04:42:00.455427 6551 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 04:42:00.458279 6551 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c83f376ffcf047df95345791e10d5ed4: Bootstrap starting.
I20260810 04:42:00.458830 6551 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c83f376ffcf047df95345791e10d5ed4: Neither blocks nor log segments found. Creating new log.
I20260810 04:42:00.459108 6551 log.cc:824] T 00000000000000000000000000000000 P c83f376ffcf047df95345791e10d5ed4: Log is configured to *not* fsync() on all Append() calls
I20260810 04:42:00.459779 6551 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c83f376ffcf047df95345791e10d5ed4: No bootstrap required, opened a new log
I20260810 04:42:00.460578 6554 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:42:00.455949 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50466 (local address 127.22.167.126:39377)
0810 04:42:00.456146 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0810 04:42:00.456154 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 04:42:00.456186 (+ 32us) server_negotiation.cc:407] Connection header received
0810 04:42:00.456604 (+ 418us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:42:00.456611 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:42:00.456674 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:42:00.456799 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0810 04:42:00.457013 (+ 214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:42:00.457877 (+ 864us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:42:00.458734 (+ 857us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:42:00.459214 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:42:00.459325 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:42:00.459989 (+ 664us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 04:42:00.459993 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 04:42:00.459994 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 04:42:00.460155 (+ 161us) server_negotiation.cc:922] calling sasl_server_start()
0810 04:42:00.460196 (+ 41us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 04:42:00.460206 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 04:42:00.460259 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0810 04:42:00.460334 (+ 75us) server_negotiation.cc:299] Negotiation successful
0810 04:42:00.460413 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":126,"thread_start_us":86,"threads_started":1}
I20260810 04:42:00.461724 6551 raft_consensus.cc:348] T 00000000000000000000000000000000 P c83f376ffcf047df95345791e10d5ed4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c83f376ffcf047df95345791e10d5ed4" member_type: VOTER last_known_addr { host: "127.22.167.126" port: 39377 } }
I20260810 04:42:00.461853 6551 raft_consensus.cc:374] T 00000000000000000000000000000000 P c83f376ffcf047df95345791e10d5ed4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 04:42:00.461884 6551 raft_consensus.cc:729] T 00000000000000000000000000000000 P c83f376ffcf047df95345791e10d5ed4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c83f376ffcf047df95345791e10d5ed4, State: Initialized, Role: FOLLOWER
I20260810 04:42:00.461995 6551 consensus_queue.cc:260] T 00000000000000000000000000000000 P c83f376ffcf047df95345791e10d5ed4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c83f376ffcf047df95345791e10d5ed4" member_type: VOTER last_known_addr { host: "127.22.167.126" port: 39377 } }
I20260810 04:42:00.462070 6551 raft_consensus.cc:388] T 00000000000000000000000000000000 P c83f376ffcf047df95345791e10d5ed4 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 04:42:00.462102 6551 raft_consensus.cc:482] T 00000000000000000000000000000000 P c83f376ffcf047df95345791e10d5ed4 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 04:42:00.462174 6551 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c83f376ffcf047df95345791e10d5ed4 [term 0 FOLLOWER]: Advancing to term 1
I20260810 04:42:00.462643 6551 raft_consensus.cc:504] T 00000000000000000000000000000000 P c83f376ffcf047df95345791e10d5ed4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c83f376ffcf047df95345791e10d5ed4" member_type: VOTER last_known_addr { host: "127.22.167.126" port: 39377 } }
I20260810 04:42:00.462733 6551 leader_election.cc:302] T 00000000000000000000000000000000 P c83f376ffcf047df95345791e10d5ed4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c83f376ffcf047df95345791e10d5ed4; no voters:
I20260810 04:42:00.462850 6551 leader_election.cc:288] T 00000000000000000000000000000000 P c83f376ffcf047df95345791e10d5ed4 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 04:42:00.462910 6556 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c83f376ffcf047df95345791e10d5ed4 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 04:42:00.463068 6551 sys_catalog.cc:564] T 00000000000000000000000000000000 P c83f376ffcf047df95345791e10d5ed4 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 04:42:00.463085 6556 raft_consensus.cc:686] T 00000000000000000000000000000000 P c83f376ffcf047df95345791e10d5ed4 [term 1 LEADER]: Becoming Leader. State: Replica: c83f376ffcf047df95345791e10d5ed4, State: Running, Role: LEADER
I20260810 04:42:00.463244 6556 consensus_queue.cc:237] T 00000000000000000000000000000000 P c83f376ffcf047df95345791e10d5ed4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c83f376ffcf047df95345791e10d5ed4" member_type: VOTER last_known_addr { host: "127.22.167.126" port: 39377 } }
I20260810 04:42:00.463591 6557 sys_catalog.cc:455] T 00000000000000000000000000000000 P c83f376ffcf047df95345791e10d5ed4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c83f376ffcf047df95345791e10d5ed4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c83f376ffcf047df95345791e10d5ed4" member_type: VOTER last_known_addr { host: "127.22.167.126" port: 39377 } } }
I20260810 04:42:00.463590 6560 sys_catalog.cc:455] T 00000000000000000000000000000000 P c83f376ffcf047df95345791e10d5ed4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c83f376ffcf047df95345791e10d5ed4. Latest consensus state: current_term: 1 leader_uuid: "c83f376ffcf047df95345791e10d5ed4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c83f376ffcf047df95345791e10d5ed4" member_type: VOTER last_known_addr { host: "127.22.167.126" port: 39377 } } }
I20260810 04:42:00.463722 6557 sys_catalog.cc:458] T 00000000000000000000000000000000 P c83f376ffcf047df95345791e10d5ed4 [sys.catalog]: This master's current role is: LEADER
I20260810 04:42:00.463792 6560 sys_catalog.cc:458] T 00000000000000000000000000000000 P c83f376ffcf047df95345791e10d5ed4 [sys.catalog]: This master's current role is: LEADER
I20260810 04:42:00.464212 6573 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 04:42:00.464690 6573 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 04:42:00.466271 6573 catalog_manager.cc:1347] Generated new cluster ID: a83aacf9fcef40c08f7b0689cef72c6d
I20260810 04:42:00.466315 6573 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 04:42:00.479328 6573 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 04:42:00.479820 6573 catalog_manager.cc:1504] Loading token signing keys...
I20260810 04:42:00.486984 6573 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c83f376ffcf047df95345791e10d5ed4: Generated new TSK 0
I20260810 04:42:00.487140 6573 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 04:42:00.491080 23197 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskemv_AM/build/release/bin/kudu
/tmp/dist-test-taskemv_AM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786336863388938-23197-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786336863388938-23197-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786336863388938-23197-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786336863388938-23197-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.167.65:0
--local_ip_for_outbound_sockets=127.22.167.65
--webserver_interface=127.22.167.65
--webserver_port=0
--tserver_master_addrs=127.22.167.126:39377
--builtin_ntp_servers=127.22.167.84:41771
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260810 04:42:00.587801 6591 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 04:42:00.587990 6591 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 04:42:00.588011 6591 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 04:42:00.589115 6591 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 04:42:00.589155 6591 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 04:42:00.589178 6591 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.167.65
I20260810 04:42:00.590514 6591 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.167.84:41771
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786336863388938-23197-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786336863388938-23197-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.167.65:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786336863388938-23197-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.22.167.65
--webserver_port=0
--tserver_master_addrs=127.22.167.126:39377
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6591
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.167.65
--log_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786336863388938-23197-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 03:54:01 UTC on ccycloud.quasar-icijmt.root.comops.site
build id 4577009
I20260810 04:42:00.590739 6591 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 04:42:00.591020 6591 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 04:42:00.596990 6610 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 04:42:00.597021 6605 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 04:42:00.597198 6591 server_base.cc:1034] running on GCE node
W20260810 04:42:00.596995 6604 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 04:42:00.597419 6591 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 04:42:00.597646 6591 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 04:42:00.598798 6591 hybrid_clock.cc:648] HybridClock initialized: now 1786336920598784 us; error 35 us; skew 500 ppm
I20260810 04:42:00.599974 6591 webserver.cc:460] Webserver started at http://127.22.167.65:41335/ using document root <none> and password file <none>
I20260810 04:42:00.600214 6591 fs_manager.cc:359] Metadata directory not provided
I20260810 04:42:00.600256 6591 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 04:42:00.600380 6591 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 04:42:00.601375 6591 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786336863388938-23197-0/minicluster-data/ts-0/data/instance:
uuid: "ba748c4bef814b3091aaabd882a1a172"
format_stamp: "Formatted at 2026-08-10 04:42:00 on dist-test-slave-bqcl"
server_key: "efa276674c1149f7f27b1ade51a378b9"
server_key_iv: "ce5a36883ede1ac83dc359fcd5491e08"
server_key_version: "encryptionkey@0"
I20260810 04:42:00.601686 6591 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786336863388938-23197-0/minicluster-data/ts-0/wal/instance:
uuid: "ba748c4bef814b3091aaabd882a1a172"
format_stamp: "Formatted at 2026-08-10 04:42:00 on dist-test-slave-bqcl"
server_key: "efa276674c1149f7f27b1ade51a378b9"
server_key_iv: "ce5a36883ede1ac83dc359fcd5491e08"
server_key_version: "encryptionkey@0"
I20260810 04:42:00.602957 6591 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260810 04:42:00.603652 6619 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 04:42:00.603826 6591 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260810 04:42:00.603909 6591 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786336863388938-23197-0/minicluster-data/ts-0/data,/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786336863388938-23197-0/minicluster-data/ts-0/wal
uuid: "ba748c4bef814b3091aaabd882a1a172"
format_stamp: "Formatted at 2026-08-10 04:42:00 on dist-test-slave-bqcl"
server_key: "efa276674c1149f7f27b1ade51a378b9"
server_key_iv: "ce5a36883ede1ac83dc359fcd5491e08"
server_key_version: "encryptionkey@0"
I20260810 04:42:00.603961 6591 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786336863388938-23197-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786336863388938-23197-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786336863388938-23197-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 04:42:00.614980 6591 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 04:42:00.615211 6591 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 04:42:00.615306 6591 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 04:42:00.615468 6591 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 04:42:00.615717 6591 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 04:42:00.615746 6591 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 04:42:00.615772 6591 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 04:42:00.615788 6591 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 04:42:00.621232 6591 rpc_server.cc:307] RPC server started. Bound to: 127.22.167.65:41133
I20260810 04:42:00.621297 6811 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.167.65:41133 every 8 connection(s)
I20260810 04:42:00.621573 6591 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786336863388938-23197-0/minicluster-data/ts-0/data/info.pb
I20260810 04:42:00.626541 6554 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:42:00.622237 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.167.65:36661 (local address 127.22.167.126:39377)
0810 04:42:00.622282 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0810 04:42:00.622284 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 04:42:00.622525 (+ 241us) server_negotiation.cc:407] Connection header received
0810 04:42:00.623045 (+ 520us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:42:00.623047 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:42:00.623063 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:42:00.623118 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0810 04:42:00.623787 (+ 669us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:42:00.624294 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:42:00.625269 (+ 975us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:42:00.625466 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:42:00.625521 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:42:00.626268 (+ 747us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 04:42:00.626270 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 04:42:00.626270 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 04:42:00.626275 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0810 04:42:00.626292 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 04:42:00.626296 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 04:42:00.626343 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0810 04:42:00.626417 (+ 74us) server_negotiation.cc:299] Negotiation successful
0810 04:42:00.626458 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260810 04:42:00.626739 6817 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:42:00.622233 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.126:39377 (local address 127.22.167.65:36661)
0810 04:42:00.622416 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0810 04:42:00.622448 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0810 04:42:00.622934 (+ 486us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 04:42:00.623185 (+ 251us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 04:42:00.623189 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 04:42:00.623197 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0810 04:42:00.623697 (+ 500us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:42:00.623702 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:42:00.624379 (+ 677us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:42:00.624380 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:42:00.625175 (+ 795us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:42:00.625178 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:42:00.625539 (+ 361us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:42:00.625540 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:42:00.625662 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:42:00.626112 (+ 450us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 04:42:00.626112 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0810 04:42:00.626173 (+ 61us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 04:42:00.626196 (+ 23us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 04:42:00.626348 (+ 152us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 04:42:00.626354 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 04:42:00.626357 (+ 3us) client_negotiation.cc:769] Sending connection context
0810 04:42:00.626416 (+ 59us) client_negotiation.cc:240] Negotiation successful
0810 04:42:00.626552 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":83,"threads_started":1}
I20260810 04:42:00.627241 6814 heartbeater.cc:347] Connected to a master server at 127.22.167.126:39377
I20260810 04:42:00.627315 6814 heartbeater.cc:464] Registering TS with master...
I20260810 04:42:00.627460 6814 heartbeater.cc:511] Master 127.22.167.126:39377 requested a full tablet report, sending...
I20260810 04:42:00.627915 6497 ts_manager.cc:194] Registered new tserver with Master: ba748c4bef814b3091aaabd882a1a172 (127.22.167.65:41133)
I20260810 04:42:00.628304 23197 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskemv_AM/build/release/bin/kudu as pid 6591
I20260810 04:42:00.628388 23197 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786336863388938-23197-0/minicluster-data/ts-0/wal/instance
I20260810 04:42:00.628583 23197 external_mini_cluster.cc:1428] Setting key c5885c4d663b63ddd85130f47b895293
I20260810 04:42:00.628566 6497 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.22.167.65:36661
I20260810 04:42:00.629423 23197 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskemv_AM/build/release/bin/kudu
/tmp/dist-test-taskemv_AM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786336863388938-23197-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786336863388938-23197-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786336863388938-23197-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786336863388938-23197-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.167.66:0
--local_ip_for_outbound_sockets=127.22.167.66
--webserver_interface=127.22.167.66
--webserver_port=0
--tserver_master_addrs=127.22.167.126:39377
--builtin_ntp_servers=127.22.167.84:41771
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260810 04:42:00.711447 6819 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 04:42:00.711650 6819 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 04:42:00.711680 6819 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 04:42:00.712779 6819 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 04:42:00.712816 6819 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 04:42:00.712847 6819 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.167.66
I20260810 04:42:00.714171 6819 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.167.84:41771
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786336863388938-23197-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786336863388938-23197-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.167.66:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786336863388938-23197-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.22.167.66
--webserver_port=0
--tserver_master_addrs=127.22.167.126:39377
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6819
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.167.66
--log_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786336863388938-23197-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 03:54:01 UTC on ccycloud.quasar-icijmt.root.comops.site
build id 4577009
I20260810 04:42:00.714401 6819 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 04:42:00.714671 6819 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 04:42:00.719321 6830 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 04:42:00.719449 6819 server_base.cc:1034] running on GCE node
W20260810 04:42:00.719336 6826 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 04:42:00.719347 6825 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 04:42:00.719790 6819 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 04:42:00.720009 6819 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 04:42:00.721145 6819 hybrid_clock.cc:648] HybridClock initialized: now 1786336920721129 us; error 30 us; skew 500 ppm
I20260810 04:42:00.722363 6819 webserver.cc:460] Webserver started at http://127.22.167.66:38721/ using document root <none> and password file <none>
I20260810 04:42:00.722579 6819 fs_manager.cc:359] Metadata directory not provided
I20260810 04:42:00.722626 6819 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 04:42:00.722764 6819 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 04:42:00.723748 6819 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786336863388938-23197-0/minicluster-data/ts-1/data/instance:
uuid: "50167f449f264f4dbe72d13e11ac7a03"
format_stamp: "Formatted at 2026-08-10 04:42:00 on dist-test-slave-bqcl"
server_key: "a96298cefe472980ae8a1f969f3ca8e1"
server_key_iv: "0ede492a31ee75577e367fc0131aac6f"
server_key_version: "encryptionkey@0"
I20260810 04:42:00.724083 6819 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786336863388938-23197-0/minicluster-data/ts-1/wal/instance:
uuid: "50167f449f264f4dbe72d13e11ac7a03"
format_stamp: "Formatted at 2026-08-10 04:42:00 on dist-test-slave-bqcl"
server_key: "a96298cefe472980ae8a1f969f3ca8e1"
server_key_iv: "0ede492a31ee75577e367fc0131aac6f"
server_key_version: "encryptionkey@0"
I20260810 04:42:00.725247 6819 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260810 04:42:00.725854 6841 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 04:42:00.726002 6819 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260810 04:42:00.726083 6819 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786336863388938-23197-0/minicluster-data/ts-1/data,/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786336863388938-23197-0/minicluster-data/ts-1/wal
uuid: "50167f449f264f4dbe72d13e11ac7a03"
format_stamp: "Formatted at 2026-08-10 04:42:00 on dist-test-slave-bqcl"
server_key: "a96298cefe472980ae8a1f969f3ca8e1"
server_key_iv: "0ede492a31ee75577e367fc0131aac6f"
server_key_version: "encryptionkey@0"
I20260810 04:42:00.726167 6819 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786336863388938-23197-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786336863388938-23197-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786336863388938-23197-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 04:42:00.754052 6819 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 04:42:00.754347 6819 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 04:42:00.754477 6819 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 04:42:00.754694 6819 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 04:42:00.755009 6819 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 04:42:00.755045 6819 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 04:42:00.755074 6819 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 04:42:00.755096 6819 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 04:42:00.760257 6819 rpc_server.cc:307] RPC server started. Bound to: 127.22.167.66:44707
I20260810 04:42:00.760324 7043 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.167.66:44707 every 8 connection(s)
I20260810 04:42:00.760602 6819 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786336863388938-23197-0/minicluster-data/ts-1/data/info.pb
I20260810 04:42:00.764603 23197 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskemv_AM/build/release/bin/kudu as pid 6819
I20260810 04:42:00.764714 23197 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786336863388938-23197-0/minicluster-data/ts-1/wal/instance
I20260810 04:42:00.764928 23197 external_mini_cluster.cc:1428] Setting key 8348b2e4d46d03aa84a035bcb51682cb
I20260810 04:42:00.765691 6554 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:42:00.761291 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.167.66:40707 (local address 127.22.167.126:39377)
0810 04:42:00.761351 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0810 04:42:00.761353 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 04:42:00.761561 (+ 208us) server_negotiation.cc:407] Connection header received
0810 04:42:00.762031 (+ 470us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:42:00.762032 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:42:00.762042 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:42:00.762097 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0810 04:42:00.762807 (+ 710us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:42:00.763291 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:42:00.764310 (+ 1019us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:42:00.764507 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:42:00.764552 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:42:00.765439 (+ 887us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 04:42:00.765440 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 04:42:00.765440 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 04:42:00.765445 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0810 04:42:00.765460 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 04:42:00.765464 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 04:42:00.765510 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0810 04:42:00.765585 (+ 75us) server_negotiation.cc:299] Negotiation successful
0810 04:42:00.765611 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260810 04:42:00.765815 23197 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskemv_AM/build/release/bin/kudu
/tmp/dist-test-taskemv_AM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786336863388938-23197-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786336863388938-23197-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786336863388938-23197-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786336863388938-23197-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.167.67:0
--local_ip_for_outbound_sockets=127.22.167.67
--webserver_interface=127.22.167.67
--webserver_port=0
--tserver_master_addrs=127.22.167.126:39377
--builtin_ntp_servers=127.22.167.84:41771
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260810 04:42:00.765893 7048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:42:00.761263 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.126:39377 (local address 127.22.167.66:40707)
0810 04:42:00.761443 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0810 04:42:00.761484 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0810 04:42:00.761954 (+ 470us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 04:42:00.762193 (+ 239us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 04:42:00.762196 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 04:42:00.762204 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0810 04:42:00.762736 (+ 532us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:42:00.762742 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:42:00.763359 (+ 617us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:42:00.763360 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:42:00.764240 (+ 880us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:42:00.764242 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:42:00.764590 (+ 348us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:42:00.764592 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:42:00.764749 (+ 157us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:42:00.765300 (+ 551us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 04:42:00.765300 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0810 04:42:00.765328 (+ 28us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 04:42:00.765351 (+ 23us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 04:42:00.765516 (+ 165us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 04:42:00.765521 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 04:42:00.765524 (+ 3us) client_negotiation.cc:769] Sending connection context
0810 04:42:00.765586 (+ 62us) client_negotiation.cc:240] Negotiation successful
0810 04:42:00.765685 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":80,"threads_started":1}
I20260810 04:42:00.766367 7045 heartbeater.cc:347] Connected to a master server at 127.22.167.126:39377
I20260810 04:42:00.766455 7045 heartbeater.cc:464] Registering TS with master...
I20260810 04:42:00.766619 7045 heartbeater.cc:511] Master 127.22.167.126:39377 requested a full tablet report, sending...
I20260810 04:42:00.766956 6497 ts_manager.cc:194] Registered new tserver with Master: 50167f449f264f4dbe72d13e11ac7a03 (127.22.167.66:44707)
I20260810 04:42:00.767403 6497 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.22.167.66:40707
W20260810 04:42:00.846864 7049 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 04:42:00.847054 7049 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 04:42:00.847074 7049 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 04:42:00.848135 7049 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 04:42:00.848171 7049 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 04:42:00.848196 7049 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.167.67
I20260810 04:42:00.849431 7049 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.167.84:41771
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786336863388938-23197-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786336863388938-23197-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.167.67:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786336863388938-23197-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.22.167.67
--webserver_port=0
--tserver_master_addrs=127.22.167.126:39377
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7049
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.167.67
--log_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786336863388938-23197-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 03:54:01 UTC on ccycloud.quasar-icijmt.root.comops.site
build id 4577009
I20260810 04:42:00.849604 7049 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 04:42:00.849831 7049 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 04:42:00.854517 7060 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 04:42:00.854527 7058 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 04:42:00.854527 7056 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 04:42:00.854581 7049 server_base.cc:1034] running on GCE node
I20260810 04:42:00.854919 7049 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 04:42:00.855109 7049 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 04:42:00.856257 7049 hybrid_clock.cc:648] HybridClock initialized: now 1786336920856251 us; error 34 us; skew 500 ppm
I20260810 04:42:00.857390 7049 webserver.cc:460] Webserver started at http://127.22.167.67:41879/ using document root <none> and password file <none>
I20260810 04:42:00.857599 7049 fs_manager.cc:359] Metadata directory not provided
I20260810 04:42:00.857643 7049 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 04:42:00.857758 7049 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 04:42:00.858795 7049 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786336863388938-23197-0/minicluster-data/ts-2/data/instance:
uuid: "1e17d70cdf06411498c810bc1576cc3d"
format_stamp: "Formatted at 2026-08-10 04:42:00 on dist-test-slave-bqcl"
server_key: "acac5a9da9382e46eaeabd27c8c61837"
server_key_iv: "465991c64ad0851a6212885d41e346b2"
server_key_version: "encryptionkey@0"
I20260810 04:42:00.859114 7049 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786336863388938-23197-0/minicluster-data/ts-2/wal/instance:
uuid: "1e17d70cdf06411498c810bc1576cc3d"
format_stamp: "Formatted at 2026-08-10 04:42:00 on dist-test-slave-bqcl"
server_key: "acac5a9da9382e46eaeabd27c8c61837"
server_key_iv: "465991c64ad0851a6212885d41e346b2"
server_key_version: "encryptionkey@0"
I20260810 04:42:00.860289 7049 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260810 04:42:00.860908 7072 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 04:42:00.861094 7049 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260810 04:42:00.861179 7049 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786336863388938-23197-0/minicluster-data/ts-2/data,/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786336863388938-23197-0/minicluster-data/ts-2/wal
uuid: "1e17d70cdf06411498c810bc1576cc3d"
format_stamp: "Formatted at 2026-08-10 04:42:00 on dist-test-slave-bqcl"
server_key: "acac5a9da9382e46eaeabd27c8c61837"
server_key_iv: "465991c64ad0851a6212885d41e346b2"
server_key_version: "encryptionkey@0"
I20260810 04:42:00.861244 7049 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786336863388938-23197-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786336863388938-23197-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786336863388938-23197-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 04:42:00.883868 7049 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 04:42:00.884147 7049 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 04:42:00.884282 7049 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 04:42:00.884496 7049 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 04:42:00.884819 7049 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 04:42:00.884855 7049 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 04:42:00.884879 7049 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 04:42:00.884902 7049 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 04:42:00.890242 7049 rpc_server.cc:307] RPC server started. Bound to: 127.22.167.67:41333
I20260810 04:42:00.890286 7265 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.167.67:41333 every 8 connection(s)
I20260810 04:42:00.890636 7049 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786336863388938-23197-0/minicluster-data/ts-2/data/info.pb
I20260810 04:42:00.891471 23197 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskemv_AM/build/release/bin/kudu as pid 7049
I20260810 04:42:00.891563 23197 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786336863388938-23197-0/minicluster-data/ts-2/wal/instance
I20260810 04:42:00.891748 23197 external_mini_cluster.cc:1428] Setting key 868670b78312046cc0c0970de2ec321d
I20260810 04:42:00.895467 6554 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:42:00.891272 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.167.67:54947 (local address 127.22.167.126:39377)
0810 04:42:00.891333 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0810 04:42:00.891335 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 04:42:00.891547 (+ 212us) server_negotiation.cc:407] Connection header received
0810 04:42:00.892106 (+ 559us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:42:00.892107 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:42:00.892120 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:42:00.892172 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0810 04:42:00.892957 (+ 785us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:42:00.893432 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:42:00.894382 (+ 950us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:42:00.894565 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:42:00.894612 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:42:00.895270 (+ 658us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 04:42:00.895271 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 04:42:00.895271 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 04:42:00.895274 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0810 04:42:00.895288 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 04:42:00.895291 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 04:42:00.895330 (+ 39us) server_negotiation.cc:1035] Waiting for connection context
0810 04:42:00.895390 (+ 60us) server_negotiation.cc:299] Negotiation successful
0810 04:42:00.895410 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260810 04:42:00.895646 7269 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:42:00.891250 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.126:39377 (local address 127.22.167.67:54947)
0810 04:42:00.891439 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0810 04:42:00.891471 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0810 04:42:00.892005 (+ 534us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 04:42:00.892252 (+ 247us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 04:42:00.892257 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 04:42:00.892267 (+ 10us) client_negotiation.cc:189] Negotiated authn=SASL
0810 04:42:00.892884 (+ 617us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:42:00.892890 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:42:00.893493 (+ 603us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:42:00.893493 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:42:00.894321 (+ 828us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:42:00.894324 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:42:00.894618 (+ 294us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:42:00.894618 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:42:00.894731 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:42:00.895153 (+ 422us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 04:42:00.895153 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0810 04:42:00.895183 (+ 30us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 04:42:00.895204 (+ 21us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 04:42:00.895335 (+ 131us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 04:42:00.895338 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 04:42:00.895340 (+ 2us) client_negotiation.cc:769] Sending connection context
0810 04:42:00.895384 (+ 44us) client_negotiation.cc:240] Negotiation successful
0810 04:42:00.895459 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":86,"threads_started":1}
I20260810 04:42:00.896013 7266 heartbeater.cc:347] Connected to a master server at 127.22.167.126:39377
I20260810 04:42:00.896095 7266 heartbeater.cc:464] Registering TS with master...
I20260810 04:42:00.896265 7266 heartbeater.cc:511] Master 127.22.167.126:39377 requested a full tablet report, sending...
I20260810 04:42:00.896552 6497 ts_manager.cc:194] Registered new tserver with Master: 1e17d70cdf06411498c810bc1576cc3d (127.22.167.67:41333)
I20260810 04:42:00.896903 6497 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.22.167.67:54947
I20260810 04:42:00.903241 23197 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260810 04:42:00.904906 6554 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 04:42:00.904444 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50468 (local address 127.22.167.126:39377)
0810 04:42:00.904492 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0810 04:42:00.904494 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 04:42:00.904611 (+ 117us) server_negotiation.cc:407] Connection header received
0810 04:42:00.904656 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:42:00.904657 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:42:00.904683 (+ 26us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:42:00.904727 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0810 04:42:00.904845 (+ 118us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:50468: BlockingRecv error: recv got EOF from 127.0.0.1:50468 (error 108)
Metrics: {"server-negotiator.queue_time_us":27}
I20260810 04:42:00.904947 23197 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskemv_AM/build/release/bin/kudu with pid 6591
I20260810 04:42:00.921823 23197 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskemv_AM/build/release/bin/kudu with pid 6819
I20260810 04:42:00.938949 23197 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskemv_AM/build/release/bin/kudu with pid 7049
I20260810 04:42:00.947724 23197 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskemv_AM/build/release/bin/kudu with pid 6437
2026-08-10T04:42:00Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationInsecureCluster (640 ms)
[ RUN ] SecurityITest.TestRequireEncryptionInsecureCluster
2026-08-10T04:42:00Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T04:42:00Z Disabled control of system clock
I20260810 04:42:00.981076 23197 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskemv_AM/build/release/bin/kudu
/tmp/dist-test-taskemv_AM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786336863388938-23197-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786336863388938-23197-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786336863388938-23197-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786336863388938-23197-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.167.126:41787
--webserver_interface=127.22.167.126
--webserver_port=0
--builtin_ntp_servers=127.22.167.84:37289
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.167.126:41787
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260810 04:42:01.061152 7284 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 04:42:01.061354 7284 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 04:42:01.061385 7284 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 04:42:01.062498 7284 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 04:42:01.062539 7284 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 04:42:01.062556 7284 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 04:42:01.062573 7284 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 04:42:01.062587 7284 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 04:42:01.063880 7284 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.167.84:37289
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786336863388938-23197-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786336863388938-23197-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.22.167.126:41787
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.167.126:41787
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786336863388938-23197-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.167.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7284
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786336863388938-23197-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 03:54:01 UTC on ccycloud.quasar-icijmt.root.comops.site
build id 4577009
I20260810 04:42:01.064109 7284 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 04:42:01.064361 7284 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 04:42:01.068882 7290 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 04:42:01.068962 7292 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 04:42:01.068915 7289 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 04:42:01.069042 7284 server_base.cc:1034] running on GCE node
I20260810 04:42:01.069289 7284 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 04:42:01.069561 7284 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 04:42:01.070703 7284 hybrid_clock.cc:648] HybridClock initialized: now 1786336921070688 us; error 29 us; skew 500 ppm
I20260810 04:42:01.071841 7284 webserver.cc:460] Webserver started at http://127.22.167.126:42947/ using document root <none> and password file <none>
I20260810 04:42:01.072041 7284 fs_manager.cc:359] Metadata directory not provided
I20260810 04:42:01.072084 7284 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 04:42:01.072213 7284 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 04:42:01.073242 7284 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786336863388938-23197-0/minicluster-data/master-0/data/instance:
uuid: "9ed63600bcde42aa9cbf8e044666f74a"
format_stamp: "Formatted at 2026-08-10 04:42:01 on dist-test-slave-bqcl"
server_key: "f80ec70fc980366326d5916c68872032"
server_key_iv: "16a1ed133370111a5f1bdb19323b6958"
server_key_version: "encryptionkey@0"
I20260810 04:42:01.073560 7284 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786336863388938-23197-0/minicluster-data/master-0/wal/instance:
uuid: "9ed63600bcde42aa9cbf8e044666f74a"
format_stamp: "Formatted at 2026-08-10 04:42:01 on dist-test-slave-bqcl"
server_key: "f80ec70fc980366326d5916c68872032"
server_key_iv: "16a1ed133370111a5f1bdb19323b6958"
server_key_version: "encryptionkey@0"
I20260810 04:42:01.074826 7284 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260810 04:42:01.075464 7306 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 04:42:01.075593 7284 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260810 04:42:01.075686 7284 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786336863388938-23197-0/minicluster-data/master-0/data,/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786336863388938-23197-0/minicluster-data/master-0/wal
uuid: "9ed63600bcde42aa9cbf8e044666f74a"
format_stamp: "Formatted at 2026-08-10 04:42:01 on dist-test-slave-bqcl"
server_key: "f80ec70fc980366326d5916c68872032"
server_key_iv: "16a1ed133370111a5f1bdb19323b6958"
server_key_version: "encryptionkey@0"
I20260810 04:42:01.075744 7284 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786336863388938-23197-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786336863388938-23197-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786336863388938-23197-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 04:42:01.078929 7284 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 04:42:01.079175 7284 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 04:42:01.079294 7284 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 04:42:01.082983 7284 rpc_server.cc:307] RPC server started. Bound to: 127.22.167.126:41787
I20260810 04:42:01.083015 7399 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.167.126:41787 every 8 connection(s)
I20260810 04:42:01.083303 7284 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786336863388938-23197-0/minicluster-data/master-0/data/info.pb
I20260810 04:42:01.083892 7400 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 04:42:01.084936 23197 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskemv_AM/build/release/bin/kudu as pid 7284
I20260810 04:42:01.085031 23197 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786336863388938-23197-0/minicluster-data/master-0/wal/instance
I20260810 04:42:01.085208 23197 external_mini_cluster.cc:1428] Setting key d224ed25e3aa1c490cffbb4642ad0a18
I20260810 04:42:01.087406 7400 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9ed63600bcde42aa9cbf8e044666f74a: Bootstrap starting.
I20260810 04:42:01.087767 7403 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:42:01.085688 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33244 (local address 127.22.167.126:41787)
0810 04:42:01.085919 (+ 231us) server_negotiation.cc:206] Beginning negotiation
0810 04:42:01.085928 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 04:42:01.085963 (+ 35us) server_negotiation.cc:407] Connection header received
0810 04:42:01.086527 (+ 564us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:42:01.086536 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:42:01.086592 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:42:01.086752 (+ 160us) server_negotiation.cc:226] Negotiated authn=SASL
0810 04:42:01.087307 (+ 555us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 04:42:01.087311 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 04:42:01.087311 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 04:42:01.087316 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0810 04:42:01.087362 (+ 46us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 04:42:01.087376 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 04:42:01.087461 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0810 04:42:01.087533 (+ 72us) server_negotiation.cc:299] Negotiation successful
0810 04:42:01.087588 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":144,"thread_start_us":75,"threads_started":1}
I20260810 04:42:01.088120 7400 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9ed63600bcde42aa9cbf8e044666f74a: Neither blocks nor log segments found. Creating new log.
I20260810 04:42:01.088440 7400 log.cc:824] T 00000000000000000000000000000000 P 9ed63600bcde42aa9cbf8e044666f74a: Log is configured to *not* fsync() on all Append() calls
I20260810 04:42:01.089089 7400 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9ed63600bcde42aa9cbf8e044666f74a: No bootstrap required, opened a new log
I20260810 04:42:01.090508 7400 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9ed63600bcde42aa9cbf8e044666f74a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ed63600bcde42aa9cbf8e044666f74a" member_type: VOTER last_known_addr { host: "127.22.167.126" port: 41787 } }
I20260810 04:42:01.090644 7400 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9ed63600bcde42aa9cbf8e044666f74a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 04:42:01.090670 7400 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9ed63600bcde42aa9cbf8e044666f74a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9ed63600bcde42aa9cbf8e044666f74a, State: Initialized, Role: FOLLOWER
I20260810 04:42:01.090794 7400 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9ed63600bcde42aa9cbf8e044666f74a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ed63600bcde42aa9cbf8e044666f74a" member_type: VOTER last_known_addr { host: "127.22.167.126" port: 41787 } }
I20260810 04:42:01.090866 7400 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9ed63600bcde42aa9cbf8e044666f74a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 04:42:01.090901 7400 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9ed63600bcde42aa9cbf8e044666f74a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 04:42:01.090955 7400 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9ed63600bcde42aa9cbf8e044666f74a [term 0 FOLLOWER]: Advancing to term 1
I20260810 04:42:01.091481 7400 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9ed63600bcde42aa9cbf8e044666f74a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ed63600bcde42aa9cbf8e044666f74a" member_type: VOTER last_known_addr { host: "127.22.167.126" port: 41787 } }
I20260810 04:42:01.091581 7400 leader_election.cc:302] T 00000000000000000000000000000000 P 9ed63600bcde42aa9cbf8e044666f74a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9ed63600bcde42aa9cbf8e044666f74a; no voters:
I20260810 04:42:01.091737 7400 leader_election.cc:288] T 00000000000000000000000000000000 P 9ed63600bcde42aa9cbf8e044666f74a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 04:42:01.091974 7400 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9ed63600bcde42aa9cbf8e044666f74a [sys.catalog]: configured and running, proceeding with master startup.
I20260810 04:42:01.091998 7405 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9ed63600bcde42aa9cbf8e044666f74a [term 1 FOLLOWER]: Leader election won for term 1
I20260810 04:42:01.092119 7405 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9ed63600bcde42aa9cbf8e044666f74a [term 1 LEADER]: Becoming Leader. State: Replica: 9ed63600bcde42aa9cbf8e044666f74a, State: Running, Role: LEADER
I20260810 04:42:01.092233 7405 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9ed63600bcde42aa9cbf8e044666f74a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ed63600bcde42aa9cbf8e044666f74a" member_type: VOTER last_known_addr { host: "127.22.167.126" port: 41787 } }
I20260810 04:42:01.092592 7405 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9ed63600bcde42aa9cbf8e044666f74a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9ed63600bcde42aa9cbf8e044666f74a. Latest consensus state: current_term: 1 leader_uuid: "9ed63600bcde42aa9cbf8e044666f74a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ed63600bcde42aa9cbf8e044666f74a" member_type: VOTER last_known_addr { host: "127.22.167.126" port: 41787 } } }
I20260810 04:42:01.092694 7405 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9ed63600bcde42aa9cbf8e044666f74a [sys.catalog]: This master's current role is: LEADER
I20260810 04:42:01.092701 7406 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9ed63600bcde42aa9cbf8e044666f74a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9ed63600bcde42aa9cbf8e044666f74a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ed63600bcde42aa9cbf8e044666f74a" member_type: VOTER last_known_addr { host: "127.22.167.126" port: 41787 } } }
I20260810 04:42:01.092824 7406 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9ed63600bcde42aa9cbf8e044666f74a [sys.catalog]: This master's current role is: LEADER
I20260810 04:42:01.093307 7422 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 04:42:01.093717 7422 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 04:42:01.095175 7422 catalog_manager.cc:1347] Generated new cluster ID: 1a9cd41b1f3946b8805ae647a0736803
I20260810 04:42:01.095239 7422 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 04:42:01.103271 7422 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 04:42:01.103421 7422 catalog_manager.cc:1504] Loading token signing keys...
I20260810 04:42:01.112207 7422 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9ed63600bcde42aa9cbf8e044666f74a: Generated new TSK 0
I20260810 04:42:01.112373 7422 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 04:42:01.121888 23197 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskemv_AM/build/release/bin/kudu
/tmp/dist-test-taskemv_AM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786336863388938-23197-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786336863388938-23197-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786336863388938-23197-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786336863388938-23197-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.167.65:0
--local_ip_for_outbound_sockets=127.22.167.65
--webserver_interface=127.22.167.65
--webserver_port=0
--tserver_master_addrs=127.22.167.126:41787
--builtin_ntp_servers=127.22.167.84:37289
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260810 04:42:01.203768 7439 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 04:42:01.203991 7439 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 04:42:01.204015 7439 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 04:42:01.205210 7439 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 04:42:01.205268 7439 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 04:42:01.205294 7439 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.167.65
I20260810 04:42:01.206686 7439 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.167.84:37289
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786336863388938-23197-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786336863388938-23197-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.167.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786336863388938-23197-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.22.167.65
--webserver_port=0
--tserver_master_addrs=127.22.167.126:41787
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7439
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.167.65
--log_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786336863388938-23197-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 03:54:01 UTC on ccycloud.quasar-icijmt.root.comops.site
build id 4577009
I20260810 04:42:01.206929 7439 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 04:42:01.207223 7439 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 04:42:01.212146 7452 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 04:42:01.212137 7450 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 04:42:01.212165 7448 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 04:42:01.212445 7439 server_base.cc:1034] running on GCE node
I20260810 04:42:01.212625 7439 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 04:42:01.212970 7439 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 04:42:01.214157 7439 hybrid_clock.cc:648] HybridClock initialized: now 1786336921214138 us; error 33 us; skew 500 ppm
I20260810 04:42:01.215405 7439 webserver.cc:460] Webserver started at http://127.22.167.65:44183/ using document root <none> and password file <none>
I20260810 04:42:01.215631 7439 fs_manager.cc:359] Metadata directory not provided
I20260810 04:42:01.215682 7439 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 04:42:01.215811 7439 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 04:42:01.216892 7439 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786336863388938-23197-0/minicluster-data/ts-0/data/instance:
uuid: "48f3f4fc925745d49d6ad832dc7c6ec3"
format_stamp: "Formatted at 2026-08-10 04:42:01 on dist-test-slave-bqcl"
server_key: "c3aca7f88b5ee615d92f41cc2045c9c7"
server_key_iv: "4abddcd035dd99ec7c0b4401a6f7b215"
server_key_version: "encryptionkey@0"
I20260810 04:42:01.217234 7439 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786336863388938-23197-0/minicluster-data/ts-0/wal/instance:
uuid: "48f3f4fc925745d49d6ad832dc7c6ec3"
format_stamp: "Formatted at 2026-08-10 04:42:01 on dist-test-slave-bqcl"
server_key: "c3aca7f88b5ee615d92f41cc2045c9c7"
server_key_iv: "4abddcd035dd99ec7c0b4401a6f7b215"
server_key_version: "encryptionkey@0"
I20260810 04:42:01.218528 7439 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260810 04:42:01.219190 7462 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 04:42:01.219357 7439 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260810 04:42:01.219441 7439 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786336863388938-23197-0/minicluster-data/ts-0/data,/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786336863388938-23197-0/minicluster-data/ts-0/wal
uuid: "48f3f4fc925745d49d6ad832dc7c6ec3"
format_stamp: "Formatted at 2026-08-10 04:42:01 on dist-test-slave-bqcl"
server_key: "c3aca7f88b5ee615d92f41cc2045c9c7"
server_key_iv: "4abddcd035dd99ec7c0b4401a6f7b215"
server_key_version: "encryptionkey@0"
I20260810 04:42:01.219501 7439 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786336863388938-23197-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786336863388938-23197-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786336863388938-23197-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 04:42:01.222848 7439 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 04:42:01.223104 7439 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 04:42:01.223235 7439 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 04:42:01.223429 7439 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 04:42:01.223729 7439 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 04:42:01.223764 7439 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 04:42:01.223793 7439 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 04:42:01.223816 7439 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 04:42:01.229456 7439 rpc_server.cc:307] RPC server started. Bound to: 127.22.167.65:39987
I20260810 04:42:01.229501 7661 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.167.65:39987 every 8 connection(s)
I20260810 04:42:01.229815 7439 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786336863388938-23197-0/minicluster-data/ts-0/data/info.pb
I20260810 04:42:01.232775 7403 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:42:01.230523 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.167.65:58197 (local address 127.22.167.126:41787)
0810 04:42:01.230600 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0810 04:42:01.230602 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 04:42:01.230843 (+ 241us) server_negotiation.cc:407] Connection header received
0810 04:42:01.231289 (+ 446us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:42:01.231290 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:42:01.231296 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:42:01.231345 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0810 04:42:01.232480 (+ 1135us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 04:42:01.232481 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 04:42:01.232482 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 04:42:01.232486 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0810 04:42:01.232503 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 04:42:01.232507 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 04:42:01.232565 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0810 04:42:01.232704 (+ 139us) server_negotiation.cc:299] Negotiation successful
0810 04:42:01.232718 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260810 04:42:01.232980 7669 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:42:01.230500 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.126:41787 (local address 127.22.167.65:58197)
0810 04:42:01.230730 (+ 230us) negotiation.cc:107] Waiting for socket to connect
0810 04:42:01.230764 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0810 04:42:01.231210 (+ 446us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 04:42:01.231404 (+ 194us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 04:42:01.231407 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 04:42:01.231415 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0810 04:42:01.232320 (+ 905us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 04:42:01.232321 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0810 04:42:01.232352 (+ 31us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 04:42:01.232389 (+ 37us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 04:42:01.232572 (+ 183us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 04:42:01.232577 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 04:42:01.232581 (+ 4us) client_negotiation.cc:769] Sending connection context
0810 04:42:01.232699 (+ 118us) client_negotiation.cc:240] Negotiation successful
0810 04:42:01.232764 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":95,"threads_started":1}
I20260810 04:42:01.233508 7663 heartbeater.cc:347] Connected to a master server at 127.22.167.126:41787
I20260810 04:42:01.233594 7663 heartbeater.cc:464] Registering TS with master...
I20260810 04:42:01.233798 7663 heartbeater.cc:511] Master 127.22.167.126:41787 requested a full tablet report, sending...
I20260810 04:42:01.234547 7339 ts_manager.cc:194] Registered new tserver with Master: 48f3f4fc925745d49d6ad832dc7c6ec3 (127.22.167.65:39987)
I20260810 04:42:01.237238 23197 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskemv_AM/build/release/bin/kudu as pid 7439
I20260810 04:42:01.237314 23197 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786336863388938-23197-0/minicluster-data/ts-0/wal/instance
I20260810 04:42:01.237463 23197 external_mini_cluster.cc:1428] Setting key e9868dd2a174cc3ff3056be60a6fe3ed
I20260810 04:42:01.238332 23197 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskemv_AM/build/release/bin/kudu
/tmp/dist-test-taskemv_AM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786336863388938-23197-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786336863388938-23197-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786336863388938-23197-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786336863388938-23197-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.167.66:0
--local_ip_for_outbound_sockets=127.22.167.66
--webserver_interface=127.22.167.66
--webserver_port=0
--tserver_master_addrs=127.22.167.126:41787
--builtin_ntp_servers=127.22.167.84:37289
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260810 04:42:01.320924 7670 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 04:42:01.321131 7670 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 04:42:01.321152 7670 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 04:42:01.322367 7670 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 04:42:01.322409 7670 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 04:42:01.322434 7670 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.167.66
I20260810 04:42:01.323810 7670 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.167.84:37289
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786336863388938-23197-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786336863388938-23197-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.167.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786336863388938-23197-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.22.167.66
--webserver_port=0
--tserver_master_addrs=127.22.167.126:41787
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7670
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.167.66
--log_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786336863388938-23197-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 03:54:01 UTC on ccycloud.quasar-icijmt.root.comops.site
build id 4577009
I20260810 04:42:01.324040 7670 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 04:42:01.324319 7670 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 04:42:01.329100 7683 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 04:42:01.329162 7680 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 04:42:01.329309 7670 server_base.cc:1034] running on GCE node
W20260810 04:42:01.329097 7678 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 04:42:01.329577 7670 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 04:42:01.329825 7670 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 04:42:01.330960 7670 hybrid_clock.cc:648] HybridClock initialized: now 1786336921330950 us; error 32 us; skew 500 ppm
I20260810 04:42:01.332157 7670 webserver.cc:460] Webserver started at http://127.22.167.66:39155/ using document root <none> and password file <none>
I20260810 04:42:01.332391 7670 fs_manager.cc:359] Metadata directory not provided
I20260810 04:42:01.332438 7670 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 04:42:01.332563 7670 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 04:42:01.333673 7670 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786336863388938-23197-0/minicluster-data/ts-1/data/instance:
uuid: "1595891147fd44ef9b8b59b20d17a05b"
format_stamp: "Formatted at 2026-08-10 04:42:01 on dist-test-slave-bqcl"
server_key: "5c4203daf10dce7ac882470b26bbd2f4"
server_key_iv: "6d031d1604acfd456682db70689e7b4d"
server_key_version: "encryptionkey@0"
I20260810 04:42:01.334012 7670 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786336863388938-23197-0/minicluster-data/ts-1/wal/instance:
uuid: "1595891147fd44ef9b8b59b20d17a05b"
format_stamp: "Formatted at 2026-08-10 04:42:01 on dist-test-slave-bqcl"
server_key: "5c4203daf10dce7ac882470b26bbd2f4"
server_key_iv: "6d031d1604acfd456682db70689e7b4d"
server_key_version: "encryptionkey@0"
I20260810 04:42:01.335227 7670 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260810 04:42:01.335882 7693 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 04:42:01.336037 7670 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260810 04:42:01.336120 7670 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786336863388938-23197-0/minicluster-data/ts-1/data,/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786336863388938-23197-0/minicluster-data/ts-1/wal
uuid: "1595891147fd44ef9b8b59b20d17a05b"
format_stamp: "Formatted at 2026-08-10 04:42:01 on dist-test-slave-bqcl"
server_key: "5c4203daf10dce7ac882470b26bbd2f4"
server_key_iv: "6d031d1604acfd456682db70689e7b4d"
server_key_version: "encryptionkey@0"
I20260810 04:42:01.336194 7670 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786336863388938-23197-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786336863388938-23197-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786336863388938-23197-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 04:42:01.339581 7670 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 04:42:01.339824 7670 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 04:42:01.339941 7670 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 04:42:01.340126 7670 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 04:42:01.340413 7670 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 04:42:01.340446 7670 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 04:42:01.340471 7670 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 04:42:01.340495 7670 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 04:42:01.345979 7670 rpc_server.cc:307] RPC server started. Bound to: 127.22.167.66:40139
I20260810 04:42:01.346019 7880 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.167.66:40139 every 8 connection(s)
I20260810 04:42:01.346352 7670 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786336863388938-23197-0/minicluster-data/ts-1/data/info.pb
I20260810 04:42:01.348991 7403 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:42:01.347029 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.167.66:57641 (local address 127.22.167.126:41787)
0810 04:42:01.347102 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0810 04:42:01.347105 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 04:42:01.347279 (+ 174us) server_negotiation.cc:407] Connection header received
0810 04:42:01.347730 (+ 451us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:42:01.347731 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:42:01.347736 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:42:01.347789 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0810 04:42:01.348812 (+ 1023us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 04:42:01.348813 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 04:42:01.348813 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 04:42:01.348817 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0810 04:42:01.348831 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 04:42:01.348835 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 04:42:01.348877 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0810 04:42:01.348927 (+ 50us) server_negotiation.cc:299] Negotiation successful
0810 04:42:01.348937 (+ 10us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260810 04:42:01.349146 7884 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:42:01.346983 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.126:41787 (local address 127.22.167.66:57641)
0810 04:42:01.347170 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0810 04:42:01.347203 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0810 04:42:01.347632 (+ 429us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 04:42:01.347843 (+ 211us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 04:42:01.347846 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 04:42:01.347854 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0810 04:42:01.348687 (+ 833us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 04:42:01.348687 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0810 04:42:01.348715 (+ 28us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 04:42:01.348741 (+ 26us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 04:42:01.348877 (+ 136us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 04:42:01.348879 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 04:42:01.348881 (+ 2us) client_negotiation.cc:769] Sending connection context
0810 04:42:01.348929 (+ 48us) client_negotiation.cc:240] Negotiation successful
0810 04:42:01.348974 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":71,"threads_started":1}
I20260810 04:42:01.349574 7882 heartbeater.cc:347] Connected to a master server at 127.22.167.126:41787
I20260810 04:42:01.349660 7882 heartbeater.cc:464] Registering TS with master...
I20260810 04:42:01.349797 7882 heartbeater.cc:511] Master 127.22.167.126:41787 requested a full tablet report, sending...
I20260810 04:42:01.350113 7339 ts_manager.cc:194] Registered new tserver with Master: 1595891147fd44ef9b8b59b20d17a05b (127.22.167.66:40139)
I20260810 04:42:01.353502 23197 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskemv_AM/build/release/bin/kudu as pid 7670
I20260810 04:42:01.353582 23197 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786336863388938-23197-0/minicluster-data/ts-1/wal/instance
I20260810 04:42:01.353729 23197 external_mini_cluster.cc:1428] Setting key 766829f0db27e450e2a86d210c91f8de
I20260810 04:42:01.354571 23197 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskemv_AM/build/release/bin/kudu
/tmp/dist-test-taskemv_AM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786336863388938-23197-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786336863388938-23197-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786336863388938-23197-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786336863388938-23197-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.167.67:0
--local_ip_for_outbound_sockets=127.22.167.67
--webserver_interface=127.22.167.67
--webserver_port=0
--tserver_master_addrs=127.22.167.126:41787
--builtin_ntp_servers=127.22.167.84:37289
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260810 04:42:01.434721 7885 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 04:42:01.434933 7885 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 04:42:01.434954 7885 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 04:42:01.436075 7885 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 04:42:01.436115 7885 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 04:42:01.436143 7885 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.167.67
I20260810 04:42:01.437449 7885 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.167.84:37289
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786336863388938-23197-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786336863388938-23197-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.167.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786336863388938-23197-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.22.167.67
--webserver_port=0
--tserver_master_addrs=127.22.167.126:41787
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7885
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.167.67
--log_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786336863388938-23197-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 03:54:01 UTC on ccycloud.quasar-icijmt.root.comops.site
build id 4577009
I20260810 04:42:01.437668 7885 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 04:42:01.437951 7885 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 04:42:01.442736 7898 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 04:42:01.442871 7885 server_base.cc:1034] running on GCE node
W20260810 04:42:01.442930 7893 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 04:42:01.442754 7892 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 04:42:01.444931 7885 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 04:42:01.445159 7885 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 04:42:01.446290 7885 hybrid_clock.cc:648] HybridClock initialized: now 1786336921446283 us; error 40 us; skew 500 ppm
I20260810 04:42:01.447522 7885 webserver.cc:460] Webserver started at http://127.22.167.67:45899/ using document root <none> and password file <none>
I20260810 04:42:01.447742 7885 fs_manager.cc:359] Metadata directory not provided
I20260810 04:42:01.447791 7885 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 04:42:01.447921 7885 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 04:42:01.448974 7885 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786336863388938-23197-0/minicluster-data/ts-2/data/instance:
uuid: "35b43d8b22fb4759a5c57b7729013a93"
format_stamp: "Formatted at 2026-08-10 04:42:01 on dist-test-slave-bqcl"
server_key: "39f8c651c9385caaea68f15f0f7be304"
server_key_iv: "0de0dbab194ee3e27608d73e10de0c11"
server_key_version: "encryptionkey@0"
I20260810 04:42:01.449309 7885 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786336863388938-23197-0/minicluster-data/ts-2/wal/instance:
uuid: "35b43d8b22fb4759a5c57b7729013a93"
format_stamp: "Formatted at 2026-08-10 04:42:01 on dist-test-slave-bqcl"
server_key: "39f8c651c9385caaea68f15f0f7be304"
server_key_iv: "0de0dbab194ee3e27608d73e10de0c11"
server_key_version: "encryptionkey@0"
I20260810 04:42:01.450569 7885 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260810 04:42:01.451224 7908 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 04:42:01.451370 7885 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260810 04:42:01.451453 7885 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786336863388938-23197-0/minicluster-data/ts-2/data,/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786336863388938-23197-0/minicluster-data/ts-2/wal
uuid: "35b43d8b22fb4759a5c57b7729013a93"
format_stamp: "Formatted at 2026-08-10 04:42:01 on dist-test-slave-bqcl"
server_key: "39f8c651c9385caaea68f15f0f7be304"
server_key_iv: "0de0dbab194ee3e27608d73e10de0c11"
server_key_version: "encryptionkey@0"
I20260810 04:42:01.451510 7885 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786336863388938-23197-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786336863388938-23197-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786336863388938-23197-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 04:42:01.454721 7885 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 04:42:01.454953 7885 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 04:42:01.455066 7885 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 04:42:01.455250 7885 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 04:42:01.455534 7885 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 04:42:01.455567 7885 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 04:42:01.455592 7885 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 04:42:01.455610 7885 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 04:42:01.460875 7885 rpc_server.cc:307] RPC server started. Bound to: 127.22.167.67:42231
I20260810 04:42:01.460929 8111 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.167.67:42231 every 8 connection(s)
I20260810 04:42:01.461195 7885 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786336863388938-23197-0/minicluster-data/ts-2/data/info.pb
I20260810 04:42:01.463940 7403 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:42:01.461841 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.167.67:54357 (local address 127.22.167.126:41787)
0810 04:42:01.461904 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0810 04:42:01.461906 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 04:42:01.462080 (+ 174us) server_negotiation.cc:407] Connection header received
0810 04:42:01.462551 (+ 471us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:42:01.462553 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:42:01.462559 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:42:01.462612 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0810 04:42:01.463705 (+ 1093us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 04:42:01.463707 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 04:42:01.463707 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 04:42:01.463711 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0810 04:42:01.463728 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 04:42:01.463732 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 04:42:01.463781 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0810 04:42:01.463857 (+ 76us) server_negotiation.cc:299] Negotiation successful
0810 04:42:01.463870 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260810 04:42:01.464124 8116 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:42:01.461803 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.126:41787 (local address 127.22.167.67:54357)
0810 04:42:01.461976 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0810 04:42:01.462007 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0810 04:42:01.462457 (+ 450us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 04:42:01.462691 (+ 234us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 04:42:01.462695 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 04:42:01.462702 (+ 7us) client_negotiation.cc:189] Negotiated authn=SASL
0810 04:42:01.463561 (+ 859us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 04:42:01.463561 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0810 04:42:01.463588 (+ 27us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 04:42:01.463614 (+ 26us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 04:42:01.463788 (+ 174us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 04:42:01.463794 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 04:42:01.463797 (+ 3us) client_negotiation.cc:769] Sending connection context
0810 04:42:01.463863 (+ 66us) client_negotiation.cc:240] Negotiation successful
0810 04:42:01.463924 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":78,"threads_started":1}
I20260810 04:42:01.464555 8112 heartbeater.cc:347] Connected to a master server at 127.22.167.126:41787
I20260810 04:42:01.464633 8112 heartbeater.cc:464] Registering TS with master...
I20260810 04:42:01.464773 8112 heartbeater.cc:511] Master 127.22.167.126:41787 requested a full tablet report, sending...
I20260810 04:42:01.465103 7339 ts_manager.cc:194] Registered new tserver with Master: 35b43d8b22fb4759a5c57b7729013a93 (127.22.167.67:42231)
I20260810 04:42:01.469715 23197 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskemv_AM/build/release/bin/kudu as pid 7885
I20260810 04:42:01.469798 23197 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786336863388938-23197-0/minicluster-data/ts-2/wal/instance
I20260810 04:42:01.469936 23197 external_mini_cluster.cc:1428] Setting key 13d2ec7be3127680c042db752551c92e
I20260810 04:42:01.470711 23197 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260810 04:42:01.472393 7403 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 04:42:01.471936 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33260 (local address 127.22.167.126:41787)
0810 04:42:01.471979 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0810 04:42:01.471981 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 04:42:01.472083 (+ 102us) server_negotiation.cc:407] Connection header received
0810 04:42:01.472126 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:42:01.472127 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:42:01.472130 (+ 3us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:42:01.472180 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0810 04:42:01.472329 (+ 149us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:33260: BlockingRecv error: recv got EOF from 127.0.0.1:33260 (error 108)
Metrics: {"server-negotiator.queue_time_us":22}
I20260810 04:42:01.472406 23197 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskemv_AM/build/release/bin/kudu with pid 7439
I20260810 04:42:01.496743 23197 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskemv_AM/build/release/bin/kudu with pid 7670
I20260810 04:42:01.509125 23197 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskemv_AM/build/release/bin/kudu with pid 7885
I20260810 04:42:01.522171 23197 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskemv_AM/build/release/bin/kudu with pid 7284
2026-08-10T04:42:01Z chronyd exiting
[ OK ] SecurityITest.TestRequireEncryptionInsecureCluster (572 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationSecureCluster
Loading random data
Initializing database '/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786336863388938-23197-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 04:42:01 dist-test-slave-bqcl krb5kdc[8127](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 04:42:01 dist-test-slave-bqcl krb5kdc[8127](info): set up 2 sockets
Aug 10 04:42:01 dist-test-slave-bqcl krb5kdc[8127](info): commencing operation
krb5kdc: starting...
W20260810 04:42:02.685899 23197 mini_kdc.cc:121] Time spent starting KDC: real 1.140s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786336863388938-23197-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786336863388938-23197-0/krb5kdc/test-user.keytab.
Aug 10 04:42:02 dist-test-slave-bqcl krb5kdc[8127](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786336922, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T04:42:02Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T04:42:02Z Disabled control of system clock
WARNING: no policy specified for kudu/127.22.167.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.167.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.167.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786336863388938-23197-0/krb5kdc/kudu_127.22.167.126.keytab.
Entry for principal kudu/127.22.167.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786336863388938-23197-0/krb5kdc/kudu_127.22.167.126.keytab.
I20260810 04:42:02.802925 23197 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskemv_AM/build/release/bin/kudu
/tmp/dist-test-taskemv_AM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786336863388938-23197-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786336863388938-23197-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786336863388938-23197-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786336863388938-23197-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.167.126:40437
--webserver_interface=127.22.167.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786336863388938-23197-0/krb5kdc/kudu_127.22.167.126.keytab
--principal=kudu/127.22.167.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.167.84:39609
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.167.126:40437
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786336863388938-23197-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786336863388938-23197-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786336863388938-23197-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 04:42:02.883600 8154 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 04:42:02.883800 8154 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 04:42:02.883829 8154 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 04:42:02.884903 8154 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 04:42:02.884943 8154 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 04:42:02.884961 8154 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 04:42:02.884977 8154 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 04:42:02.884992 8154 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 04:42:02.886448 8154 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.167.84:39609
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786336863388938-23197-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786336863388938-23197-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.22.167.126:40437
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786336863388938-23197-0/krb5kdc/kudu_127.22.167.126.keytab
--principal=kudu/127.22.167.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.167.126:40437
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786336863388938-23197-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.167.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8154
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786336863388938-23197-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 03:54:01 UTC on ccycloud.quasar-icijmt.root.comops.site
build id 4577009
I20260810 04:42:02.886679 8154 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 04:42:02.886886 8154 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 04:42:02.891332 8165 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 04:42:02.891340 8167 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 04:42:02.891567 8161 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 04:42:02.891570 8154 server_base.cc:1034] running on GCE node
I20260810 04:42:02.891848 8154 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 04:42:02.892123 8154 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 04:42:02.893260 8154 hybrid_clock.cc:648] HybridClock initialized: now 1786336922893245 us; error 29 us; skew 500 ppm
Aug 10 04:42:02 dist-test-slave-bqcl krb5kdc[8127](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786336922, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.167.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 04:42:02.895718 8154 init.cc:377] Logged in from keytab as kudu/127.22.167.126@KRBTEST.COM (short username kudu)
I20260810 04:42:02.896126 8154 webserver.cc:460] Webserver started at http://127.22.167.126:40449/ using document root <none> and password file <none>
I20260810 04:42:02.896315 8154 fs_manager.cc:359] Metadata directory not provided
I20260810 04:42:02.896354 8154 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 04:42:02.896443 8154 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 04:42:02.897596 8154 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786336863388938-23197-0/minicluster-data/master-0/data/instance:
uuid: "04541f691a694152b004dfc4275e5647"
format_stamp: "Formatted at 2026-08-10 04:42:02 on dist-test-slave-bqcl"
server_key: "de648e3307b011e266c405ebea346b8f"
server_key_iv: "30decacd9edd1aa5614b838876ade3e9"
server_key_version: "encryptionkey@0"
I20260810 04:42:02.897933 8154 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786336863388938-23197-0/minicluster-data/master-0/wal/instance:
uuid: "04541f691a694152b004dfc4275e5647"
format_stamp: "Formatted at 2026-08-10 04:42:02 on dist-test-slave-bqcl"
server_key: "de648e3307b011e266c405ebea346b8f"
server_key_iv: "30decacd9edd1aa5614b838876ade3e9"
server_key_version: "encryptionkey@0"
I20260810 04:42:02.899155 8154 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260810 04:42:02.899808 8179 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 04:42:02.899991 8154 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260810 04:42:02.900081 8154 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786336863388938-23197-0/minicluster-data/master-0/data,/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786336863388938-23197-0/minicluster-data/master-0/wal
uuid: "04541f691a694152b004dfc4275e5647"
format_stamp: "Formatted at 2026-08-10 04:42:02 on dist-test-slave-bqcl"
server_key: "de648e3307b011e266c405ebea346b8f"
server_key_iv: "30decacd9edd1aa5614b838876ade3e9"
server_key_version: "encryptionkey@0"
I20260810 04:42:02.900149 8154 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786336863388938-23197-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786336863388938-23197-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786336863388938-23197-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 04:42:02.917129 8154 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 04:42:02.919216 8154 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 04:42:02.919348 8154 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 04:42:02.923194 8154 rpc_server.cc:307] RPC server started. Bound to: 127.22.167.126:40437
I20260810 04:42:02.923259 8252 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.167.126:40437 every 8 connection(s)
I20260810 04:42:02.923547 8154 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786336863388938-23197-0/minicluster-data/master-0/data/info.pb
I20260810 04:42:02.924468 8254 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 04:42:02.926671 8254 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 04541f691a694152b004dfc4275e5647: Bootstrap starting.
I20260810 04:42:02.927253 8254 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 04541f691a694152b004dfc4275e5647: Neither blocks nor log segments found. Creating new log.
I20260810 04:42:02.927260 23197 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskemv_AM/build/release/bin/kudu as pid 8154
I20260810 04:42:02.927349 23197 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786336863388938-23197-0/minicluster-data/master-0/wal/instance
I20260810 04:42:02.927547 23197 external_mini_cluster.cc:1428] Setting key f44ea4192d9a3bc84cee2fc1c01e41a5
I20260810 04:42:02.927591 8254 log.cc:824] T 00000000000000000000000000000000 P 04541f691a694152b004dfc4275e5647: Log is configured to *not* fsync() on all Append() calls
I20260810 04:42:02.928475 8254 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 04541f691a694152b004dfc4275e5647: No bootstrap required, opened a new log
I20260810 04:42:02.930810 8254 raft_consensus.cc:348] T 00000000000000000000000000000000 P 04541f691a694152b004dfc4275e5647 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "04541f691a694152b004dfc4275e5647" member_type: VOTER last_known_addr { host: "127.22.167.126" port: 40437 } }
I20260810 04:42:02.930970 8254 raft_consensus.cc:374] T 00000000000000000000000000000000 P 04541f691a694152b004dfc4275e5647 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 04:42:02.931013 8254 raft_consensus.cc:729] T 00000000000000000000000000000000 P 04541f691a694152b004dfc4275e5647 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 04541f691a694152b004dfc4275e5647, State: Initialized, Role: FOLLOWER
I20260810 04:42:02.931173 8254 consensus_queue.cc:260] T 00000000000000000000000000000000 P 04541f691a694152b004dfc4275e5647 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "04541f691a694152b004dfc4275e5647" member_type: VOTER last_known_addr { host: "127.22.167.126" port: 40437 } }
I20260810 04:42:02.931263 8254 raft_consensus.cc:388] T 00000000000000000000000000000000 P 04541f691a694152b004dfc4275e5647 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 04:42:02.931298 8254 raft_consensus.cc:482] T 00000000000000000000000000000000 P 04541f691a694152b004dfc4275e5647 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 04:42:02.931344 8254 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 04541f691a694152b004dfc4275e5647 [term 0 FOLLOWER]: Advancing to term 1
I20260810 04:42:02.931897 8254 raft_consensus.cc:504] T 00000000000000000000000000000000 P 04541f691a694152b004dfc4275e5647 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "04541f691a694152b004dfc4275e5647" member_type: VOTER last_known_addr { host: "127.22.167.126" port: 40437 } }
I20260810 04:42:02.931999 8254 leader_election.cc:302] T 00000000000000000000000000000000 P 04541f691a694152b004dfc4275e5647 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 04541f691a694152b004dfc4275e5647; no voters:
I20260810 04:42:02.932154 8254 leader_election.cc:288] T 00000000000000000000000000000000 P 04541f691a694152b004dfc4275e5647 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 04:42:02.932202 8265 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 04541f691a694152b004dfc4275e5647 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 04:42:02.932348 8265 raft_consensus.cc:686] T 00000000000000000000000000000000 P 04541f691a694152b004dfc4275e5647 [term 1 LEADER]: Becoming Leader. State: Replica: 04541f691a694152b004dfc4275e5647, State: Running, Role: LEADER
I20260810 04:42:02.932410 8254 sys_catalog.cc:564] T 00000000000000000000000000000000 P 04541f691a694152b004dfc4275e5647 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 04:42:02.932477 8265 consensus_queue.cc:237] T 00000000000000000000000000000000 P 04541f691a694152b004dfc4275e5647 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "04541f691a694152b004dfc4275e5647" member_type: VOTER last_known_addr { host: "127.22.167.126" port: 40437 } }
I20260810 04:42:02.932811 8265 sys_catalog.cc:455] T 00000000000000000000000000000000 P 04541f691a694152b004dfc4275e5647 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 04541f691a694152b004dfc4275e5647. Latest consensus state: current_term: 1 leader_uuid: "04541f691a694152b004dfc4275e5647" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "04541f691a694152b004dfc4275e5647" member_type: VOTER last_known_addr { host: "127.22.167.126" port: 40437 } } }
I20260810 04:42:02.932869 8267 sys_catalog.cc:455] T 00000000000000000000000000000000 P 04541f691a694152b004dfc4275e5647 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "04541f691a694152b004dfc4275e5647" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "04541f691a694152b004dfc4275e5647" member_type: VOTER last_known_addr { host: "127.22.167.126" port: 40437 } } }
I20260810 04:42:02.932967 8265 sys_catalog.cc:458] T 00000000000000000000000000000000 P 04541f691a694152b004dfc4275e5647 [sys.catalog]: This master's current role is: LEADER
I20260810 04:42:02.933032 8267 sys_catalog.cc:458] T 00000000000000000000000000000000 P 04541f691a694152b004dfc4275e5647 [sys.catalog]: This master's current role is: LEADER
I20260810 04:42:02.933336 8278 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 04:42:02.934021 8278 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Aug 10 04:42:02 dist-test-slave-bqcl krb5kdc[8127](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786336922, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.167.126@KRBTEST.COM
I20260810 04:42:02.935662 8278 catalog_manager.cc:1347] Generated new cluster ID: 333d3e7a2cf040e89057a459fcb3a5e6
I20260810 04:42:02.935719 8278 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 04:42:02.938303 8261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:42:02.928042 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58018 (local address 127.22.167.126:40437)
0810 04:42:02.928209 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0810 04:42:02.928219 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 04:42:02.928254 (+ 35us) server_negotiation.cc:407] Connection header received
0810 04:42:02.928804 (+ 550us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:42:02.928813 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:42:02.928873 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:42:02.929030 (+ 157us) server_negotiation.cc:226] Negotiated authn=SASL
0810 04:42:02.929577 (+ 547us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:42:02.930983 (+ 1406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:42:02.931808 (+ 825us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:42:02.932418 (+ 610us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:42:02.932852 (+ 434us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:42:02.935413 (+ 2561us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 04:42:02.935417 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 04:42:02.935418 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 04:42:02.935429 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0810 04:42:02.936925 (+ 1496us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 04:42:02.937237 (+ 312us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 04:42:02.937239 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 04:42:02.937240 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 04:42:02.937325 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 04:42:02.937495 (+ 170us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 04:42:02.937496 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 04:42:02.937496 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 04:42:02.937724 (+ 228us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 04:42:02.937881 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0810 04:42:02.937973 (+ 92us) server_negotiation.cc:299] Negotiation successful
0810 04:42:02.938103 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":111,"thread_start_us":72,"threads_started":1}
I20260810 04:42:02.949822 8278 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 04:42:02.950555 8278 catalog_manager.cc:1504] Loading token signing keys...
I20260810 04:42:02.956176 8278 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 04541f691a694152b004dfc4275e5647: Generated new TSK 0
I20260810 04:42:02.956346 8278 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.22.167.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.167.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.167.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786336863388938-23197-0/krb5kdc/kudu_127.22.167.65.keytab.
Entry for principal kudu/127.22.167.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786336863388938-23197-0/krb5kdc/kudu_127.22.167.65.keytab.
I20260810 04:42:02.988493 23197 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskemv_AM/build/release/bin/kudu
/tmp/dist-test-taskemv_AM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786336863388938-23197-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786336863388938-23197-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786336863388938-23197-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786336863388938-23197-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.167.65:0
--local_ip_for_outbound_sockets=127.22.167.65
--webserver_interface=127.22.167.65
--webserver_port=0
--tserver_master_addrs=127.22.167.126:40437
--keytab_file=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786336863388938-23197-0/krb5kdc/kudu_127.22.167.65.keytab
--principal=kudu/127.22.167.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.167.84:39609
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786336863388938-23197-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786336863388938-23197-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786336863388938-23197-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 04:42:03.069898 8301 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 04:42:03.070102 8301 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 04:42:03.070144 8301 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 04:42:03.071264 8301 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 04:42:03.071303 8301 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 04:42:03.071331 8301 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.167.65
I20260810 04:42:03.072758 8301 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.167.84:39609
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786336863388938-23197-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786336863388938-23197-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786336863388938-23197-0/krb5kdc/kudu_127.22.167.65.keytab
--principal=kudu/127.22.167.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.167.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786336863388938-23197-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.22.167.65
--webserver_port=0
--tserver_master_addrs=127.22.167.126:40437
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8301
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.167.65
--log_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786336863388938-23197-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 03:54:01 UTC on ccycloud.quasar-icijmt.root.comops.site
build id 4577009
I20260810 04:42:03.072984 8301 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 04:42:03.073231 8301 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260810 04:42:03.078101 8301 server_base.cc:1034] running on GCE node
W20260810 04:42:03.078038 8310 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 04:42:03.078055 8311 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 04:42:03.078016 8314 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 04:42:03.078444 8301 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 04:42:03.078687 8301 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 04:42:03.079823 8301 hybrid_clock.cc:648] HybridClock initialized: now 1786336923079810 us; error 29 us; skew 500 ppm
Aug 10 04:42:03 dist-test-slave-bqcl krb5kdc[8127](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786336923, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.167.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 04:42:03.082249 8301 init.cc:377] Logged in from keytab as kudu/127.22.167.65@KRBTEST.COM (short username kudu)
I20260810 04:42:03.082693 8301 webserver.cc:460] Webserver started at http://127.22.167.65:45591/ using document root <none> and password file <none>
I20260810 04:42:03.082896 8301 fs_manager.cc:359] Metadata directory not provided
I20260810 04:42:03.082940 8301 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 04:42:03.083068 8301 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 04:42:03.084106 8301 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786336863388938-23197-0/minicluster-data/ts-0/data/instance:
uuid: "918ee8dee798468cb38c285c74c97974"
format_stamp: "Formatted at 2026-08-10 04:42:03 on dist-test-slave-bqcl"
server_key: "f441208be68fb328dc4291b09a8a2600"
server_key_iv: "bb408614a5e6c6cd97780afdbfdef633"
server_key_version: "encryptionkey@0"
I20260810 04:42:03.084441 8301 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786336863388938-23197-0/minicluster-data/ts-0/wal/instance:
uuid: "918ee8dee798468cb38c285c74c97974"
format_stamp: "Formatted at 2026-08-10 04:42:03 on dist-test-slave-bqcl"
server_key: "f441208be68fb328dc4291b09a8a2600"
server_key_iv: "bb408614a5e6c6cd97780afdbfdef633"
server_key_version: "encryptionkey@0"
I20260810 04:42:03.085675 8301 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260810 04:42:03.086371 8328 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 04:42:03.086511 8301 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260810 04:42:03.086591 8301 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786336863388938-23197-0/minicluster-data/ts-0/data,/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786336863388938-23197-0/minicluster-data/ts-0/wal
uuid: "918ee8dee798468cb38c285c74c97974"
format_stamp: "Formatted at 2026-08-10 04:42:03 on dist-test-slave-bqcl"
server_key: "f441208be68fb328dc4291b09a8a2600"
server_key_iv: "bb408614a5e6c6cd97780afdbfdef633"
server_key_version: "encryptionkey@0"
I20260810 04:42:03.086670 8301 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786336863388938-23197-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786336863388938-23197-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786336863388938-23197-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 04:42:03.104101 8301 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 04:42:03.105782 8301 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 04:42:03.105903 8301 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 04:42:03.106105 8301 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 04:42:03.106431 8301 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 04:42:03.106464 8301 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 04:42:03.106487 8301 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 04:42:03.106510 8301 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 04:42:03.112023 8301 rpc_server.cc:307] RPC server started. Bound to: 127.22.167.65:37805
I20260810 04:42:03.112078 8498 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.167.65:37805 every 8 connection(s)
I20260810 04:42:03.112375 8301 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786336863388938-23197-0/minicluster-data/ts-0/data/info.pb
I20260810 04:42:03.113754 23197 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskemv_AM/build/release/bin/kudu as pid 8301
I20260810 04:42:03.113843 23197 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786336863388938-23197-0/minicluster-data/ts-0/wal/instance
I20260810 04:42:03.114041 23197 external_mini_cluster.cc:1428] Setting key de6b0aa1cca59902f668bb9ab0a00c2a
Aug 10 04:42:03 dist-test-slave-bqcl krb5kdc[8127](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786336923, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.167.65@KRBTEST.COM for kudu/127.22.167.126@KRBTEST.COM
I20260810 04:42:03.122287 8261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:42:03.113008 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.167.65:57085 (local address 127.22.167.126:40437)
0810 04:42:03.113068 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0810 04:42:03.113070 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 04:42:03.113253 (+ 183us) server_negotiation.cc:407] Connection header received
0810 04:42:03.113759 (+ 506us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:42:03.113760 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:42:03.113774 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:42:03.113829 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0810 04:42:03.115010 (+ 1181us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:42:03.115494 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:42:03.116742 (+ 1248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:42:03.117062 (+ 320us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:42:03.117125 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:42:03.119643 (+ 2518us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 04:42:03.119645 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 04:42:03.119645 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 04:42:03.119652 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 04:42:03.120913 (+ 1261us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 04:42:03.121368 (+ 455us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 04:42:03.121369 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 04:42:03.121369 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 04:42:03.121415 (+ 46us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 04:42:03.121655 (+ 240us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 04:42:03.121656 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 04:42:03.121656 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 04:42:03.121758 (+ 102us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 04:42:03.121802 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0810 04:42:03.122108 (+ 306us) server_negotiation.cc:299] Negotiation successful
0810 04:42:03.122216 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260810 04:42:03.122526 8504 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:42:03.112986 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.126:40437 (local address 127.22.167.65:57085)
0810 04:42:03.113147 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0810 04:42:03.113180 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0810 04:42:03.113649 (+ 469us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 04:42:03.113942 (+ 293us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 04:42:03.113947 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 04:42:03.114614 (+ 667us) client_negotiation.cc:189] Negotiated authn=SASL
0810 04:42:03.114909 (+ 295us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:42:03.114916 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:42:03.115582 (+ 666us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:42:03.115583 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:42:03.116656 (+ 1073us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:42:03.116667 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:42:03.117138 (+ 471us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:42:03.117139 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:42:03.117262 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:42:03.117787 (+ 525us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 04:42:03.117795 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0810 04:42:03.119553 (+ 1758us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 04:42:03.120991 (+ 1438us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 04:42:03.120998 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 04:42:03.121000 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 04:42:03.121297 (+ 297us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 04:42:03.121465 (+ 168us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 04:42:03.121466 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 04:42:03.121466 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 04:42:03.121600 (+ 134us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 04:42:03.121812 (+ 212us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 04:42:03.121818 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 04:42:03.121981 (+ 163us) client_negotiation.cc:769] Sending connection context
0810 04:42:03.122100 (+ 119us) client_negotiation.cc:240] Negotiation successful
0810 04:42:03.122298 (+ 198us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":87,"threads_started":1}
I20260810 04:42:03.123014 8501 heartbeater.cc:347] Connected to a master server at 127.22.167.126:40437
I20260810 04:42:03.123111 8501 heartbeater.cc:464] Registering TS with master...
I20260810 04:42:03.123281 8501 heartbeater.cc:511] Master 127.22.167.126:40437 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.22.167.66@KRBTEST.COM; defaulting to no policy
I20260810 04:42:03.123800 8203 ts_manager.cc:194] Registered new tserver with Master: 918ee8dee798468cb38c285c74c97974 (127.22.167.65:37805)
I20260810 04:42:03.124356 8203 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.167.65@KRBTEST.COM'} at 127.22.167.65:57085
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.167.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.167.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786336863388938-23197-0/krb5kdc/kudu_127.22.167.66.keytab.
Entry for principal kudu/127.22.167.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786336863388938-23197-0/krb5kdc/kudu_127.22.167.66.keytab.
I20260810 04:42:03.140914 23197 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskemv_AM/build/release/bin/kudu
/tmp/dist-test-taskemv_AM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786336863388938-23197-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786336863388938-23197-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786336863388938-23197-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786336863388938-23197-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.167.66:0
--local_ip_for_outbound_sockets=127.22.167.66
--webserver_interface=127.22.167.66
--webserver_port=0
--tserver_master_addrs=127.22.167.126:40437
--keytab_file=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786336863388938-23197-0/krb5kdc/kudu_127.22.167.66.keytab
--principal=kudu/127.22.167.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.167.84:39609
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786336863388938-23197-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786336863388938-23197-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786336863388938-23197-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 04:42:03.221020 8508 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 04:42:03.221215 8508 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 04:42:03.221246 8508 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 04:42:03.222369 8508 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 04:42:03.222410 8508 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 04:42:03.222445 8508 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.167.66
I20260810 04:42:03.223857 8508 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.167.84:39609
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786336863388938-23197-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786336863388938-23197-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786336863388938-23197-0/krb5kdc/kudu_127.22.167.66.keytab
--principal=kudu/127.22.167.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.167.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786336863388938-23197-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.22.167.66
--webserver_port=0
--tserver_master_addrs=127.22.167.126:40437
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8508
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.167.66
--log_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786336863388938-23197-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 03:54:01 UTC on ccycloud.quasar-icijmt.root.comops.site
build id 4577009
I20260810 04:42:03.224084 8508 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 04:42:03.224295 8508 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 04:42:03.229094 8519 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 04:42:03.229140 8508 server_base.cc:1034] running on GCE node
W20260810 04:42:03.229210 8518 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 04:42:03.229108 8527 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 04:42:03.229463 8508 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 04:42:03.229696 8508 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 04:42:03.230839 8508 hybrid_clock.cc:648] HybridClock initialized: now 1786336923230829 us; error 27 us; skew 500 ppm
Aug 10 04:42:03 dist-test-slave-bqcl krb5kdc[8127](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786336923, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.167.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 04:42:03.233510 8508 init.cc:377] Logged in from keytab as kudu/127.22.167.66@KRBTEST.COM (short username kudu)
I20260810 04:42:03.233942 8508 webserver.cc:460] Webserver started at http://127.22.167.66:32923/ using document root <none> and password file <none>
I20260810 04:42:03.234213 8508 fs_manager.cc:359] Metadata directory not provided
I20260810 04:42:03.234266 8508 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 04:42:03.234398 8508 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 04:42:03.235416 8508 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786336863388938-23197-0/minicluster-data/ts-1/data/instance:
uuid: "0b7c258d17b34723a0134a7c33273a10"
format_stamp: "Formatted at 2026-08-10 04:42:03 on dist-test-slave-bqcl"
server_key: "1e34822830ef17179c717cda2d7cf51b"
server_key_iv: "7d30a95b039a908e78c4afe494cb3b4f"
server_key_version: "encryptionkey@0"
I20260810 04:42:03.235759 8508 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786336863388938-23197-0/minicluster-data/ts-1/wal/instance:
uuid: "0b7c258d17b34723a0134a7c33273a10"
format_stamp: "Formatted at 2026-08-10 04:42:03 on dist-test-slave-bqcl"
server_key: "1e34822830ef17179c717cda2d7cf51b"
server_key_iv: "7d30a95b039a908e78c4afe494cb3b4f"
server_key_version: "encryptionkey@0"
I20260810 04:42:03.236970 8508 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260810 04:42:03.237620 8539 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 04:42:03.237761 8508 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260810 04:42:03.237841 8508 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786336863388938-23197-0/minicluster-data/ts-1/data,/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786336863388938-23197-0/minicluster-data/ts-1/wal
uuid: "0b7c258d17b34723a0134a7c33273a10"
format_stamp: "Formatted at 2026-08-10 04:42:03 on dist-test-slave-bqcl"
server_key: "1e34822830ef17179c717cda2d7cf51b"
server_key_iv: "7d30a95b039a908e78c4afe494cb3b4f"
server_key_version: "encryptionkey@0"
I20260810 04:42:03.237906 8508 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786336863388938-23197-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786336863388938-23197-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786336863388938-23197-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 04:42:03.253811 8508 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 04:42:03.255975 8508 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 04:42:03.256112 8508 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 04:42:03.256340 8508 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 04:42:03.256634 8508 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 04:42:03.256667 8508 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 04:42:03.256696 8508 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 04:42:03.256714 8508 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 04:42:03.262219 8508 rpc_server.cc:307] RPC server started. Bound to: 127.22.167.66:33781
I20260810 04:42:03.262269 8731 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.167.66:33781 every 8 connection(s)
I20260810 04:42:03.262578 8508 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786336863388938-23197-0/minicluster-data/ts-1/data/info.pb
I20260810 04:42:03.265929 23197 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskemv_AM/build/release/bin/kudu as pid 8508
I20260810 04:42:03.266027 23197 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786336863388938-23197-0/minicluster-data/ts-1/wal/instance
I20260810 04:42:03.266260 23197 external_mini_cluster.cc:1428] Setting key 341ea8021ac53d3db65b56f00756df31
Aug 10 04:42:03 dist-test-slave-bqcl krb5kdc[8127](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786336923, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.167.66@KRBTEST.COM for kudu/127.22.167.126@KRBTEST.COM
I20260810 04:42:03.271786 8261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:42:03.263271 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.167.66:48969 (local address 127.22.167.126:40437)
0810 04:42:03.263335 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0810 04:42:03.263338 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 04:42:03.263519 (+ 181us) server_negotiation.cc:407] Connection header received
0810 04:42:03.263981 (+ 462us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:42:03.263982 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:42:03.263992 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:42:03.264040 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0810 04:42:03.264928 (+ 888us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:42:03.265448 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:42:03.266468 (+ 1020us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:42:03.266759 (+ 291us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:42:03.266827 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:42:03.269031 (+ 2204us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 04:42:03.269032 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 04:42:03.269033 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 04:42:03.269041 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0810 04:42:03.270347 (+ 1306us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 04:42:03.270836 (+ 489us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 04:42:03.270837 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 04:42:03.270837 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 04:42:03.270885 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 04:42:03.271168 (+ 283us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 04:42:03.271168 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 04:42:03.271169 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 04:42:03.271274 (+ 105us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 04:42:03.271330 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0810 04:42:03.271603 (+ 273us) server_negotiation.cc:299] Negotiation successful
0810 04:42:03.271691 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260810 04:42:03.272001 8736 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:42:03.263230 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.126:40437 (local address 127.22.167.66:48969)
0810 04:42:03.263402 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0810 04:42:03.263439 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0810 04:42:03.263905 (+ 466us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 04:42:03.264105 (+ 200us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 04:42:03.264108 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 04:42:03.264664 (+ 556us) client_negotiation.cc:189] Negotiated authn=SASL
0810 04:42:03.264861 (+ 197us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:42:03.264866 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:42:03.265513 (+ 647us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:42:03.265513 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:42:03.266397 (+ 884us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:42:03.266400 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:42:03.266884 (+ 484us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:42:03.266885 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:42:03.266997 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:42:03.267457 (+ 460us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 04:42:03.267465 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0810 04:42:03.268939 (+ 1474us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 04:42:03.270419 (+ 1480us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 04:42:03.270427 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 04:42:03.270429 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 04:42:03.270726 (+ 297us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 04:42:03.270946 (+ 220us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 04:42:03.270947 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 04:42:03.270947 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 04:42:03.271108 (+ 161us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 04:42:03.271342 (+ 234us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 04:42:03.271349 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 04:42:03.271479 (+ 130us) client_negotiation.cc:769] Sending connection context
0810 04:42:03.271572 (+ 93us) client_negotiation.cc:240] Negotiation successful
0810 04:42:03.271751 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":91,"threads_started":1}
I20260810 04:42:03.273315 8734 heartbeater.cc:347] Connected to a master server at 127.22.167.126:40437
I20260810 04:42:03.273406 8734 heartbeater.cc:464] Registering TS with master...
I20260810 04:42:03.273592 8734 heartbeater.cc:511] Master 127.22.167.126:40437 requested a full tablet report, sending...
I20260810 04:42:03.274003 8203 ts_manager.cc:194] Registered new tserver with Master: 0b7c258d17b34723a0134a7c33273a10 (127.22.167.66:33781)
I20260810 04:42:03.274538 8203 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.167.66@KRBTEST.COM'} at 127.22.167.66:48969
WARNING: no policy specified for kudu/127.22.167.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.167.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.167.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786336863388938-23197-0/krb5kdc/kudu_127.22.167.67.keytab.
Entry for principal kudu/127.22.167.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786336863388938-23197-0/krb5kdc/kudu_127.22.167.67.keytab.
I20260810 04:42:03.293020 23197 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskemv_AM/build/release/bin/kudu
/tmp/dist-test-taskemv_AM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786336863388938-23197-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786336863388938-23197-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786336863388938-23197-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786336863388938-23197-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.167.67:0
--local_ip_for_outbound_sockets=127.22.167.67
--webserver_interface=127.22.167.67
--webserver_port=0
--tserver_master_addrs=127.22.167.126:40437
--keytab_file=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786336863388938-23197-0/krb5kdc/kudu_127.22.167.67.keytab
--principal=kudu/127.22.167.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.167.84:39609
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786336863388938-23197-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786336863388938-23197-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786336863388938-23197-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 04:42:03.374868 8743 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 04:42:03.375058 8743 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 04:42:03.375079 8743 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 04:42:03.376140 8743 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 04:42:03.376178 8743 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 04:42:03.376201 8743 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.167.67
I20260810 04:42:03.377557 8743 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.167.84:39609
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786336863388938-23197-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786336863388938-23197-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786336863388938-23197-0/krb5kdc/kudu_127.22.167.67.keytab
--principal=kudu/127.22.167.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.167.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786336863388938-23197-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.22.167.67
--webserver_port=0
--tserver_master_addrs=127.22.167.126:40437
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8743
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.167.67
--log_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786336863388938-23197-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 03:54:01 UTC on ccycloud.quasar-icijmt.root.comops.site
build id 4577009
I20260810 04:42:03.377774 8743 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 04:42:03.377962 8743 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 04:42:03.382570 8760 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 04:42:03.382694 8759 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 04:42:03.382602 8762 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 04:42:03.382721 8743 server_base.cc:1034] running on GCE node
I20260810 04:42:03.383014 8743 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 04:42:03.383241 8743 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 04:42:03.384375 8743 hybrid_clock.cc:648] HybridClock initialized: now 1786336923384362 us; error 30 us; skew 500 ppm
Aug 10 04:42:03 dist-test-slave-bqcl krb5kdc[8127](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786336923, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.167.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 04:42:03.386739 8743 init.cc:377] Logged in from keytab as kudu/127.22.167.67@KRBTEST.COM (short username kudu)
I20260810 04:42:03.387132 8743 webserver.cc:460] Webserver started at http://127.22.167.67:45041/ using document root <none> and password file <none>
I20260810 04:42:03.387324 8743 fs_manager.cc:359] Metadata directory not provided
I20260810 04:42:03.387362 8743 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 04:42:03.387454 8743 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 04:42:03.388465 8743 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786336863388938-23197-0/minicluster-data/ts-2/data/instance:
uuid: "a7a85b58432a4fb0bfe5a2b37ed3dcdd"
format_stamp: "Formatted at 2026-08-10 04:42:03 on dist-test-slave-bqcl"
server_key: "3397e1975001cf92f05111cb7b71e1d0"
server_key_iv: "ce1063d6c0c34a2c2a8cf5b5cde8e89a"
server_key_version: "encryptionkey@0"
I20260810 04:42:03.388775 8743 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786336863388938-23197-0/minicluster-data/ts-2/wal/instance:
uuid: "a7a85b58432a4fb0bfe5a2b37ed3dcdd"
format_stamp: "Formatted at 2026-08-10 04:42:03 on dist-test-slave-bqcl"
server_key: "3397e1975001cf92f05111cb7b71e1d0"
server_key_iv: "ce1063d6c0c34a2c2a8cf5b5cde8e89a"
server_key_version: "encryptionkey@0"
I20260810 04:42:03.390009 8743 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260810 04:42:03.390736 8773 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 04:42:03.390878 8743 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260810 04:42:03.390962 8743 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786336863388938-23197-0/minicluster-data/ts-2/data,/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786336863388938-23197-0/minicluster-data/ts-2/wal
uuid: "a7a85b58432a4fb0bfe5a2b37ed3dcdd"
format_stamp: "Formatted at 2026-08-10 04:42:03 on dist-test-slave-bqcl"
server_key: "3397e1975001cf92f05111cb7b71e1d0"
server_key_iv: "ce1063d6c0c34a2c2a8cf5b5cde8e89a"
server_key_version: "encryptionkey@0"
I20260810 04:42:03.391016 8743 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786336863388938-23197-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786336863388938-23197-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786336863388938-23197-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 04:42:03.422052 8743 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 04:42:03.424187 8743 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 04:42:03.424310 8743 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 04:42:03.424499 8743 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 04:42:03.424755 8743 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 04:42:03.424785 8743 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 04:42:03.424806 8743 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 04:42:03.424820 8743 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 04:42:03.430169 8743 rpc_server.cc:307] RPC server started. Bound to: 127.22.167.67:33111
I20260810 04:42:03.430249 8967 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.167.67:33111 every 8 connection(s)
I20260810 04:42:03.430532 8743 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786336863388938-23197-0/minicluster-data/ts-2/data/info.pb
Aug 10 04:42:03 dist-test-slave-bqcl krb5kdc[8127](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786336923, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.167.67@KRBTEST.COM for kudu/127.22.167.126@KRBTEST.COM
I20260810 04:42:03.438530 23197 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskemv_AM/build/release/bin/kudu as pid 8743
I20260810 04:42:03.438616 23197 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786336863388938-23197-0/minicluster-data/ts-2/wal/instance
I20260810 04:42:03.438783 23197 external_mini_cluster.cc:1428] Setting key 19bdcbbd7a2be5b8da7b3be1515bcbfa
I20260810 04:42:03.439707 8261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:42:03.431186 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.167.67:51133 (local address 127.22.167.126:40437)
0810 04:42:03.431229 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0810 04:42:03.431230 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0810 04:42:03.431442 (+ 212us) server_negotiation.cc:407] Connection header received
0810 04:42:03.432021 (+ 579us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:42:03.432022 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:42:03.432034 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:42:03.432088 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0810 04:42:03.433094 (+ 1006us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:42:03.433627 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:42:03.434775 (+ 1148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:42:03.434962 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:42:03.435018 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:42:03.436976 (+ 1958us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 04:42:03.436977 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 04:42:03.436978 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 04:42:03.436985 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 04:42:03.438240 (+ 1255us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 04:42:03.438694 (+ 454us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 04:42:03.438695 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 04:42:03.438695 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 04:42:03.438750 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 04:42:03.439012 (+ 262us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 04:42:03.439013 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 04:42:03.439013 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 04:42:03.439152 (+ 139us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 04:42:03.439219 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0810 04:42:03.439522 (+ 303us) server_negotiation.cc:299] Negotiation successful
0810 04:42:03.439607 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260810 04:42:03.439831 8975 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:42:03.431157 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.126:40437 (local address 127.22.167.67:51133)
0810 04:42:03.431328 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0810 04:42:03.431364 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0810 04:42:03.431930 (+ 566us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 04:42:03.432166 (+ 236us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 04:42:03.432170 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 04:42:03.432758 (+ 588us) client_negotiation.cc:189] Negotiated authn=SASL
0810 04:42:03.432992 (+ 234us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:42:03.432999 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:42:03.433715 (+ 716us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:42:03.433716 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:42:03.434688 (+ 972us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:42:03.434691 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:42:03.435030 (+ 339us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:42:03.435031 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:42:03.435161 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:42:03.435572 (+ 411us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 04:42:03.435581 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0810 04:42:03.436892 (+ 1311us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 04:42:03.438309 (+ 1417us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 04:42:03.438315 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 04:42:03.438317 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 04:42:03.438586 (+ 269us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 04:42:03.438811 (+ 225us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 04:42:03.438813 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 04:42:03.438813 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 04:42:03.438949 (+ 136us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 04:42:03.439218 (+ 269us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 04:42:03.439223 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 04:42:03.439381 (+ 158us) client_negotiation.cc:769] Sending connection context
0810 04:42:03.439479 (+ 98us) client_negotiation.cc:240] Negotiation successful
0810 04:42:03.439634 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":94,"threads_started":1}
I20260810 04:42:03.440229 8970 heartbeater.cc:347] Connected to a master server at 127.22.167.126:40437
I20260810 04:42:03.440320 8970 heartbeater.cc:464] Registering TS with master...
I20260810 04:42:03.440487 8970 heartbeater.cc:511] Master 127.22.167.126:40437 requested a full tablet report, sending...
I20260810 04:42:03.440831 8203 ts_manager.cc:194] Registered new tserver with Master: a7a85b58432a4fb0bfe5a2b37ed3dcdd (127.22.167.67:33111)
I20260810 04:42:03.441180 8203 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.167.67@KRBTEST.COM'} at 127.22.167.67:51133
I20260810 04:42:03.450331 23197 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260810 04:42:03.457028 8261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:42:03.451517 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58030 (local address 127.22.167.126:40437)
0810 04:42:03.451572 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0810 04:42:03.451574 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 04:42:03.451663 (+ 89us) server_negotiation.cc:407] Connection header received
0810 04:42:03.451719 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:42:03.451720 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:42:03.451729 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:42:03.451773 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0810 04:42:03.452282 (+ 509us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:42:03.452755 (+ 473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:42:03.453448 (+ 693us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:42:03.453633 (+ 185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:42:03.453685 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:42:03.454772 (+ 1087us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 04:42:03.454774 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 04:42:03.454774 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 04:42:03.454781 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 04:42:03.455966 (+ 1185us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 04:42:03.456313 (+ 347us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 04:42:03.456314 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 04:42:03.456314 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 04:42:03.456365 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 04:42:03.456528 (+ 163us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 04:42:03.456528 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 04:42:03.456529 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 04:42:03.456672 (+ 143us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 04:42:03.456772 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0810 04:42:03.456865 (+ 93us) server_negotiation.cc:299] Negotiation successful
0810 04:42:03.456941 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260810 04:42:03.458459 8203 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58030:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 04:42:03.459967 8203 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 04:42:03.465709 8999 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:42:03.461986 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.67:33111 (local address 127.0.0.1:58996)
0810 04:42:03.462206 (+ 220us) negotiation.cc:107] Waiting for socket to connect
0810 04:42:03.462250 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0810 04:42:03.462315 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 04:42:03.462597 (+ 282us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 04:42:03.462599 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 04:42:03.462601 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 04:42:03.462649 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:42:03.462650 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:42:03.463587 (+ 937us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:42:03.463588 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:42:03.464542 (+ 954us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:42:03.464544 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:42:03.465416 (+ 872us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:42:03.465417 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:42:03.465495 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:42:03.465510 (+ 15us) client_negotiation.cc:769] Sending connection context
0810 04:42:03.465577 (+ 67us) client_negotiation.cc:240] Negotiation successful
0810 04:42:03.465627 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":193,"thread_start_us":50,"threads_started":1}
I20260810 04:42:03.465744 9002 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:42:03.462176 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58996 (local address 127.22.167.67:33111)
0810 04:42:03.462372 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0810 04:42:03.462377 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 04:42:03.462386 (+ 9us) server_negotiation.cc:407] Connection header received
0810 04:42:03.462425 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:42:03.462428 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:42:03.462444 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:42:03.462517 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 04:42:03.462722 (+ 205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:42:03.463491 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:42:03.464636 (+ 1145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:42:03.465343 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:42:03.465409 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:42:03.465468 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0810 04:42:03.465609 (+ 141us) server_negotiation.cc:299] Negotiation successful
0810 04:42:03.465652 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":166,"thread_start_us":80,"threads_started":1}
I20260810 04:42:03.466214 9003 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:42:03.461999 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.65:37805 (local address 127.0.0.1:55450)
0810 04:42:03.462408 (+ 409us) negotiation.cc:107] Waiting for socket to connect
0810 04:42:03.462417 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0810 04:42:03.462468 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 04:42:03.462768 (+ 300us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 04:42:03.462769 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 04:42:03.462772 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 04:42:03.462827 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:42:03.462828 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:42:03.464177 (+ 1349us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:42:03.464178 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:42:03.465127 (+ 949us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:42:03.465129 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:42:03.465956 (+ 827us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:42:03.465957 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:42:03.466022 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:42:03.466026 (+ 4us) client_negotiation.cc:769] Sending connection context
0810 04:42:03.466083 (+ 57us) client_negotiation.cc:240] Negotiation successful
0810 04:42:03.466145 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":389,"mutex_wait_us":25,"thread_start_us":88,"threads_started":1}
I20260810 04:42:03.466259 9001 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:42:03.462155 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55450 (local address 127.22.167.65:37805)
0810 04:42:03.462585 (+ 430us) server_negotiation.cc:206] Beginning negotiation
0810 04:42:03.462589 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 04:42:03.462599 (+ 10us) server_negotiation.cc:407] Connection header received
0810 04:42:03.462636 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:42:03.462638 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:42:03.462653 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:42:03.462895 (+ 242us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 04:42:03.462966 (+ 71us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:42:03.464078 (+ 1112us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:42:03.465205 (+ 1127us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:42:03.465879 (+ 674us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:42:03.465942 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:42:03.465999 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0810 04:42:03.466110 (+ 111us) server_negotiation.cc:299] Negotiation successful
0810 04:42:03.466179 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":393,"thread_start_us":88,"threads_started":1}
I20260810 04:42:03.466478 9000 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:42:03.462206 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.66:33781 (local address 127.0.0.1:53896)
0810 04:42:03.462418 (+ 212us) negotiation.cc:107] Waiting for socket to connect
0810 04:42:03.462426 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0810 04:42:03.462461 (+ 35us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 04:42:03.462904 (+ 443us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 04:42:03.462905 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 04:42:03.462906 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 04:42:03.462939 (+ 33us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:42:03.462940 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:42:03.464178 (+ 1238us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:42:03.464179 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:42:03.465159 (+ 980us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:42:03.465162 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:42:03.466168 (+ 1006us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:42:03.466169 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:42:03.466233 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:42:03.466237 (+ 4us) client_negotiation.cc:769] Sending connection context
0810 04:42:03.466395 (+ 158us) client_negotiation.cc:240] Negotiation successful
0810 04:42:03.466430 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":196,"mutex_wait_us":29,"thread_start_us":63,"threads_started":1}
I20260810 04:42:03.466770 9004 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:42:03.462394 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53896 (local address 127.22.167.66:33781)
0810 04:42:03.462586 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0810 04:42:03.462589 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 04:42:03.462599 (+ 10us) server_negotiation.cc:407] Connection header received
0810 04:42:03.462636 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:42:03.462638 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:42:03.462653 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:42:03.462828 (+ 175us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 04:42:03.463011 (+ 183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:42:03.464061 (+ 1050us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:42:03.465281 (+ 1220us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:42:03.466061 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:42:03.466118 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:42:03.466623 (+ 505us) server_negotiation.cc:1035] Waiting for connection context
0810 04:42:03.466649 (+ 26us) server_negotiation.cc:299] Negotiation successful
0810 04:42:03.466688 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":167,"thread_start_us":92,"threads_started":1}
I20260810 04:42:03.466749 8858 tablet_service.cc:1467] Processing CreateTablet for tablet 74c2e5314b304d16b8f09acdf6eaf021 (DEFAULT_TABLE table=test-table [id=867d8463129e43ee8a8bffdf9a304d10]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 04:42:03.466945 8399 tablet_service.cc:1467] Processing CreateTablet for tablet 74c2e5314b304d16b8f09acdf6eaf021 (DEFAULT_TABLE table=test-table [id=867d8463129e43ee8a8bffdf9a304d10]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 04:42:03.467108 8858 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 74c2e5314b304d16b8f09acdf6eaf021. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 04:42:03.467262 8399 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 74c2e5314b304d16b8f09acdf6eaf021. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 04:42:03.469537 9005 tablet_bootstrap.cc:492] T 74c2e5314b304d16b8f09acdf6eaf021 P a7a85b58432a4fb0bfe5a2b37ed3dcdd: Bootstrap starting.
I20260810 04:42:03.469659 9007 tablet_bootstrap.cc:492] T 74c2e5314b304d16b8f09acdf6eaf021 P 918ee8dee798468cb38c285c74c97974: Bootstrap starting.
I20260810 04:42:03.470253 9005 tablet_bootstrap.cc:654] T 74c2e5314b304d16b8f09acdf6eaf021 P a7a85b58432a4fb0bfe5a2b37ed3dcdd: Neither blocks nor log segments found. Creating new log.
I20260810 04:42:03.470364 9007 tablet_bootstrap.cc:654] T 74c2e5314b304d16b8f09acdf6eaf021 P 918ee8dee798468cb38c285c74c97974: Neither blocks nor log segments found. Creating new log.
I20260810 04:42:03.470484 8627 tablet_service.cc:1467] Processing CreateTablet for tablet 74c2e5314b304d16b8f09acdf6eaf021 (DEFAULT_TABLE table=test-table [id=867d8463129e43ee8a8bffdf9a304d10]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 04:42:03.470830 8627 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 74c2e5314b304d16b8f09acdf6eaf021. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 04:42:03.470816 9005 log.cc:824] T 74c2e5314b304d16b8f09acdf6eaf021 P a7a85b58432a4fb0bfe5a2b37ed3dcdd: Log is configured to *not* fsync() on all Append() calls
I20260810 04:42:03.470852 9007 log.cc:824] T 74c2e5314b304d16b8f09acdf6eaf021 P 918ee8dee798468cb38c285c74c97974: Log is configured to *not* fsync() on all Append() calls
I20260810 04:42:03.471606 9007 tablet_bootstrap.cc:492] T 74c2e5314b304d16b8f09acdf6eaf021 P 918ee8dee798468cb38c285c74c97974: No bootstrap required, opened a new log
I20260810 04:42:03.471684 9007 ts_tablet_manager.cc:1397] T 74c2e5314b304d16b8f09acdf6eaf021 P 918ee8dee798468cb38c285c74c97974: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260810 04:42:03.473583 9007 raft_consensus.cc:348] T 74c2e5314b304d16b8f09acdf6eaf021 P 918ee8dee798468cb38c285c74c97974 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a7a85b58432a4fb0bfe5a2b37ed3dcdd" member_type: VOTER last_known_addr { host: "127.22.167.67" port: 33111 } } peers { permanent_uuid: "918ee8dee798468cb38c285c74c97974" member_type: VOTER last_known_addr { host: "127.22.167.65" port: 37805 } } peers { permanent_uuid: "0b7c258d17b34723a0134a7c33273a10" member_type: VOTER last_known_addr { host: "127.22.167.66" port: 33781 } }
I20260810 04:42:03.473754 9007 raft_consensus.cc:374] T 74c2e5314b304d16b8f09acdf6eaf021 P 918ee8dee798468cb38c285c74c97974 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 04:42:03.473803 9007 raft_consensus.cc:729] T 74c2e5314b304d16b8f09acdf6eaf021 P 918ee8dee798468cb38c285c74c97974 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 918ee8dee798468cb38c285c74c97974, State: Initialized, Role: FOLLOWER
I20260810 04:42:03.473923 9007 consensus_queue.cc:260] T 74c2e5314b304d16b8f09acdf6eaf021 P 918ee8dee798468cb38c285c74c97974 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a7a85b58432a4fb0bfe5a2b37ed3dcdd" member_type: VOTER last_known_addr { host: "127.22.167.67" port: 33111 } } peers { permanent_uuid: "918ee8dee798468cb38c285c74c97974" member_type: VOTER last_known_addr { host: "127.22.167.65" port: 37805 } } peers { permanent_uuid: "0b7c258d17b34723a0134a7c33273a10" member_type: VOTER last_known_addr { host: "127.22.167.66" port: 33781 } }
I20260810 04:42:03.474020 9014 tablet_bootstrap.cc:492] T 74c2e5314b304d16b8f09acdf6eaf021 P 0b7c258d17b34723a0134a7c33273a10: Bootstrap starting.
I20260810 04:42:03.474247 9007 ts_tablet_manager.cc:1428] T 74c2e5314b304d16b8f09acdf6eaf021 P 918ee8dee798468cb38c285c74c97974: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260810 04:42:03.474390 8501 heartbeater.cc:503] Master 127.22.167.126:40437 was elected leader, sending a full tablet report...
I20260810 04:42:03.474658 9014 tablet_bootstrap.cc:654] T 74c2e5314b304d16b8f09acdf6eaf021 P 0b7c258d17b34723a0134a7c33273a10: Neither blocks nor log segments found. Creating new log.
I20260810 04:42:03.474728 9005 tablet_bootstrap.cc:492] T 74c2e5314b304d16b8f09acdf6eaf021 P a7a85b58432a4fb0bfe5a2b37ed3dcdd: No bootstrap required, opened a new log
I20260810 04:42:03.474802 9005 ts_tablet_manager.cc:1397] T 74c2e5314b304d16b8f09acdf6eaf021 P a7a85b58432a4fb0bfe5a2b37ed3dcdd: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.002s
I20260810 04:42:03.475193 9014 log.cc:824] T 74c2e5314b304d16b8f09acdf6eaf021 P 0b7c258d17b34723a0134a7c33273a10: Log is configured to *not* fsync() on all Append() calls
I20260810 04:42:03.475826 9014 tablet_bootstrap.cc:492] T 74c2e5314b304d16b8f09acdf6eaf021 P 0b7c258d17b34723a0134a7c33273a10: No bootstrap required, opened a new log
I20260810 04:42:03.475884 9014 ts_tablet_manager.cc:1397] T 74c2e5314b304d16b8f09acdf6eaf021 P 0b7c258d17b34723a0134a7c33273a10: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260810 04:42:03.476223 9005 raft_consensus.cc:348] T 74c2e5314b304d16b8f09acdf6eaf021 P a7a85b58432a4fb0bfe5a2b37ed3dcdd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a7a85b58432a4fb0bfe5a2b37ed3dcdd" member_type: VOTER last_known_addr { host: "127.22.167.67" port: 33111 } } peers { permanent_uuid: "918ee8dee798468cb38c285c74c97974" member_type: VOTER last_known_addr { host: "127.22.167.65" port: 37805 } } peers { permanent_uuid: "0b7c258d17b34723a0134a7c33273a10" member_type: VOTER last_known_addr { host: "127.22.167.66" port: 33781 } }
I20260810 04:42:03.476361 9005 raft_consensus.cc:374] T 74c2e5314b304d16b8f09acdf6eaf021 P a7a85b58432a4fb0bfe5a2b37ed3dcdd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 04:42:03.476405 9005 raft_consensus.cc:729] T 74c2e5314b304d16b8f09acdf6eaf021 P a7a85b58432a4fb0bfe5a2b37ed3dcdd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a7a85b58432a4fb0bfe5a2b37ed3dcdd, State: Initialized, Role: FOLLOWER
I20260810 04:42:03.476542 9005 consensus_queue.cc:260] T 74c2e5314b304d16b8f09acdf6eaf021 P a7a85b58432a4fb0bfe5a2b37ed3dcdd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a7a85b58432a4fb0bfe5a2b37ed3dcdd" member_type: VOTER last_known_addr { host: "127.22.167.67" port: 33111 } } peers { permanent_uuid: "918ee8dee798468cb38c285c74c97974" member_type: VOTER last_known_addr { host: "127.22.167.65" port: 37805 } } peers { permanent_uuid: "0b7c258d17b34723a0134a7c33273a10" member_type: VOTER last_known_addr { host: "127.22.167.66" port: 33781 } }
I20260810 04:42:03.476799 9005 ts_tablet_manager.cc:1428] T 74c2e5314b304d16b8f09acdf6eaf021 P a7a85b58432a4fb0bfe5a2b37ed3dcdd: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260810 04:42:03.476836 8970 heartbeater.cc:503] Master 127.22.167.126:40437 was elected leader, sending a full tablet report...
I20260810 04:42:03.477692 9014 raft_consensus.cc:348] T 74c2e5314b304d16b8f09acdf6eaf021 P 0b7c258d17b34723a0134a7c33273a10 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a7a85b58432a4fb0bfe5a2b37ed3dcdd" member_type: VOTER last_known_addr { host: "127.22.167.67" port: 33111 } } peers { permanent_uuid: "918ee8dee798468cb38c285c74c97974" member_type: VOTER last_known_addr { host: "127.22.167.65" port: 37805 } } peers { permanent_uuid: "0b7c258d17b34723a0134a7c33273a10" member_type: VOTER last_known_addr { host: "127.22.167.66" port: 33781 } }
I20260810 04:42:03.477854 9014 raft_consensus.cc:374] T 74c2e5314b304d16b8f09acdf6eaf021 P 0b7c258d17b34723a0134a7c33273a10 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 04:42:03.477900 9014 raft_consensus.cc:729] T 74c2e5314b304d16b8f09acdf6eaf021 P 0b7c258d17b34723a0134a7c33273a10 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0b7c258d17b34723a0134a7c33273a10, State: Initialized, Role: FOLLOWER
I20260810 04:42:03.478003 9014 consensus_queue.cc:260] T 74c2e5314b304d16b8f09acdf6eaf021 P 0b7c258d17b34723a0134a7c33273a10 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a7a85b58432a4fb0bfe5a2b37ed3dcdd" member_type: VOTER last_known_addr { host: "127.22.167.67" port: 33111 } } peers { permanent_uuid: "918ee8dee798468cb38c285c74c97974" member_type: VOTER last_known_addr { host: "127.22.167.65" port: 37805 } } peers { permanent_uuid: "0b7c258d17b34723a0134a7c33273a10" member_type: VOTER last_known_addr { host: "127.22.167.66" port: 33781 } }
I20260810 04:42:03.478246 9014 ts_tablet_manager.cc:1428] T 74c2e5314b304d16b8f09acdf6eaf021 P 0b7c258d17b34723a0134a7c33273a10: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260810 04:42:03.478271 8734 heartbeater.cc:503] Master 127.22.167.126:40437 was elected leader, sending a full tablet report...
W20260810 04:42:03.513113 8735 tablet.cc:2369] T 74c2e5314b304d16b8f09acdf6eaf021 P 0b7c258d17b34723a0134a7c33273a10: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 04:42:03.521947 9018 raft_consensus.cc:482] T 74c2e5314b304d16b8f09acdf6eaf021 P a7a85b58432a4fb0bfe5a2b37ed3dcdd [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 04:42:03.522051 9018 raft_consensus.cc:504] T 74c2e5314b304d16b8f09acdf6eaf021 P a7a85b58432a4fb0bfe5a2b37ed3dcdd [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a7a85b58432a4fb0bfe5a2b37ed3dcdd" member_type: VOTER last_known_addr { host: "127.22.167.67" port: 33111 } } peers { permanent_uuid: "918ee8dee798468cb38c285c74c97974" member_type: VOTER last_known_addr { host: "127.22.167.65" port: 37805 } } peers { permanent_uuid: "0b7c258d17b34723a0134a7c33273a10" member_type: VOTER last_known_addr { host: "127.22.167.66" port: 33781 } }
I20260810 04:42:03.522395 9018 leader_election.cc:288] T 74c2e5314b304d16b8f09acdf6eaf021 P a7a85b58432a4fb0bfe5a2b37ed3dcdd [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 918ee8dee798468cb38c285c74c97974 (127.22.167.65:37805), 0b7c258d17b34723a0134a7c33273a10 (127.22.167.66:33781)
I20260810 04:42:03.526161 9001 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:42:03.522633 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.167.67:40337 (local address 127.22.167.65:37805)
0810 04:42:03.522682 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0810 04:42:03.522684 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 04:42:03.522694 (+ 10us) server_negotiation.cc:407] Connection header received
0810 04:42:03.522722 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:42:03.522722 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:42:03.522734 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:42:03.522790 (+ 56us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 04:42:03.523021 (+ 231us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:42:03.523790 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:42:03.525202 (+ 1412us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:42:03.525798 (+ 596us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:42:03.525868 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:42:03.525877 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0810 04:42:03.526031 (+ 154us) server_negotiation.cc:299] Negotiation successful
0810 04:42:03.526068 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260810 04:42:03.526161 9004 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:42:03.522741 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.167.67:42513 (local address 127.22.167.66:33781)
0810 04:42:03.522793 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0810 04:42:03.522796 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 04:42:03.522945 (+ 149us) server_negotiation.cc:407] Connection header received
0810 04:42:03.522993 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:42:03.522995 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:42:03.523008 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:42:03.523058 (+ 50us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 04:42:03.523155 (+ 97us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:42:03.523830 (+ 675us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:42:03.525206 (+ 1376us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:42:03.525793 (+ 587us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:42:03.525868 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:42:03.525877 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0810 04:42:03.526031 (+ 154us) server_negotiation.cc:299] Negotiation successful
0810 04:42:03.526068 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260810 04:42:03.526185 8975 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:42:03.522558 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.65:37805 (local address 127.22.167.67:40337)
0810 04:42:03.522608 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0810 04:42:03.522619 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0810 04:42:03.522659 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 04:42:03.522854 (+ 195us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 04:42:03.522856 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 04:42:03.522858 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 04:42:03.522942 (+ 84us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:42:03.522944 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:42:03.523869 (+ 925us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:42:03.523869 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:42:03.525119 (+ 1250us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:42:03.525122 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:42:03.525870 (+ 748us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:42:03.525871 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:42:03.525967 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:42:03.525969 (+ 2us) client_negotiation.cc:769] Sending connection context
0810 04:42:03.526019 (+ 50us) client_negotiation.cc:240] Negotiation successful
0810 04:42:03.526067 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":29,"mutex_wait_us":66}
I20260810 04:42:03.526193 9020 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:42:03.522653 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.66:33781 (local address 127.22.167.67:42513)
0810 04:42:03.522875 (+ 222us) negotiation.cc:107] Waiting for socket to connect
0810 04:42:03.522885 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0810 04:42:03.522944 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 04:42:03.523067 (+ 123us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 04:42:03.523067 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 04:42:03.523068 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 04:42:03.523117 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:42:03.523117 (+ 0us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:42:03.523892 (+ 775us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:42:03.523893 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:42:03.525119 (+ 1226us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:42:03.525122 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:42:03.525870 (+ 748us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:42:03.525871 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:42:03.525963 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:42:03.525967 (+ 4us) client_negotiation.cc:769] Sending connection context
0810 04:42:03.526019 (+ 52us) client_negotiation.cc:240] Negotiation successful
0810 04:42:03.526067 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":204,"mutex_wait_us":11,"thread_start_us":147,"threads_started":1}
I20260810 04:42:03.526578 8664 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "74c2e5314b304d16b8f09acdf6eaf021" candidate_uuid: "a7a85b58432a4fb0bfe5a2b37ed3dcdd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0b7c258d17b34723a0134a7c33273a10" is_pre_election: true
I20260810 04:42:03.526588 8428 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "74c2e5314b304d16b8f09acdf6eaf021" candidate_uuid: "a7a85b58432a4fb0bfe5a2b37ed3dcdd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "918ee8dee798468cb38c285c74c97974" is_pre_election: true
I20260810 04:42:03.526718 8664 raft_consensus.cc:2454] T 74c2e5314b304d16b8f09acdf6eaf021 P 0b7c258d17b34723a0134a7c33273a10 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a7a85b58432a4fb0bfe5a2b37ed3dcdd in term 0.
I20260810 04:42:03.526748 8428 raft_consensus.cc:2454] T 74c2e5314b304d16b8f09acdf6eaf021 P 918ee8dee798468cb38c285c74c97974 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a7a85b58432a4fb0bfe5a2b37ed3dcdd in term 0.
I20260810 04:42:03.526937 8777 leader_election.cc:302] T 74c2e5314b304d16b8f09acdf6eaf021 P a7a85b58432a4fb0bfe5a2b37ed3dcdd [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0b7c258d17b34723a0134a7c33273a10, a7a85b58432a4fb0bfe5a2b37ed3dcdd; no voters:
I20260810 04:42:03.527088 9018 raft_consensus.cc:2781] T 74c2e5314b304d16b8f09acdf6eaf021 P a7a85b58432a4fb0bfe5a2b37ed3dcdd [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 04:42:03.527150 9018 raft_consensus.cc:482] T 74c2e5314b304d16b8f09acdf6eaf021 P a7a85b58432a4fb0bfe5a2b37ed3dcdd [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 04:42:03.527171 9018 raft_consensus.cc:3037] T 74c2e5314b304d16b8f09acdf6eaf021 P a7a85b58432a4fb0bfe5a2b37ed3dcdd [term 0 FOLLOWER]: Advancing to term 1
I20260810 04:42:03.527841 9018 raft_consensus.cc:504] T 74c2e5314b304d16b8f09acdf6eaf021 P a7a85b58432a4fb0bfe5a2b37ed3dcdd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a7a85b58432a4fb0bfe5a2b37ed3dcdd" member_type: VOTER last_known_addr { host: "127.22.167.67" port: 33111 } } peers { permanent_uuid: "918ee8dee798468cb38c285c74c97974" member_type: VOTER last_known_addr { host: "127.22.167.65" port: 37805 } } peers { permanent_uuid: "0b7c258d17b34723a0134a7c33273a10" member_type: VOTER last_known_addr { host: "127.22.167.66" port: 33781 } }
I20260810 04:42:03.527945 9018 leader_election.cc:288] T 74c2e5314b304d16b8f09acdf6eaf021 P a7a85b58432a4fb0bfe5a2b37ed3dcdd [CANDIDATE]: Term 1 election: Requested vote from peers 918ee8dee798468cb38c285c74c97974 (127.22.167.65:37805), 0b7c258d17b34723a0134a7c33273a10 (127.22.167.66:33781)
I20260810 04:42:03.528108 8428 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "74c2e5314b304d16b8f09acdf6eaf021" candidate_uuid: "a7a85b58432a4fb0bfe5a2b37ed3dcdd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "918ee8dee798468cb38c285c74c97974"
I20260810 04:42:03.528132 8664 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "74c2e5314b304d16b8f09acdf6eaf021" candidate_uuid: "a7a85b58432a4fb0bfe5a2b37ed3dcdd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0b7c258d17b34723a0134a7c33273a10"
I20260810 04:42:03.528187 8428 raft_consensus.cc:3037] T 74c2e5314b304d16b8f09acdf6eaf021 P 918ee8dee798468cb38c285c74c97974 [term 0 FOLLOWER]: Advancing to term 1
I20260810 04:42:03.528189 8664 raft_consensus.cc:3037] T 74c2e5314b304d16b8f09acdf6eaf021 P 0b7c258d17b34723a0134a7c33273a10 [term 0 FOLLOWER]: Advancing to term 1
I20260810 04:42:03.528906 8428 raft_consensus.cc:2454] T 74c2e5314b304d16b8f09acdf6eaf021 P 918ee8dee798468cb38c285c74c97974 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a7a85b58432a4fb0bfe5a2b37ed3dcdd in term 1.
I20260810 04:42:03.528908 8664 raft_consensus.cc:2454] T 74c2e5314b304d16b8f09acdf6eaf021 P 0b7c258d17b34723a0134a7c33273a10 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a7a85b58432a4fb0bfe5a2b37ed3dcdd in term 1.
I20260810 04:42:03.529055 8777 leader_election.cc:302] T 74c2e5314b304d16b8f09acdf6eaf021 P a7a85b58432a4fb0bfe5a2b37ed3dcdd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0b7c258d17b34723a0134a7c33273a10, a7a85b58432a4fb0bfe5a2b37ed3dcdd; no voters:
I20260810 04:42:03.529145 9018 raft_consensus.cc:2781] T 74c2e5314b304d16b8f09acdf6eaf021 P a7a85b58432a4fb0bfe5a2b37ed3dcdd [term 1 FOLLOWER]: Leader election won for term 1
I20260810 04:42:03.529289 9018 raft_consensus.cc:686] T 74c2e5314b304d16b8f09acdf6eaf021 P a7a85b58432a4fb0bfe5a2b37ed3dcdd [term 1 LEADER]: Becoming Leader. State: Replica: a7a85b58432a4fb0bfe5a2b37ed3dcdd, State: Running, Role: LEADER
I20260810 04:42:03.529368 9018 consensus_queue.cc:237] T 74c2e5314b304d16b8f09acdf6eaf021 P a7a85b58432a4fb0bfe5a2b37ed3dcdd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a7a85b58432a4fb0bfe5a2b37ed3dcdd" member_type: VOTER last_known_addr { host: "127.22.167.67" port: 33111 } } peers { permanent_uuid: "918ee8dee798468cb38c285c74c97974" member_type: VOTER last_known_addr { host: "127.22.167.65" port: 37805 } } peers { permanent_uuid: "0b7c258d17b34723a0134a7c33273a10" member_type: VOTER last_known_addr { host: "127.22.167.66" port: 33781 } }
I20260810 04:42:03.530225 8202 catalog_manager.cc:5495] T 74c2e5314b304d16b8f09acdf6eaf021 P a7a85b58432a4fb0bfe5a2b37ed3dcdd reported cstate change: term changed from 0 to 1, leader changed from <none> to a7a85b58432a4fb0bfe5a2b37ed3dcdd (127.22.167.67). New cstate: current_term: 1 leader_uuid: "a7a85b58432a4fb0bfe5a2b37ed3dcdd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a7a85b58432a4fb0bfe5a2b37ed3dcdd" member_type: VOTER last_known_addr { host: "127.22.167.67" port: 33111 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "918ee8dee798468cb38c285c74c97974" member_type: VOTER last_known_addr { host: "127.22.167.65" port: 37805 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0b7c258d17b34723a0134a7c33273a10" member_type: VOTER last_known_addr { host: "127.22.167.66" port: 33781 } health_report { overall_health: UNKNOWN } } }
I20260810 04:42:03.540151 9002 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:42:03.538094 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59008 (local address 127.22.167.67:33111)
0810 04:42:03.538171 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0810 04:42:03.538173 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 04:42:03.538183 (+ 10us) server_negotiation.cc:407] Connection header received
0810 04:42:03.538236 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:42:03.538236 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:42:03.538253 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:42:03.538310 (+ 57us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 04:42:03.538422 (+ 112us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:42:03.538881 (+ 459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:42:03.539558 (+ 677us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:42:03.539737 (+ 179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:42:03.539779 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:42:03.539885 (+ 106us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 04:42:03.539981 (+ 96us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 04:42:03.540036 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0810 04:42:03.540050 (+ 14us) server_negotiation.cc:299] Negotiation successful
0810 04:42:03.540064 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260810 04:42:03.541826 8428 raft_consensus.cc:1261] T 74c2e5314b304d16b8f09acdf6eaf021 P 918ee8dee798468cb38c285c74c97974 [term 1 FOLLOWER]: Refusing update from remote peer a7a85b58432a4fb0bfe5a2b37ed3dcdd: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 04:42:03.541831 8664 raft_consensus.cc:1261] T 74c2e5314b304d16b8f09acdf6eaf021 P 0b7c258d17b34723a0134a7c33273a10 [term 1 FOLLOWER]: Refusing update from remote peer a7a85b58432a4fb0bfe5a2b37ed3dcdd: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 04:42:03.542158 9018 consensus_queue.cc:1035] T 74c2e5314b304d16b8f09acdf6eaf021 P a7a85b58432a4fb0bfe5a2b37ed3dcdd [LEADER]: Connected to new peer: Peer: permanent_uuid: "918ee8dee798468cb38c285c74c97974" member_type: VOTER last_known_addr { host: "127.22.167.65" port: 37805 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 04:42:03.542312 9022 consensus_queue.cc:1035] T 74c2e5314b304d16b8f09acdf6eaf021 P a7a85b58432a4fb0bfe5a2b37ed3dcdd [LEADER]: Connected to new peer: Peer: permanent_uuid: "0b7c258d17b34723a0134a7c33273a10" member_type: VOTER last_known_addr { host: "127.22.167.66" port: 33781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 04:42:03.544987 9034 mvcc.cc:205] Tried to move back new op lower bound from 7316836038824968192 to 7316836038776946688. Current Snapshot: MvccSnapshot[applied={T|T < 7316836038824968192}]
I20260810 04:42:03.546444 9032 mvcc.cc:205] Tried to move back new op lower bound from 7316836038824968192 to 7316836038776946688. Current Snapshot: MvccSnapshot[applied={T|T < 7316836038824968192 or (T in {7316836038824968192})}]
I20260810 04:42:03.546531 8203 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58030:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 04:42:03.546622 8203 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58030:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 04:42:03.547639 8203 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 04541f691a694152b004dfc4275e5647: Sending DeleteTablet for 3 replicas of tablet 74c2e5314b304d16b8f09acdf6eaf021
I20260810 04:42:03.547969 8858 tablet_service.cc:1514] Processing DeleteTablet for tablet 74c2e5314b304d16b8f09acdf6eaf021 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 04:42:03 UTC) from {username='kudu', principal='kudu/127.22.167.126@KRBTEST.COM'} at 127.0.0.1:58996
I20260810 04:42:03.548094 8399 tablet_service.cc:1514] Processing DeleteTablet for tablet 74c2e5314b304d16b8f09acdf6eaf021 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 04:42:03 UTC) from {username='kudu', principal='kudu/127.22.167.126@KRBTEST.COM'} at 127.0.0.1:55450
I20260810 04:42:03.548190 9047 tablet_replica.cc:330] T 74c2e5314b304d16b8f09acdf6eaf021 P a7a85b58432a4fb0bfe5a2b37ed3dcdd: stopping tablet replica
I20260810 04:42:03.548285 9047 raft_consensus.cc:2229] T 74c2e5314b304d16b8f09acdf6eaf021 P a7a85b58432a4fb0bfe5a2b37ed3dcdd [term 1 LEADER]: Raft consensus shutting down.
I20260810 04:42:03.548329 8627 tablet_service.cc:1514] Processing DeleteTablet for tablet 74c2e5314b304d16b8f09acdf6eaf021 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 04:42:03 UTC) from {username='kudu', principal='kudu/127.22.167.126@KRBTEST.COM'} at 127.0.0.1:53896
I20260810 04:42:03.548369 9047 raft_consensus.cc:2258] T 74c2e5314b304d16b8f09acdf6eaf021 P a7a85b58432a4fb0bfe5a2b37ed3dcdd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 04:42:03.548362 23197 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskemv_AM/build/release/bin/kudu with pid 8301
I20260810 04:42:03.551199 9049 tablet_replica.cc:330] T 74c2e5314b304d16b8f09acdf6eaf021 P 0b7c258d17b34723a0134a7c33273a10: stopping tablet replica
I20260810 04:42:03.551301 9049 raft_consensus.cc:2229] T 74c2e5314b304d16b8f09acdf6eaf021 P 0b7c258d17b34723a0134a7c33273a10 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 04:42:03.551400 9049 raft_consensus.cc:2258] T 74c2e5314b304d16b8f09acdf6eaf021 P 0b7c258d17b34723a0134a7c33273a10 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 04:42:03.555966 9049 ts_tablet_manager.cc:1905] T 74c2e5314b304d16b8f09acdf6eaf021 P 0b7c258d17b34723a0134a7c33273a10: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 04:42:03.557623 9049 ts_tablet_manager.cc:1918] T 74c2e5314b304d16b8f09acdf6eaf021 P 0b7c258d17b34723a0134a7c33273a10: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 04:42:03.557695 9049 log.cc:1196] T 74c2e5314b304d16b8f09acdf6eaf021 P 0b7c258d17b34723a0134a7c33273a10: Deleting WAL directory at /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786336863388938-23197-0/minicluster-data/ts-1/wal/wals/74c2e5314b304d16b8f09acdf6eaf021
I20260810 04:42:03.557927 9049 ts_tablet_manager.cc:1939] T 74c2e5314b304d16b8f09acdf6eaf021 P 0b7c258d17b34723a0134a7c33273a10: Deleting consensus metadata
I20260810 04:42:03.565819 8185 catalog_manager.cc:4841] TS 0b7c258d17b34723a0134a7c33273a10 (127.22.167.66:33781): tablet 74c2e5314b304d16b8f09acdf6eaf021 (table test-table [id=867d8463129e43ee8a8bffdf9a304d10]) successfully deleted
I20260810 04:42:03.566442 9047 ts_tablet_manager.cc:1905] T 74c2e5314b304d16b8f09acdf6eaf021 P a7a85b58432a4fb0bfe5a2b37ed3dcdd: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 04:42:03.569532 9047 ts_tablet_manager.cc:1918] T 74c2e5314b304d16b8f09acdf6eaf021 P a7a85b58432a4fb0bfe5a2b37ed3dcdd: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 04:42:03.569619 9047 log.cc:1196] T 74c2e5314b304d16b8f09acdf6eaf021 P a7a85b58432a4fb0bfe5a2b37ed3dcdd: Deleting WAL directory at /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786336863388938-23197-0/minicluster-data/ts-2/wal/wals/74c2e5314b304d16b8f09acdf6eaf021
I20260810 04:42:03.569897 9047 ts_tablet_manager.cc:1939] T 74c2e5314b304d16b8f09acdf6eaf021 P a7a85b58432a4fb0bfe5a2b37ed3dcdd: Deleting consensus metadata
I20260810 04:42:03.570406 8183 catalog_manager.cc:4841] TS a7a85b58432a4fb0bfe5a2b37ed3dcdd (127.22.167.67:33111): tablet 74c2e5314b304d16b8f09acdf6eaf021 (table test-table [id=867d8463129e43ee8a8bffdf9a304d10]) successfully deleted
W20260810 04:42:03.572907 8185 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.22.167.65:37805 (error 108)
I20260810 04:42:03.573120 23197 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskemv_AM/build/release/bin/kudu with pid 8508
I20260810 04:42:03.574891 8999 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:42:03.574783 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.65:37805 (local address 127.0.0.1:55460)
0810 04:42:03.574830 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0810 04:42:03.574859 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.167.65:37805: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
W20260810 04:42:03.574983 8185 catalog_manager.cc:4584] TS 918ee8dee798468cb38c285c74c97974 (127.22.167.65:37805): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 74c2e5314b304d16b8f09acdf6eaf021: Network error: Client connection negotiation failed: client connection to 127.22.167.65:37805: connect: Connection refused (error 111)
I20260810 04:42:03.596488 23197 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskemv_AM/build/release/bin/kudu with pid 8743
I20260810 04:42:03.607008 23197 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskemv_AM/build/release/bin/kudu with pid 8154
2026-08-10T04:42:03Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationSecureCluster (2083 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegration
Loading random data
Initializing database '/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786336863388938-23197-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 04:42:03 dist-test-slave-bqcl krb5kdc[9055](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 04:42:03 dist-test-slave-bqcl krb5kdc[9055](info): set up 2 sockets
Aug 10 04:42:03 dist-test-slave-bqcl krb5kdc[9055](info): commencing operation
krb5kdc: starting...
W20260810 04:42:04.762818 23197 mini_kdc.cc:121] Time spent starting KDC: real 1.133s user 0.006s sys 0.000s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786336863388938-23197-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786336863388938-23197-0/krb5kdc/test-user.keytab.
Aug 10 04:42:04 dist-test-slave-bqcl krb5kdc[9055](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786336924, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T04:42:04Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T04:42:04Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.22.167.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.22.167.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.22.167.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786336863388938-23197-0/krb5kdc/rangeradmin_127.22.167.84@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.22.167.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786336863388938-23197-0/krb5kdc/rangeradmin_127.22.167.84@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.22.167.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.22.167.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.22.167.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786336863388938-23197-0/krb5kdc/rangerlookup_127.22.167.84@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.22.167.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786336863388938-23197-0/krb5kdc/rangerlookup_127.22.167.84@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.22.167.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.22.167.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.22.167.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786336863388938-23197-0/krb5kdc/HTTP_127.22.167.84@KRBTEST.COM.keytab.
Entry for principal HTTP/127.22.167.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786336863388938-23197-0/krb5kdc/HTTP_127.22.167.84@KRBTEST.COM.keytab.
I20260810 04:42:04.917055 23197 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786336863388938-23197-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskemv_AM/build/release/bin/postgres/pg_ctl -D /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786336863388938-23197-0/minicluster-data/postgres -l logfile start
2026-08-10 04:42:07.173 UTC [9106] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-10 04:42:07.173 UTC [9106] LOG: listening on IPv4 address "127.22.167.84", port 59443
2026-08-10 04:42:07.177 UTC [9106] LOG: listening on Unix socket "/tmp/.s.PGSQL.59443"
2026-08-10 04:42:07.205 UTC [9109] LOG: database system was shut down at 2026-08-10 04:42:05 UTC
2026-08-10 04:42:07.209 UTC [9106] LOG: database system is ready to accept connections
I20260810 04:42:08.279632 23197 mini_postgres.cc:96] Postgres bound to 59443
2026-08-10 04:42:08.286 UTC [9122] FATAL: database "slave" does not exist
127.22.167.84:59443 - accepting connections
I20260810 04:42:08.287461 23197 mini_ranger.cc:161] Starting Ranger...
I20260810 04:42:08.301983 23197 mini_ranger.cc:84] Created miniranger Postgres user
I20260810 04:42:08.724633 23197 mini_ranger.cc:87] Created ranger Postgres database
I20260810 04:42:08.724731 23197 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786336863388938-23197-0/minicluster-data/ranger-admin
2026-08-10 04:42:09,010 [I] DB FLAVOR :POSTGRES
2026-08-10 04:42:09,011 [I] --------- Verifying Ranger DB connection ---------
2026-08-10 04:42:09,011 [I] Checking connection..
2026-08-10 04:42:09,011 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskemv_AM/build/release/bin/postgresql.jar:/tmp/dist-test-taskemv_AM/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.167.84:59443/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-10 04:42:09,252 [I] Checking connection passed.
2026-08-10 04:42:09,252 [I] --------- Verifying version history table ---------
2026-08-10 04:42:09,252 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskemv_AM/build/release/bin/postgresql.jar:/tmp/dist-test-taskemv_AM/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.167.84:59443/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-10 04:42:09,487 [I] Table x_db_version_h does not exist in database ranger
2026-08-10 04:42:09,487 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskemv_AM/build/release/bin/postgresql.jar:/tmp/dist-test-taskemv_AM/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.167.84:59443/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-10 04:42:09,736 [I] Table x_db_version_h does not exist in database ranger
2026-08-10 04:42:09,736 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-10 04:42:09,736 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskemv_AM/build/release/bin/postgresql.jar:/tmp/dist-test-taskemv_AM/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.167.84:59443/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskemv_AM/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-10 04:42:09.993 UTC [9223] WARNING: there is no transaction in progress
2026-08-10 04:42:10,001 [I] create_dbversion_catalog.sql file imported successfully
2026-08-10 04:42:10,002 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskemv_AM/build/release/bin/postgresql.jar:/tmp/dist-test-taskemv_AM/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.167.84:59443/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-10 04:42:10,258 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-10 04:42:10,258 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-10 04:42:10,259 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskemv_AM/build/release/bin/postgresql.jar:/tmp/dist-test-taskemv_AM/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.167.84:59443/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-10 04:42:10,498 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskemv_AM/build/release/bin/postgresql.jar:/tmp/dist-test-taskemv_AM/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.167.84:59443/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-10 04:42:10,752 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskemv_AM/build/release/bin/postgresql.jar:/tmp/dist-test-taskemv_AM/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.167.84:59443/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-bqcl.c.gcp-upstream.internal','N') ;"
2026-08-10 04:42:10,975 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskemv_AM/build/release/bin/postgresql.jar:/tmp/dist-test-taskemv_AM/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.167.84:59443/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-10 04:42:11,233 [I] Table x_portal_user does not exist in database ranger
2026-08-10 04:42:11,233 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskemv_AM/build/release/bin/postgresql.jar:/tmp/dist-test-taskemv_AM/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.167.84:59443/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-10 04:42:11,489 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-10 04:42:11,490 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskemv_AM/build/release/bin/postgresql.jar:/tmp/dist-test-taskemv_AM/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.167.84:59443/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-10 04:42:11,737 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskemv_AM/build/release/bin/postgresql.jar:/tmp/dist-test-taskemv_AM/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.167.84:59443/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-10 04:42:11,984 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-10 04:42:11,984 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskemv_AM/build/release/bin/postgresql.jar:/tmp/dist-test-taskemv_AM/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.167.84:59443/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskemv_AM/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-10 04:42:12.880 UTC [9382] WARNING: there is no transaction in progress
2026-08-10 04:42:12.896 UTC [9382] WARNING: there is no transaction in progress
2026-08-10 04:42:12.911 UTC [9382] WARNING: there is no transaction in progress
2026-08-10 04:42:12.926 UTC [9382] WARNING: there is no transaction in progress
2026-08-10 04:42:12.941 UTC [9382] WARNING: there is no transaction in progress
2026-08-10 04:42:12.957 UTC [9382] WARNING: there is no transaction in progress
2026-08-10 04:42:13.010 UTC [9382] WARNING: there is no transaction in progress
2026-08-10 04:42:13.016 UTC [9382] WARNING: there is no transaction in progress
2026-08-10 04:42:13.029 UTC [9382] WARNING: there is no transaction in progress
2026-08-10 04:42:13.038 UTC [9382] WARNING: there is no transaction in progress
2026-08-10 04:42:13.048 UTC [9382] WARNING: there is no transaction in progress
2026-08-10 04:42:13.060 UTC [9382] WARNING: there is no transaction in progress
2026-08-10 04:42:13.069 UTC [9382] WARNING: there is no transaction in progress
2026-08-10 04:42:13.075 UTC [9382] WARNING: there is no transaction in progress
2026-08-10 04:42:13.081 UTC [9382] WARNING: there is no transaction in progress
2026-08-10 04:42:13.087 UTC [9382] WARNING: there is no transaction in progress
2026-08-10 04:42:13,656 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-10 04:42:13,656 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskemv_AM/build/release/bin/postgresql.jar:/tmp/dist-test-taskemv_AM/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.167.84:59443/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-10 04:42:13,898 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-10 04:42:13,899 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskemv_AM/build/release/bin/postgresql.jar:/tmp/dist-test-taskemv_AM/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.167.84:59443/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-10 04:42:14,161 [I] Table x_portal_user already exists in database 'ranger'
2026-08-10 04:42:14,162 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskemv_AM/build/release/bin/postgresql.jar:/tmp/dist-test-taskemv_AM/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.167.84:59443/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-10 04:42:14,421 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-10 04:42:14,421 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskemv_AM/build/release/bin/postgresql.jar:/tmp/dist-test-taskemv_AM/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.167.84:59443/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-10 04:42:14,677 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskemv_AM/build/release/bin/postgresql.jar:/tmp/dist-test-taskemv_AM/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.167.84:59443/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-10 04:42:14,937 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskemv_AM/build/release/bin/postgresql.jar:/tmp/dist-test-taskemv_AM/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.167.84:59443/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-bqcl.c.gcp-upstream.internal';"
2026-08-10 04:42:15,186 [I] CORE_DB_SCHEMA import status has been updated
2026-08-10 04:42:15,186 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskemv_AM/build/release/bin/postgresql.jar:/tmp/dist-test-taskemv_AM/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.167.84:59443/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-10 04:42:15,442 [I] DB_PATCHES have already been applied
I20260810 04:42:15.448406 23197 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786336863388938-23197-0/minicluster-data/ranger-admin:/tmp/dist-test-taskemv_AM/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskemv_AM/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskemv_AM/build/release/bin/postgresql.jar:/tmp/dist-test-taskemv_AM/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260810 04:42:15.448468 23197 mini_ranger.cc:193] Using host: 127.22.167.84
I20260810 04:42:15.452325 23197 mini_ranger.cc:227] Ranger admin URL: http://127.22.167.84:58339
Aug 10, 2026 4:42:15 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 10, 2026 4:42:15 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 10, 2026 4:42:15 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786336863388938-23197-0/minicluster-data/ranger-admin/ews/webapp
Aug 10, 2026 4:42:15 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786336863388938-23197-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 10, 2026 4:42:15 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786336863388938-23197-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 10, 2026 4:42:16 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786336863388938-23197-0/minicluster-data/ranger-admin/ews/webapp].
Aug 10, 2026 4:42:16 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260810 04:42:16.694345 23197 mini_ranger.cc:160] Time spent starting Ranger: real 8.407s user 0.001s sys 0.010s
Aug 10, 2026 4:42:37 AM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 10, 2026 4:42:37 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
Aug 10, 2026 4:42:37 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 10, 2026 4:42:37 AM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 10, 2026 4:42:37 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 10, 2026 4:42:37 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 10, 2026 4:42:37 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 10, 2026 4:42:37 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 10, 2026 4:42:37 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 10, 2026 4:42:37 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 10, 2026 4:42:37 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 10, 2026 4:42:37 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 10, 2026 4:42:37 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 10, 2026 4:42:37 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 10, 2026 4:42:37 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 10, 2026 4:42:37 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 10, 2026 4:42:37 AM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 10, 2026 4:42:38 AM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260810 04:42:38.693553 23197 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.22.167.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.22.167.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.22.167.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786336863388938-23197-0/krb5kdc/rangerkms_127.22.167.84@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.22.167.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786336863388938-23197-0/krb5kdc/rangerkms_127.22.167.84@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.22.167.84@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.22.167.84@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.22.167.84@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786336863388938-23197-0/krb5kdc/HTTP_127.22.167.84@KRBTEST.COM.keytab.
Entry for principal HTTP/127.22.167.84@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786336863388938-23197-0/krb5kdc/HTTP_127.22.167.84@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 10 04:42:38 dist-test-slave-bqcl krb5kdc[9055](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786336958, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260810 04:42:38.781316 23197 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260810 04:42:38.793664 23197 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260810 04:42:39.561154 23197 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260810 04:42:39.561231 23197 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786336863388938-23197-0/minicluster-data/ranger-kms
I20260810 04:42:39.561244 23197 mini_ranger_kms.cc:227] Using postgres at 127.22.167.84:59443
2026-08-10 04:42:40,271 [I] DB FLAVOR :POSTGRES
2026-08-10 04:42:40,272 [I] --------- Verifying Ranger DB connection ---------
2026-08-10 04:42:40,272 [I] Checking connection
2026-08-10 04:42:40,272 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskemv_AM/build/release/bin/postgresql.jar:/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786336863388938-23197-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.167.84:59443/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-10 04:42:40,617 [I] connection success
2026-08-10 04:42:40,617 [I] --------- Verifying Ranger DB tables ---------
2026-08-10 04:42:40,617 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-10 04:42:40,617 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskemv_AM/build/release/bin/postgresql.jar:/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786336863388938-23197-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.167.84:59443/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-10 04:42:40,866 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-10 04:42:40,866 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-10 04:42:40,866 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-10 04:42:40,867 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskemv_AM/build/release/bin/postgresql.jar:/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786336863388938-23197-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.167.84:59443/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786336863388938-23197-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-10 04:42:41,120 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260810 04:42:41.571319 23197 mini_ranger_kms.cc:310] Created kms service
I20260810 04:42:41.755232 23197 mini_ranger_kms.cc:326] Created kudu user
I20260810 04:42:41.795290 23197 mini_ranger_kms.cc:343] Created rangerkms user
Aug 10 04:42:41 dist-test-slave-bqcl krb5kdc[9055](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786336958, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.22.167.84@KRBTEST.COM
I20260810 04:42:42.155577 23197 mini_ranger_kms.cc:384] Added ranger policy
I20260810 04:42:42.155869 23197 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786336863388938-23197-0/minicluster-data/ranger-kms:/tmp/dist-test-taskemv_AM/build/release/bin/postgresql.jar:/tmp/dist-test-taskemv_AM/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskemv_AM/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260810 04:42:42.155915 23197 mini_ranger_kms.cc:240] Using host: 127.22.167.84
I20260810 04:42:42.160781 23197 mini_ranger_kms.cc:276] Ranger KMS PID: 9671
I20260810 04:42:42.160874 23197 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.22.167.84:38857
Aug 10, 2026 4:42:43 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 10, 2026 4:42:43 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 10, 2026 4:42:43 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786336863388938-23197-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 10, 2026 4:42:43 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786336863388938-23197-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 10, 2026 4:42:43 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786336863388938-23197-0/minicluster-data/ranger-kms/ews/webapp].
Aug 10, 2026 4:42:43 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
I20260810 04:42:44.677486 23197 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 5.896s user 0.000s sys 0.676s
I20260810 04:42:44.677595 23197 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260810 04:42:44.677680 23197 mini_ranger_kms.cc:401] 127.22.167.84:38857/kms/v1/keys
Aug 10 04:42:44 dist-test-slave-bqcl krb5kdc[9055](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786336964, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.22.167.84@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Aug 10 04:42:45 dist-test-slave-bqcl krb5kdc[9055](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786336964, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.22.167.84@KRBTEST.COM for HTTP/127.22.167.84@KRBTEST.COM
Aug 10 04:42:45 dist-test-slave-bqcl krb5kdc[9055](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786336964, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.22.167.84@KRBTEST.COM for HTTP/127.22.167.84@KRBTEST.COM
Aug 10 04:42:48 dist-test-slave-bqcl krb5kdc[9055](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786336968, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.22.167.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.167.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.167.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786336863388938-23197-0/krb5kdc/kudu_127.22.167.126.keytab.
Entry for principal kudu/127.22.167.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786336863388938-23197-0/krb5kdc/kudu_127.22.167.126.keytab.
I20260810 04:42:48.309068 23197 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskemv_AM/build/release/bin/kudu
/tmp/dist-test-taskemv_AM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786336863388938-23197-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786336863388938-23197-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786336863388938-23197-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786336863388938-23197-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.167.126:41675
--webserver_interface=127.22.167.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786336863388938-23197-0/krb5kdc/kudu_127.22.167.126.keytab
--principal=kudu/127.22.167.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.167.84:36145
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.167.126:41675
--ranger_config_path=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786336863388938-23197-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.22.167.84:38857/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786336863388938-23197-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786336863388938-23197-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786336863388938-23197-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 04:42:48.409937 9764 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 04:42:48.410257 9764 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 04:42:48.410296 9764 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 04:42:48.411859 9764 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 04:42:48.411921 9764 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 04:42:48.411942 9764 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 04:42:48.411962 9764 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 04:42:48.411988 9764 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 04:42:48.413843 9764 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.167.84:36145
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786336863388938-23197-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786336863388938-23197-0/minicluster-data/master-0/wal
--ranger_kms_url=127.22.167.84:38857/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.22.167.126:41675
--ranger_config_path=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786336863388938-23197-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786336863388938-23197-0/krb5kdc/kudu_127.22.167.126.keytab
--principal=kudu/127.22.167.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.167.126:41675
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786336863388938-23197-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.167.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9764
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786336863388938-23197-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 03:54:01 UTC on ccycloud.quasar-icijmt.root.comops.site
build id 4577009
I20260810 04:42:48.414084 9764 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 04:42:48.414338 9764 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 04:42:48.419370 9774 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 04:42:48.419402 9773 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 04:42:48.419412 9764 server_base.cc:1034] running on GCE node
W20260810 04:42:48.419394 9776 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 04:42:48.419819 9764 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 04:42:48.420092 9764 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 04:42:48.421229 9764 hybrid_clock.cc:648] HybridClock initialized: now 1786336968421217 us; error 39 us; skew 500 ppm
Aug 10 04:42:48 dist-test-slave-bqcl krb5kdc[9055](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786336968, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.167.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 04:42:48.426643 9764 init.cc:377] Logged in from keytab as kudu/127.22.167.126@KRBTEST.COM (short username kudu)
I20260810 04:42:48.427033 9764 webserver.cc:460] Webserver started at http://127.22.167.126:38711/ using document root <none> and password file <none>
I20260810 04:42:48.427222 9764 fs_manager.cc:359] Metadata directory not provided
I20260810 04:42:48.427261 9764 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 04:42:48.427383 9764 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 10 04:42:48 dist-test-slave-bqcl krb5kdc[9055](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786336968, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.167.126@KRBTEST.COM for HTTP/127.22.167.84@KRBTEST.COM
I20260810 04:42:48.490767 9764 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786336863388938-23197-0/minicluster-data/master-0/data/instance:
uuid: "ecf86695fb764288b300a8048b21bba1"
format_stamp: "Formatted at 2026-08-10 04:42:48 on dist-test-slave-bqcl"
server_key: "283f74895eacfa2083c14c3034321bf0"
server_key_iv: "04df0165442edfc6c6f20218e48dbf18"
server_key_version: "kuduclusterkey@0"
I20260810 04:42:48.491350 9764 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786336863388938-23197-0/minicluster-data/master-0/wal/instance:
uuid: "ecf86695fb764288b300a8048b21bba1"
format_stamp: "Formatted at 2026-08-10 04:42:48 on dist-test-slave-bqcl"
server_key: "283f74895eacfa2083c14c3034321bf0"
server_key_iv: "04df0165442edfc6c6f20218e48dbf18"
server_key_version: "kuduclusterkey@0"
I20260810 04:42:48.493337 9764 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.002s sys 0.001s
I20260810 04:42:48.507547 9786 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 04:42:48.507786 9764 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 04:42:48.507879 9764 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786336863388938-23197-0/minicluster-data/master-0/data,/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786336863388938-23197-0/minicluster-data/master-0/wal
uuid: "ecf86695fb764288b300a8048b21bba1"
format_stamp: "Formatted at 2026-08-10 04:42:48 on dist-test-slave-bqcl"
server_key: "283f74895eacfa2083c14c3034321bf0"
server_key_iv: "04df0165442edfc6c6f20218e48dbf18"
server_key_version: "kuduclusterkey@0"
I20260810 04:42:48.507974 9764 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786336863388938-23197-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786336863388938-23197-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786336863388938-23197-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 04:42:48.537312 9764 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 04:42:48.539604 9764 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 04:42:48.539791 9764 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 04:42:48.551810 9764 rpc_server.cc:307] RPC server started. Bound to: 127.22.167.126:41675
I20260810 04:42:48.552011 9873 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.167.126:41675 every 8 connection(s)
I20260810 04:42:48.552327 9764 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786336863388938-23197-0/minicluster-data/master-0/data/info.pb
I20260810 04:42:48.553292 9876 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 04:42:48.555467 23197 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskemv_AM/build/release/bin/kudu as pid 9764
I20260810 04:42:48.555565 23197 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786336863388938-23197-0/minicluster-data/master-0/wal/instance
I20260810 04:42:48.555763 23197 external_mini_cluster.cc:1428] Setting key 02155ea37486d00aa9eb661a1e1831da
I20260810 04:42:48.556406 9876 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ecf86695fb764288b300a8048b21bba1: Bootstrap starting.
I20260810 04:42:48.557461 9876 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ecf86695fb764288b300a8048b21bba1: Neither blocks nor log segments found. Creating new log.
I20260810 04:42:48.557997 9876 log.cc:824] T 00000000000000000000000000000000 P ecf86695fb764288b300a8048b21bba1: Log is configured to *not* fsync() on all Append() calls
I20260810 04:42:48.559005 9876 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ecf86695fb764288b300a8048b21bba1: No bootstrap required, opened a new log
I20260810 04:42:48.562237 9876 raft_consensus.cc:348] T 00000000000000000000000000000000 P ecf86695fb764288b300a8048b21bba1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ecf86695fb764288b300a8048b21bba1" member_type: VOTER last_known_addr { host: "127.22.167.126" port: 41675 } }
I20260810 04:42:48.562369 9876 raft_consensus.cc:374] T 00000000000000000000000000000000 P ecf86695fb764288b300a8048b21bba1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 04:42:48.562391 9876 raft_consensus.cc:729] T 00000000000000000000000000000000 P ecf86695fb764288b300a8048b21bba1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ecf86695fb764288b300a8048b21bba1, State: Initialized, Role: FOLLOWER
I20260810 04:42:48.562494 9876 consensus_queue.cc:260] T 00000000000000000000000000000000 P ecf86695fb764288b300a8048b21bba1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ecf86695fb764288b300a8048b21bba1" member_type: VOTER last_known_addr { host: "127.22.167.126" port: 41675 } }
I20260810 04:42:48.562564 9876 raft_consensus.cc:388] T 00000000000000000000000000000000 P ecf86695fb764288b300a8048b21bba1 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 04:42:48.562588 9876 raft_consensus.cc:482] T 00000000000000000000000000000000 P ecf86695fb764288b300a8048b21bba1 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 04:42:48.562631 9876 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ecf86695fb764288b300a8048b21bba1 [term 0 FOLLOWER]: Advancing to term 1
I20260810 04:42:48.563175 9876 raft_consensus.cc:504] T 00000000000000000000000000000000 P ecf86695fb764288b300a8048b21bba1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ecf86695fb764288b300a8048b21bba1" member_type: VOTER last_known_addr { host: "127.22.167.126" port: 41675 } }
I20260810 04:42:48.563282 9876 leader_election.cc:302] T 00000000000000000000000000000000 P ecf86695fb764288b300a8048b21bba1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ecf86695fb764288b300a8048b21bba1; no voters:
I20260810 04:42:48.563444 9876 leader_election.cc:288] T 00000000000000000000000000000000 P ecf86695fb764288b300a8048b21bba1 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 04:42:48.563580 9885 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ecf86695fb764288b300a8048b21bba1 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 04:42:48.563747 9876 sys_catalog.cc:564] T 00000000000000000000000000000000 P ecf86695fb764288b300a8048b21bba1 [sys.catalog]: configured and running, proceeding with master startup.
Aug 10 04:42:48 dist-test-slave-bqcl krb5kdc[9055](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786336968, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.167.126@KRBTEST.COM
I20260810 04:42:48.563901 9885 raft_consensus.cc:686] T 00000000000000000000000000000000 P ecf86695fb764288b300a8048b21bba1 [term 1 LEADER]: Becoming Leader. State: Replica: ecf86695fb764288b300a8048b21bba1, State: Running, Role: LEADER
I20260810 04:42:48.564451 9885 consensus_queue.cc:237] T 00000000000000000000000000000000 P ecf86695fb764288b300a8048b21bba1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ecf86695fb764288b300a8048b21bba1" member_type: VOTER last_known_addr { host: "127.22.167.126" port: 41675 } }
I20260810 04:42:48.565053 9887 sys_catalog.cc:455] T 00000000000000000000000000000000 P ecf86695fb764288b300a8048b21bba1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ecf86695fb764288b300a8048b21bba1. Latest consensus state: current_term: 1 leader_uuid: "ecf86695fb764288b300a8048b21bba1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ecf86695fb764288b300a8048b21bba1" member_type: VOTER last_known_addr { host: "127.22.167.126" port: 41675 } } }
I20260810 04:42:48.565137 9887 sys_catalog.cc:458] T 00000000000000000000000000000000 P ecf86695fb764288b300a8048b21bba1 [sys.catalog]: This master's current role is: LEADER
I20260810 04:42:48.565443 9886 sys_catalog.cc:455] T 00000000000000000000000000000000 P ecf86695fb764288b300a8048b21bba1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ecf86695fb764288b300a8048b21bba1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ecf86695fb764288b300a8048b21bba1" member_type: VOTER last_known_addr { host: "127.22.167.126" port: 41675 } } }
I20260810 04:42:48.565531 9886 sys_catalog.cc:458] T 00000000000000000000000000000000 P ecf86695fb764288b300a8048b21bba1 [sys.catalog]: This master's current role is: LEADER
I20260810 04:42:48.565800 9899 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 04:42:48.566411 9899 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 04:42:48.566924 9876 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786336863388938-23197-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260810 04:42:48.573546 9879 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:42:48.556273 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50804 (local address 127.22.167.126:41675)
0810 04:42:48.556570 (+ 297us) server_negotiation.cc:206] Beginning negotiation
0810 04:42:48.556580 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 04:42:48.556612 (+ 32us) server_negotiation.cc:407] Connection header received
0810 04:42:48.557182 (+ 570us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:42:48.557192 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:42:48.557257 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:42:48.557411 (+ 154us) server_negotiation.cc:226] Negotiated authn=SASL
0810 04:42:48.558271 (+ 860us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:42:48.559528 (+ 1257us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:42:48.560763 (+ 1235us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:42:48.561397 (+ 634us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:42:48.561520 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:42:48.564394 (+ 2874us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 04:42:48.564398 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 04:42:48.564398 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 04:42:48.564411 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0810 04:42:48.567381 (+ 2970us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 04:42:48.567835 (+ 454us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 04:42:48.567838 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 04:42:48.567839 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 04:42:48.569503 (+ 1664us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 04:42:48.569963 (+ 460us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 04:42:48.569964 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 04:42:48.569964 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 04:42:48.571362 (+ 1398us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 04:42:48.571929 (+ 567us) server_negotiation.cc:1035] Waiting for connection context
0810 04:42:48.572046 (+ 117us) server_negotiation.cc:299] Negotiation successful
0810 04:42:48.572890 (+ 844us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":250,"thread_start_us":124,"threads_started":1}
I20260810 04:42:48.575628 9899 catalog_manager.cc:1347] Generated new cluster ID: 5c6c90315dc64611953b91328b132d53
I20260810 04:42:48.575681 9899 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 04:42:48.579814 9899 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 04:42:48.580536 9899 catalog_manager.cc:1504] Loading token signing keys...
I20260810 04:42:48.591495 9899 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ecf86695fb764288b300a8048b21bba1: Generated new TSK 0
I20260810 04:42:48.591653 9899 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.22.167.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.167.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.167.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786336863388938-23197-0/krb5kdc/kudu_127.22.167.65.keytab.
Entry for principal kudu/127.22.167.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786336863388938-23197-0/krb5kdc/kudu_127.22.167.65.keytab.
I20260810 04:42:49.422360 23197 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskemv_AM/build/release/bin/kudu
/tmp/dist-test-taskemv_AM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786336863388938-23197-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786336863388938-23197-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786336863388938-23197-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786336863388938-23197-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.22.167.84:38857/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.167.65:0
--local_ip_for_outbound_sockets=127.22.167.65
--webserver_interface=127.22.167.65
--webserver_port=0
--tserver_master_addrs=127.22.167.126:41675
--keytab_file=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786336863388938-23197-0/krb5kdc/kudu_127.22.167.65.keytab
--principal=kudu/127.22.167.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.167.84:36145
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786336863388938-23197-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786336863388938-23197-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786336863388938-23197-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 04:42:49.567744 9957 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 04:42:49.568043 9957 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 04:42:49.568107 9957 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 04:42:49.569927 9957 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 04:42:49.569989 9957 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 04:42:49.570029 9957 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.167.65
I20260810 04:42:49.572363 9957 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.167.84:36145
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786336863388938-23197-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786336863388938-23197-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.22.167.84:38857/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786336863388938-23197-0/krb5kdc/kudu_127.22.167.65.keytab
--principal=kudu/127.22.167.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.167.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786336863388938-23197-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.22.167.65
--webserver_port=0
--tserver_master_addrs=127.22.167.126:41675
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9957
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.167.65
--log_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786336863388938-23197-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 03:54:01 UTC on ccycloud.quasar-icijmt.root.comops.site
build id 4577009
I20260810 04:42:49.572660 9957 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 04:42:49.572996 9957 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 04:42:49.579638 9969 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 04:42:49.579833 9971 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 04:42:49.579686 9967 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 04:42:49.579973 9957 server_base.cc:1034] running on GCE node
I20260810 04:42:49.580219 9957 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 04:42:49.580508 9957 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 04:42:49.581720 9957 hybrid_clock.cc:648] HybridClock initialized: now 1786336969581690 us; error 45 us; skew 500 ppm
Aug 10 04:42:49 dist-test-slave-bqcl krb5kdc[9055](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786336969, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.167.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 04:42:49.585868 9957 init.cc:377] Logged in from keytab as kudu/127.22.167.65@KRBTEST.COM (short username kudu)
I20260810 04:42:49.586444 9957 webserver.cc:460] Webserver started at http://127.22.167.65:41707/ using document root <none> and password file <none>
I20260810 04:42:49.586755 9957 fs_manager.cc:359] Metadata directory not provided
I20260810 04:42:49.586843 9957 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 04:42:49.586987 9957 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 10 04:42:49 dist-test-slave-bqcl krb5kdc[9055](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786336969, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.167.65@KRBTEST.COM for HTTP/127.22.167.84@KRBTEST.COM
I20260810 04:42:49.599735 9957 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786336863388938-23197-0/minicluster-data/ts-0/data/instance:
uuid: "e899545cb79146848bfadf7a0cdc441c"
format_stamp: "Formatted at 2026-08-10 04:42:49 on dist-test-slave-bqcl"
server_key: "f17c3925e26e2d68fc0e310a9e866bbc"
server_key_iv: "3363943bde0098d5521774a460b19493"
server_key_version: "kuduclusterkey@0"
I20260810 04:42:49.600310 9957 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786336863388938-23197-0/minicluster-data/ts-0/wal/instance:
uuid: "e899545cb79146848bfadf7a0cdc441c"
format_stamp: "Formatted at 2026-08-10 04:42:49 on dist-test-slave-bqcl"
server_key: "f17c3925e26e2d68fc0e310a9e866bbc"
server_key_iv: "3363943bde0098d5521774a460b19493"
server_key_version: "kuduclusterkey@0"
I20260810 04:42:49.602690 9957 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.001s sys 0.000s
I20260810 04:42:49.616575 9983 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 04:42:49.616825 9957 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.001s
I20260810 04:42:49.616926 9957 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786336863388938-23197-0/minicluster-data/ts-0/data,/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786336863388938-23197-0/minicluster-data/ts-0/wal
uuid: "e899545cb79146848bfadf7a0cdc441c"
format_stamp: "Formatted at 2026-08-10 04:42:49 on dist-test-slave-bqcl"
server_key: "f17c3925e26e2d68fc0e310a9e866bbc"
server_key_iv: "3363943bde0098d5521774a460b19493"
server_key_version: "kuduclusterkey@0"
I20260810 04:42:49.617031 9957 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786336863388938-23197-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786336863388938-23197-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786336863388938-23197-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 04:42:49.647461 9957 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 04:42:49.650064 9957 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 04:42:49.650260 9957 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 04:42:49.650686 9957 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 04:42:49.651144 9957 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 04:42:49.651181 9957 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 04:42:49.651212 9957 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 04:42:49.651229 9957 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 04:42:49.667570 9957 rpc_server.cc:307] RPC server started. Bound to: 127.22.167.65:32877
I20260810 04:42:49.668038 9957 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786336863388938-23197-0/minicluster-data/ts-0/data/info.pb
I20260810 04:42:49.668946 10174 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.167.65:32877 every 8 connection(s)
I20260810 04:42:49.670645 23197 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskemv_AM/build/release/bin/kudu as pid 9957
I20260810 04:42:49.670746 23197 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786336863388938-23197-0/minicluster-data/ts-0/wal/instance
I20260810 04:42:49.670919 23197 external_mini_cluster.cc:1428] Setting key db56130fc8440742d6241b20b4ac4196
Aug 10 04:42:49 dist-test-slave-bqcl krb5kdc[9055](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786336969, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.167.65@KRBTEST.COM for kudu/127.22.167.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.22.167.66@KRBTEST.COM; defaulting to no policy
I20260810 04:42:49.690398 10180 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:42:49.669192 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.126:41675 (local address 127.22.167.65:60793)
0810 04:42:49.669447 (+ 255us) negotiation.cc:107] Waiting for socket to connect
0810 04:42:49.669494 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0810 04:42:49.671584 (+ 2090us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 04:42:49.674970 (+ 3386us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 04:42:49.674974 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 04:42:49.675329 (+ 355us) client_negotiation.cc:189] Negotiated authn=SASL
0810 04:42:49.675769 (+ 440us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:42:49.675777 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:42:49.676525 (+ 748us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:42:49.676526 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:42:49.677947 (+ 1421us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:42:49.677952 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:42:49.678698 (+ 746us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:42:49.678700 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:42:49.678937 (+ 237us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:42:49.684009 (+ 5072us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 04:42:49.684020 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0810 04:42:49.686572 (+ 2552us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 04:42:49.688595 (+ 2023us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 04:42:49.688602 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 04:42:49.688605 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 04:42:49.688922 (+ 317us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 04:42:49.689147 (+ 225us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 04:42:49.689148 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 04:42:49.689148 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 04:42:49.689285 (+ 137us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 04:42:49.689575 (+ 290us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 04:42:49.689580 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 04:42:49.689766 (+ 186us) client_negotiation.cc:769] Sending connection context
0810 04:42:49.689951 (+ 185us) client_negotiation.cc:240] Negotiation successful
0810 04:42:49.690146 (+ 195us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":183,"thread_start_us":116,"threads_started":1}
I20260810 04:42:49.690848 10178 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:42:49.669199 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.167.65:60793 (local address 127.22.167.126:41675)
0810 04:42:49.669578 (+ 379us) server_negotiation.cc:206] Beginning negotiation
0810 04:42:49.669580 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 04:42:49.669592 (+ 12us) server_negotiation.cc:407] Connection header received
0810 04:42:49.674746 (+ 5154us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:42:49.674747 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:42:49.674767 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:42:49.674832 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0810 04:42:49.675888 (+ 1056us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:42:49.676419 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:42:49.678062 (+ 1643us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:42:49.678441 (+ 379us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:42:49.678531 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:42:49.686702 (+ 8171us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 04:42:49.686704 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 04:42:49.686704 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 04:42:49.686713 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0810 04:42:49.688497 (+ 1784us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 04:42:49.689006 (+ 509us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 04:42:49.689008 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 04:42:49.689008 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 04:42:49.689081 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 04:42:49.689356 (+ 275us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 04:42:49.689356 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 04:42:49.689357 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 04:42:49.689498 (+ 141us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 04:42:49.689551 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0810 04:42:49.690646 (+ 1095us) server_negotiation.cc:299] Negotiation successful
0810 04:42:49.690750 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":355,"thread_start_us":70,"threads_started":1}
I20260810 04:42:49.691272 10176 heartbeater.cc:347] Connected to a master server at 127.22.167.126:41675
I20260810 04:42:49.691385 10176 heartbeater.cc:464] Registering TS with master...
I20260810 04:42:49.691591 10176 heartbeater.cc:511] Master 127.22.167.126:41675 requested a full tablet report, sending...
I20260810 04:42:49.692276 9821 ts_manager.cc:194] Registered new tserver with Master: e899545cb79146848bfadf7a0cdc441c (127.22.167.65:32877)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.167.66@KRBTEST.COM" created.
I20260810 04:42:49.693166 9821 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.167.65@KRBTEST.COM'} at 127.22.167.65:60793
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.167.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786336863388938-23197-0/krb5kdc/kudu_127.22.167.66.keytab.
Entry for principal kudu/127.22.167.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786336863388938-23197-0/krb5kdc/kudu_127.22.167.66.keytab.
I20260810 04:42:49.712710 23197 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskemv_AM/build/release/bin/kudu
/tmp/dist-test-taskemv_AM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786336863388938-23197-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786336863388938-23197-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786336863388938-23197-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786336863388938-23197-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.22.167.84:38857/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.167.66:0
--local_ip_for_outbound_sockets=127.22.167.66
--webserver_interface=127.22.167.66
--webserver_port=0
--tserver_master_addrs=127.22.167.126:41675
--keytab_file=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786336863388938-23197-0/krb5kdc/kudu_127.22.167.66.keytab
--principal=kudu/127.22.167.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.167.84:36145
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786336863388938-23197-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786336863388938-23197-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786336863388938-23197-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 04:42:49.843276 10185 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 04:42:49.843559 10185 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 04:42:49.843636 10185 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 04:42:49.844907 10185 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 04:42:49.844956 10185 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 04:42:49.845005 10185 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.167.66
I20260810 04:42:49.846639 10185 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.167.84:36145
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786336863388938-23197-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786336863388938-23197-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.22.167.84:38857/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786336863388938-23197-0/krb5kdc/kudu_127.22.167.66.keytab
--principal=kudu/127.22.167.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.167.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786336863388938-23197-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.22.167.66
--webserver_port=0
--tserver_master_addrs=127.22.167.126:41675
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10185
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.167.66
--log_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786336863388938-23197-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 03:54:01 UTC on ccycloud.quasar-icijmt.root.comops.site
build id 4577009
I20260810 04:42:49.846889 10185 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 04:42:49.847129 10185 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 04:42:49.853078 10197 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 04:42:49.853102 10199 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 04:42:49.853509 10201 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 04:42:49.858728 10185 server_base.cc:1034] running on GCE node
I20260810 04:42:49.858937 10185 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 04:42:49.859166 10185 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 04:42:49.860306 10185 hybrid_clock.cc:648] HybridClock initialized: now 1786336969860287 us; error 36 us; skew 500 ppm
Aug 10 04:42:49 dist-test-slave-bqcl krb5kdc[9055](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786336969, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.167.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 04:42:49.863282 10185 init.cc:377] Logged in from keytab as kudu/127.22.167.66@KRBTEST.COM (short username kudu)
I20260810 04:42:49.863763 10185 webserver.cc:460] Webserver started at http://127.22.167.66:39877/ using document root <none> and password file <none>
I20260810 04:42:49.863992 10185 fs_manager.cc:359] Metadata directory not provided
I20260810 04:42:49.864041 10185 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 04:42:49.864179 10185 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 10 04:42:49 dist-test-slave-bqcl krb5kdc[9055](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786336969, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.167.66@KRBTEST.COM for HTTP/127.22.167.84@KRBTEST.COM
I20260810 04:42:49.875380 10185 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786336863388938-23197-0/minicluster-data/ts-1/data/instance:
uuid: "7d82a56f77f0440987f48ef02149151e"
format_stamp: "Formatted at 2026-08-10 04:42:49 on dist-test-slave-bqcl"
server_key: "ab258f282712fa03ecb15f43a10732b2"
server_key_iv: "f7c6c964c5e391376c1a91f3150773a3"
server_key_version: "kuduclusterkey@0"
I20260810 04:42:49.875866 10185 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786336863388938-23197-0/minicluster-data/ts-1/wal/instance:
uuid: "7d82a56f77f0440987f48ef02149151e"
format_stamp: "Formatted at 2026-08-10 04:42:49 on dist-test-slave-bqcl"
server_key: "ab258f282712fa03ecb15f43a10732b2"
server_key_iv: "f7c6c964c5e391376c1a91f3150773a3"
server_key_version: "kuduclusterkey@0"
I20260810 04:42:49.880249 10185 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260810 04:42:49.894227 10220 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 04:42:49.894534 10185 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260810 04:42:49.894753 10185 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786336863388938-23197-0/minicluster-data/ts-1/data,/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786336863388938-23197-0/minicluster-data/ts-1/wal
uuid: "7d82a56f77f0440987f48ef02149151e"
format_stamp: "Formatted at 2026-08-10 04:42:49 on dist-test-slave-bqcl"
server_key: "ab258f282712fa03ecb15f43a10732b2"
server_key_iv: "f7c6c964c5e391376c1a91f3150773a3"
server_key_version: "kuduclusterkey@0"
I20260810 04:42:49.894930 10185 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786336863388938-23197-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786336863388938-23197-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786336863388938-23197-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
Aug 10 04:42:49 dist-test-slave-bqcl krb5kdc[9055](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786336969, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.167.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 04:42:49.968402 10185 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 04:42:49.971083 10185 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 04:42:49.971275 10185 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 04:42:49.971676 10185 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 04:42:49.972062 10185 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 04:42:49.972110 10185 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 04:42:49.972146 10185 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 04:42:49.972173 10185 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 04:42:49.990299 10185 rpc_server.cc:307] RPC server started. Bound to: 127.22.167.66:36695
I20260810 04:42:49.990444 10399 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.167.66:36695 every 8 connection(s)
I20260810 04:42:49.990772 10185 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786336863388938-23197-0/minicluster-data/ts-1/data/info.pb
I20260810 04:42:49.994812 23197 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskemv_AM/build/release/bin/kudu as pid 10185
I20260810 04:42:49.994920 23197 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786336863388938-23197-0/minicluster-data/ts-1/wal/instance
I20260810 04:42:49.995106 23197 external_mini_cluster.cc:1428] Setting key 810fa5020d38d029c69b75698b2d1898
Aug 10 04:42:49 dist-test-slave-bqcl krb5kdc[9055](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786336969, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.167.66@KRBTEST.COM for kudu/127.22.167.126@KRBTEST.COM
I20260810 04:42:50.001905 10178 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:42:49.991863 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.167.66:36699 (local address 127.22.167.126:41675)
0810 04:42:49.991922 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0810 04:42:49.991924 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 04:42:49.991935 (+ 11us) server_negotiation.cc:407] Connection header received
0810 04:42:49.992523 (+ 588us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:42:49.992525 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:42:49.992541 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:42:49.992575 (+ 34us) server_negotiation.cc:226] Negotiated authn=SASL
0810 04:42:49.993484 (+ 909us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:42:49.994311 (+ 827us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:42:49.995751 (+ 1440us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:42:49.996029 (+ 278us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:42:49.996074 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:42:49.998566 (+ 2492us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 04:42:49.998568 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 04:42:49.998568 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 04:42:49.998576 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0810 04:42:50.000130 (+ 1554us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 04:42:50.000590 (+ 460us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 04:42:50.000592 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 04:42:50.000592 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 04:42:50.000652 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 04:42:50.000962 (+ 310us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 04:42:50.000963 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 04:42:50.000964 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 04:42:50.001135 (+ 171us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 04:42:50.001246 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0810 04:42:50.001691 (+ 445us) server_negotiation.cc:299] Negotiation successful
0810 04:42:50.001804 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260810 04:42:50.002543 10403 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:42:49.991516 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.126:41675 (local address 127.22.167.66:36699)
0810 04:42:49.991676 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0810 04:42:49.991722 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0810 04:42:49.992385 (+ 663us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 04:42:49.992779 (+ 394us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 04:42:49.992784 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 04:42:49.993066 (+ 282us) client_negotiation.cc:189] Negotiated authn=SASL
0810 04:42:49.993360 (+ 294us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:42:49.993368 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:42:49.994436 (+ 1068us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:42:49.994437 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:42:49.995655 (+ 1218us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:42:49.995658 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:42:49.996166 (+ 508us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:42:49.996167 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:42:49.996298 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:42:49.996838 (+ 540us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 04:42:49.996848 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0810 04:42:49.998457 (+ 1609us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 04:42:50.000222 (+ 1765us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 04:42:50.000229 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 04:42:50.000231 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 04:42:50.000506 (+ 275us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 04:42:50.000715 (+ 209us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 04:42:50.000716 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 04:42:50.000716 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 04:42:50.000868 (+ 152us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 04:42:50.001299 (+ 431us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 04:42:50.001307 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 04:42:50.001496 (+ 189us) client_negotiation.cc:769] Sending connection context
0810 04:42:50.002068 (+ 572us) client_negotiation.cc:240] Negotiation successful
0810 04:42:50.002284 (+ 216us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":71,"threads_started":1}
I20260810 04:42:50.006698 10400 heartbeater.cc:347] Connected to a master server at 127.22.167.126:41675
I20260810 04:42:50.006850 10400 heartbeater.cc:464] Registering TS with master...
I20260810 04:42:50.007089 10400 heartbeater.cc:511] Master 127.22.167.126:41675 requested a full tablet report, sending...
I20260810 04:42:50.007611 9821 ts_manager.cc:194] Registered new tserver with Master: 7d82a56f77f0440987f48ef02149151e (127.22.167.66:36695)
I20260810 04:42:50.008121 9821 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.167.66@KRBTEST.COM'} at 127.22.167.66:36699
WARNING: no policy specified for kudu/127.22.167.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.167.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.167.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786336863388938-23197-0/krb5kdc/kudu_127.22.167.67.keytab.
Entry for principal kudu/127.22.167.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786336863388938-23197-0/krb5kdc/kudu_127.22.167.67.keytab.
I20260810 04:42:50.037428 23197 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskemv_AM/build/release/bin/kudu
/tmp/dist-test-taskemv_AM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786336863388938-23197-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786336863388938-23197-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786336863388938-23197-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786336863388938-23197-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.22.167.84:38857/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.167.67:0
--local_ip_for_outbound_sockets=127.22.167.67
--webserver_interface=127.22.167.67
--webserver_port=0
--tserver_master_addrs=127.22.167.126:41675
--keytab_file=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786336863388938-23197-0/krb5kdc/kudu_127.22.167.67.keytab
--principal=kudu/127.22.167.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.167.84:36145
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786336863388938-23197-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786336863388938-23197-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786336863388938-23197-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 04:42:50.187557 10413 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 04:42:50.187942 10413 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 04:42:50.188061 10413 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 04:42:50.189811 10413 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 04:42:50.189926 10413 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 04:42:50.190011 10413 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.167.67
I20260810 04:42:50.192288 10413 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.167.84:36145
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786336863388938-23197-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786336863388938-23197-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.22.167.84:38857/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786336863388938-23197-0/krb5kdc/kudu_127.22.167.67.keytab
--principal=kudu/127.22.167.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.167.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786336863388938-23197-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.22.167.67
--webserver_port=0
--tserver_master_addrs=127.22.167.126:41675
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10413
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.167.67
--log_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786336863388938-23197-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 03:54:01 UTC on ccycloud.quasar-icijmt.root.comops.site
build id 4577009
I20260810 04:42:50.192526 10413 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 04:42:50.192806 10413 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 04:42:50.200374 10425 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 04:42:50.200374 10424 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 04:42:50.201035 10428 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 04:42:50.201357 10413 server_base.cc:1034] running on GCE node
I20260810 04:42:50.201567 10413 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 04:42:50.201799 10413 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 04:42:50.202955 10413 hybrid_clock.cc:648] HybridClock initialized: now 1786336970202940 us; error 26 us; skew 500 ppm
Aug 10 04:42:50 dist-test-slave-bqcl krb5kdc[9055](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786336970, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.167.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 04:42:50.206091 10413 init.cc:377] Logged in from keytab as kudu/127.22.167.67@KRBTEST.COM (short username kudu)
I20260810 04:42:50.206624 10413 webserver.cc:460] Webserver started at http://127.22.167.67:35019/ using document root <none> and password file <none>
I20260810 04:42:50.206840 10413 fs_manager.cc:359] Metadata directory not provided
I20260810 04:42:50.206887 10413 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 04:42:50.207019 10413 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 10 04:42:50 dist-test-slave-bqcl krb5kdc[9055](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786336970, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.167.67@KRBTEST.COM for HTTP/127.22.167.84@KRBTEST.COM
I20260810 04:42:50.220564 10413 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786336863388938-23197-0/minicluster-data/ts-2/data/instance:
uuid: "3b7f04a99439447e8dd21771e5ad2f2c"
format_stamp: "Formatted at 2026-08-10 04:42:50 on dist-test-slave-bqcl"
server_key: "aabd1b661412dcbf4ec9221667e328fa"
server_key_iv: "30e8d184977e12b6219e96104b46161b"
server_key_version: "kuduclusterkey@0"
I20260810 04:42:50.221167 10413 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786336863388938-23197-0/minicluster-data/ts-2/wal/instance:
uuid: "3b7f04a99439447e8dd21771e5ad2f2c"
format_stamp: "Formatted at 2026-08-10 04:42:50 on dist-test-slave-bqcl"
server_key: "aabd1b661412dcbf4ec9221667e328fa"
server_key_iv: "30e8d184977e12b6219e96104b46161b"
server_key_version: "kuduclusterkey@0"
I20260810 04:42:50.223632 10413 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.000s sys 0.002s
I20260810 04:42:50.238344 10442 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 04:42:50.238572 10413 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.001s
I20260810 04:42:50.238658 10413 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786336863388938-23197-0/minicluster-data/ts-2/data,/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786336863388938-23197-0/minicluster-data/ts-2/wal
uuid: "3b7f04a99439447e8dd21771e5ad2f2c"
format_stamp: "Formatted at 2026-08-10 04:42:50 on dist-test-slave-bqcl"
server_key: "aabd1b661412dcbf4ec9221667e328fa"
server_key_iv: "30e8d184977e12b6219e96104b46161b"
server_key_version: "kuduclusterkey@0"
I20260810 04:42:50.238732 10413 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786336863388938-23197-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786336863388938-23197-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786336863388938-23197-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 04:42:50.254307 10413 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 04:42:50.256783 10413 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 04:42:50.256968 10413 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 04:42:50.257233 10413 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 04:42:50.257606 10413 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 04:42:50.257650 10413 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 04:42:50.257678 10413 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 04:42:50.257699 10413 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 04:42:50.272730 10413 rpc_server.cc:307] RPC server started. Bound to: 127.22.167.67:35955
I20260810 04:42:50.273053 10635 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.167.67:35955 every 8 connection(s)
I20260810 04:42:50.273177 10413 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786336863388938-23197-0/minicluster-data/ts-2/data/info.pb
I20260810 04:42:50.279076 23197 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskemv_AM/build/release/bin/kudu as pid 10413
I20260810 04:42:50.279193 23197 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786336863388938-23197-0/minicluster-data/ts-2/wal/instance
I20260810 04:42:50.279397 23197 external_mini_cluster.cc:1428] Setting key 8097314c3e38f69564e3083c4dc902d0
Aug 10 04:42:50 dist-test-slave-bqcl krb5kdc[9055](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786336970, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.167.67@KRBTEST.COM for kudu/127.22.167.126@KRBTEST.COM
I20260810 04:42:50.284662 10178 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:42:50.274113 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.167.67:60687 (local address 127.22.167.126:41675)
0810 04:42:50.274180 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0810 04:42:50.274182 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 04:42:50.274522 (+ 340us) server_negotiation.cc:407] Connection header received
0810 04:42:50.275414 (+ 892us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:42:50.275416 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:42:50.275429 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:42:50.275498 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0810 04:42:50.276562 (+ 1064us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:42:50.277323 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:42:50.278593 (+ 1270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:42:50.278897 (+ 304us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:42:50.278994 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:42:50.281782 (+ 2788us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 04:42:50.281784 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 04:42:50.281784 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 04:42:50.281793 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0810 04:42:50.283378 (+ 1585us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 04:42:50.283747 (+ 369us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 04:42:50.283748 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 04:42:50.283748 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 04:42:50.283822 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 04:42:50.284024 (+ 202us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 04:42:50.284025 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 04:42:50.284026 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 04:42:50.284177 (+ 151us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 04:42:50.284247 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0810 04:42:50.284474 (+ 227us) server_negotiation.cc:299] Negotiation successful
0810 04:42:50.284567 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260810 04:42:50.284780 10638 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:42:50.274090 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.126:41675 (local address 127.22.167.67:60687)
0810 04:42:50.274333 (+ 243us) negotiation.cc:107] Waiting for socket to connect
0810 04:42:50.274383 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0810 04:42:50.275287 (+ 904us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 04:42:50.275595 (+ 308us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 04:42:50.275599 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 04:42:50.276030 (+ 431us) client_negotiation.cc:189] Negotiated authn=SASL
0810 04:42:50.276444 (+ 414us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:42:50.276451 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:42:50.277426 (+ 975us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:42:50.277427 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:42:50.278481 (+ 1054us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:42:50.278484 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:42:50.279005 (+ 521us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:42:50.279006 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:42:50.279242 (+ 236us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:42:50.280005 (+ 763us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 04:42:50.280016 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0810 04:42:50.281691 (+ 1675us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 04:42:50.283454 (+ 1763us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 04:42:50.283460 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 04:42:50.283462 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 04:42:50.283677 (+ 215us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 04:42:50.283882 (+ 205us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 04:42:50.283883 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 04:42:50.283883 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 04:42:50.283972 (+ 89us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 04:42:50.284240 (+ 268us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 04:42:50.284243 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 04:42:50.284355 (+ 112us) client_negotiation.cc:769] Sending connection context
0810 04:42:50.284428 (+ 73us) client_negotiation.cc:240] Negotiation successful
0810 04:42:50.284539 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":193,"thread_start_us":145,"threads_started":1}
I20260810 04:42:50.285383 10636 heartbeater.cc:347] Connected to a master server at 127.22.167.126:41675
I20260810 04:42:50.285493 10636 heartbeater.cc:464] Registering TS with master...
I20260810 04:42:50.285687 10636 heartbeater.cc:511] Master 127.22.167.126:41675 requested a full tablet report, sending...
I20260810 04:42:50.286043 9821 ts_manager.cc:194] Registered new tserver with Master: 3b7f04a99439447e8dd21771e5ad2f2c (127.22.167.67:35955)
I20260810 04:42:50.286419 9821 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.167.67@KRBTEST.COM'} at 127.22.167.67:60687
I20260810 04:42:50.291343 23197 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260810 04:42:50.299469 10178 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:42:50.292748 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50816 (local address 127.22.167.126:41675)
0810 04:42:50.292785 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0810 04:42:50.292786 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0810 04:42:50.293063 (+ 277us) server_negotiation.cc:407] Connection header received
0810 04:42:50.293091 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:42:50.293092 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:42:50.293104 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:42:50.293177 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0810 04:42:50.293778 (+ 601us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:42:50.294572 (+ 794us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:42:50.295255 (+ 683us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:42:50.295563 (+ 308us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:42:50.295638 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:42:50.297085 (+ 1447us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 04:42:50.297086 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 04:42:50.297086 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 04:42:50.297092 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0810 04:42:50.298299 (+ 1207us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 04:42:50.298717 (+ 418us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 04:42:50.298718 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 04:42:50.298718 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 04:42:50.298766 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 04:42:50.298949 (+ 183us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 04:42:50.298950 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 04:42:50.298950 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 04:42:50.299103 (+ 153us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 04:42:50.299221 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0810 04:42:50.299336 (+ 115us) server_negotiation.cc:299] Negotiation successful
0810 04:42:50.299405 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15}
I20260810 04:42:50.303570 9821 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50816:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 04:42:50.305322 9821 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 04:42:50.315070 10661 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:42:50.308279 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.67:35955 (local address 127.0.0.1:53060)
0810 04:42:50.308851 (+ 572us) negotiation.cc:107] Waiting for socket to connect
0810 04:42:50.308898 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0810 04:42:50.309248 (+ 350us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 04:42:50.309788 (+ 540us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 04:42:50.309790 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 04:42:50.309793 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 04:42:50.309847 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:42:50.309848 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:42:50.311272 (+ 1424us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:42:50.311273 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:42:50.312451 (+ 1178us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:42:50.312454 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:42:50.314768 (+ 2314us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:42:50.314769 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:42:50.314862 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:42:50.314880 (+ 18us) client_negotiation.cc:769] Sending connection context
0810 04:42:50.314933 (+ 53us) client_negotiation.cc:240] Negotiation successful
0810 04:42:50.314989 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":527,"thread_start_us":401,"threads_started":1}
I20260810 04:42:50.315599 10654 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:42:50.308462 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53060 (local address 127.22.167.67:35955)
0810 04:42:50.309554 (+ 1092us) server_negotiation.cc:206] Beginning negotiation
0810 04:42:50.309560 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 04:42:50.309574 (+ 14us) server_negotiation.cc:407] Connection header received
0810 04:42:50.309657 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:42:50.309660 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:42:50.309680 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:42:50.309785 (+ 105us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 04:42:50.309905 (+ 120us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:42:50.311187 (+ 1282us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:42:50.313688 (+ 2501us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:42:50.314658 (+ 970us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:42:50.315196 (+ 538us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:42:50.315383 (+ 187us) server_negotiation.cc:1035] Waiting for connection context
0810 04:42:50.315416 (+ 33us) server_negotiation.cc:299] Negotiation successful
0810 04:42:50.315457 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":961,"thread_start_us":142,"threads_started":1}
I20260810 04:42:50.316452 10665 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:42:50.308326 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.65:32877 (local address 127.0.0.1:56882)
0810 04:42:50.309175 (+ 849us) negotiation.cc:107] Waiting for socket to connect
0810 04:42:50.309185 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0810 04:42:50.309245 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 04:42:50.311973 (+ 2728us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 04:42:50.311974 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 04:42:50.311976 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 04:42:50.312024 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:42:50.312026 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:42:50.314239 (+ 2213us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:42:50.314241 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:42:50.315284 (+ 1043us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:42:50.315287 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:42:50.316200 (+ 913us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:42:50.316201 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:42:50.316261 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:42:50.316308 (+ 47us) client_negotiation.cc:769] Sending connection context
0810 04:42:50.316347 (+ 39us) client_negotiation.cc:240] Negotiation successful
0810 04:42:50.316386 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":825,"mutex_wait_us":38,"thread_start_us":367,"threads_started":1}
I20260810 04:42:50.316480 10658 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:42:50.308496 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56882 (local address 127.22.167.65:32877)
0810 04:42:50.311747 (+ 3251us) server_negotiation.cc:206] Beginning negotiation
0810 04:42:50.311751 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 04:42:50.311761 (+ 10us) server_negotiation.cc:407] Connection header received
0810 04:42:50.311796 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:42:50.311842 (+ 46us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:42:50.311861 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:42:50.312102 (+ 241us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 04:42:50.312171 (+ 69us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:42:50.313448 (+ 1277us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:42:50.315371 (+ 1923us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:42:50.316118 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:42:50.316282 (+ 164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:42:50.316359 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0810 04:42:50.316384 (+ 25us) server_negotiation.cc:299] Negotiation successful
0810 04:42:50.316420 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3217,"thread_start_us":140,"threads_started":1}
I20260810 04:42:50.316393 10520 tablet_service.cc:1467] Processing CreateTablet for tablet 4ef91c0d6a9c4554af1e53ebec0bdb55 (DEFAULT_TABLE table=test-table [id=eced672c58f1412cbdb3fad823b2e595]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 04:42:50.316823 10520 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4ef91c0d6a9c4554af1e53ebec0bdb55. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 04:42:50.316881 10659 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:42:50.308831 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.66:36695 (local address 127.0.0.1:60126)
0810 04:42:50.310222 (+ 1391us) negotiation.cc:107] Waiting for socket to connect
0810 04:42:50.310231 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0810 04:42:50.310269 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 04:42:50.311611 (+ 1342us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 04:42:50.311612 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 04:42:50.311614 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 04:42:50.311761 (+ 147us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:42:50.311763 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:42:50.313058 (+ 1295us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:42:50.313059 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:42:50.313907 (+ 848us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:42:50.313910 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:42:50.315802 (+ 1892us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:42:50.315804 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:42:50.315926 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:42:50.315931 (+ 5us) client_negotiation.cc:769] Sending connection context
0810 04:42:50.315979 (+ 48us) client_negotiation.cc:240] Negotiation successful
0810 04:42:50.316834 (+ 855us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1367,"thread_start_us":229,"threads_started":1}
I20260810 04:42:50.317195 10667 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:42:50.309013 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60126 (local address 127.22.167.66:36695)
0810 04:42:50.311372 (+ 2359us) server_negotiation.cc:206] Beginning negotiation
0810 04:42:50.311378 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 04:42:50.311389 (+ 11us) server_negotiation.cc:407] Connection header received
0810 04:42:50.311458 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:42:50.311460 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:42:50.311478 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:42:50.311580 (+ 102us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 04:42:50.311857 (+ 277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:42:50.312991 (+ 1134us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:42:50.314091 (+ 1100us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:42:50.315073 (+ 982us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:42:50.315474 (+ 401us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:42:50.317058 (+ 1584us) server_negotiation.cc:1035] Waiting for connection context
0810 04:42:50.317090 (+ 32us) server_negotiation.cc:299] Negotiation successful
0810 04:42:50.317131 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2309,"thread_start_us":143,"threads_started":1}
I20260810 04:42:50.317232 10057 tablet_service.cc:1467] Processing CreateTablet for tablet 4ef91c0d6a9c4554af1e53ebec0bdb55 (DEFAULT_TABLE table=test-table [id=eced672c58f1412cbdb3fad823b2e595]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 04:42:50.317588 10057 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4ef91c0d6a9c4554af1e53ebec0bdb55. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 04:42:50.318629 10266 tablet_service.cc:1467] Processing CreateTablet for tablet 4ef91c0d6a9c4554af1e53ebec0bdb55 (DEFAULT_TABLE table=test-table [id=eced672c58f1412cbdb3fad823b2e595]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 04:42:50.318962 10266 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4ef91c0d6a9c4554af1e53ebec0bdb55. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 04:42:50.323403 10671 tablet_bootstrap.cc:492] T 4ef91c0d6a9c4554af1e53ebec0bdb55 P 7d82a56f77f0440987f48ef02149151e: Bootstrap starting.
I20260810 04:42:50.325340 10671 tablet_bootstrap.cc:654] T 4ef91c0d6a9c4554af1e53ebec0bdb55 P 7d82a56f77f0440987f48ef02149151e: Neither blocks nor log segments found. Creating new log.
I20260810 04:42:50.325553 10670 tablet_bootstrap.cc:492] T 4ef91c0d6a9c4554af1e53ebec0bdb55 P e899545cb79146848bfadf7a0cdc441c: Bootstrap starting.
I20260810 04:42:50.326352 10669 tablet_bootstrap.cc:492] T 4ef91c0d6a9c4554af1e53ebec0bdb55 P 3b7f04a99439447e8dd21771e5ad2f2c: Bootstrap starting.
I20260810 04:42:50.326462 10671 log.cc:824] T 4ef91c0d6a9c4554af1e53ebec0bdb55 P 7d82a56f77f0440987f48ef02149151e: Log is configured to *not* fsync() on all Append() calls
I20260810 04:42:50.326980 10670 tablet_bootstrap.cc:654] T 4ef91c0d6a9c4554af1e53ebec0bdb55 P e899545cb79146848bfadf7a0cdc441c: Neither blocks nor log segments found. Creating new log.
I20260810 04:42:50.327936 10669 tablet_bootstrap.cc:654] T 4ef91c0d6a9c4554af1e53ebec0bdb55 P 3b7f04a99439447e8dd21771e5ad2f2c: Neither blocks nor log segments found. Creating new log.
I20260810 04:42:50.327937 10670 log.cc:824] T 4ef91c0d6a9c4554af1e53ebec0bdb55 P e899545cb79146848bfadf7a0cdc441c: Log is configured to *not* fsync() on all Append() calls
I20260810 04:42:50.328797 10669 log.cc:824] T 4ef91c0d6a9c4554af1e53ebec0bdb55 P 3b7f04a99439447e8dd21771e5ad2f2c: Log is configured to *not* fsync() on all Append() calls
I20260810 04:42:50.329025 10671 tablet_bootstrap.cc:492] T 4ef91c0d6a9c4554af1e53ebec0bdb55 P 7d82a56f77f0440987f48ef02149151e: No bootstrap required, opened a new log
I20260810 04:42:50.329107 10671 ts_tablet_manager.cc:1397] T 4ef91c0d6a9c4554af1e53ebec0bdb55 P 7d82a56f77f0440987f48ef02149151e: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260810 04:42:50.329598 10669 tablet_bootstrap.cc:492] T 4ef91c0d6a9c4554af1e53ebec0bdb55 P 3b7f04a99439447e8dd21771e5ad2f2c: No bootstrap required, opened a new log
I20260810 04:42:50.329720 10669 ts_tablet_manager.cc:1397] T 4ef91c0d6a9c4554af1e53ebec0bdb55 P 3b7f04a99439447e8dd21771e5ad2f2c: Time spent bootstrapping tablet: real 0.003s user 0.003s sys 0.000s
I20260810 04:42:50.330680 10670 tablet_bootstrap.cc:492] T 4ef91c0d6a9c4554af1e53ebec0bdb55 P e899545cb79146848bfadf7a0cdc441c: No bootstrap required, opened a new log
I20260810 04:42:50.330747 10670 ts_tablet_manager.cc:1397] T 4ef91c0d6a9c4554af1e53ebec0bdb55 P e899545cb79146848bfadf7a0cdc441c: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.003s
I20260810 04:42:50.332512 10671 raft_consensus.cc:348] T 4ef91c0d6a9c4554af1e53ebec0bdb55 P 7d82a56f77f0440987f48ef02149151e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b7f04a99439447e8dd21771e5ad2f2c" member_type: VOTER last_known_addr { host: "127.22.167.67" port: 35955 } } peers { permanent_uuid: "7d82a56f77f0440987f48ef02149151e" member_type: VOTER last_known_addr { host: "127.22.167.66" port: 36695 } } peers { permanent_uuid: "e899545cb79146848bfadf7a0cdc441c" member_type: VOTER last_known_addr { host: "127.22.167.65" port: 32877 } }
I20260810 04:42:50.333011 10671 raft_consensus.cc:374] T 4ef91c0d6a9c4554af1e53ebec0bdb55 P 7d82a56f77f0440987f48ef02149151e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 04:42:50.333052 10671 raft_consensus.cc:729] T 4ef91c0d6a9c4554af1e53ebec0bdb55 P 7d82a56f77f0440987f48ef02149151e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7d82a56f77f0440987f48ef02149151e, State: Initialized, Role: FOLLOWER
I20260810 04:42:50.333181 10671 consensus_queue.cc:260] T 4ef91c0d6a9c4554af1e53ebec0bdb55 P 7d82a56f77f0440987f48ef02149151e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b7f04a99439447e8dd21771e5ad2f2c" member_type: VOTER last_known_addr { host: "127.22.167.67" port: 35955 } } peers { permanent_uuid: "7d82a56f77f0440987f48ef02149151e" member_type: VOTER last_known_addr { host: "127.22.167.66" port: 36695 } } peers { permanent_uuid: "e899545cb79146848bfadf7a0cdc441c" member_type: VOTER last_known_addr { host: "127.22.167.65" port: 32877 } }
I20260810 04:42:50.333449 10671 ts_tablet_manager.cc:1428] T 4ef91c0d6a9c4554af1e53ebec0bdb55 P 7d82a56f77f0440987f48ef02149151e: Time spent starting tablet: real 0.004s user 0.004s sys 0.000s
I20260810 04:42:50.333566 10400 heartbeater.cc:503] Master 127.22.167.126:41675 was elected leader, sending a full tablet report...
I20260810 04:42:50.335696 10670 raft_consensus.cc:348] T 4ef91c0d6a9c4554af1e53ebec0bdb55 P e899545cb79146848bfadf7a0cdc441c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b7f04a99439447e8dd21771e5ad2f2c" member_type: VOTER last_known_addr { host: "127.22.167.67" port: 35955 } } peers { permanent_uuid: "7d82a56f77f0440987f48ef02149151e" member_type: VOTER last_known_addr { host: "127.22.167.66" port: 36695 } } peers { permanent_uuid: "e899545cb79146848bfadf7a0cdc441c" member_type: VOTER last_known_addr { host: "127.22.167.65" port: 32877 } }
I20260810 04:42:50.335914 10670 raft_consensus.cc:374] T 4ef91c0d6a9c4554af1e53ebec0bdb55 P e899545cb79146848bfadf7a0cdc441c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 04:42:50.335958 10670 raft_consensus.cc:729] T 4ef91c0d6a9c4554af1e53ebec0bdb55 P e899545cb79146848bfadf7a0cdc441c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e899545cb79146848bfadf7a0cdc441c, State: Initialized, Role: FOLLOWER
I20260810 04:42:50.336061 10670 consensus_queue.cc:260] T 4ef91c0d6a9c4554af1e53ebec0bdb55 P e899545cb79146848bfadf7a0cdc441c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b7f04a99439447e8dd21771e5ad2f2c" member_type: VOTER last_known_addr { host: "127.22.167.67" port: 35955 } } peers { permanent_uuid: "7d82a56f77f0440987f48ef02149151e" member_type: VOTER last_known_addr { host: "127.22.167.66" port: 36695 } } peers { permanent_uuid: "e899545cb79146848bfadf7a0cdc441c" member_type: VOTER last_known_addr { host: "127.22.167.65" port: 32877 } }
I20260810 04:42:50.336397 10670 ts_tablet_manager.cc:1428] T 4ef91c0d6a9c4554af1e53ebec0bdb55 P e899545cb79146848bfadf7a0cdc441c: Time spent starting tablet: real 0.006s user 0.004s sys 0.001s
I20260810 04:42:50.336582 10176 heartbeater.cc:503] Master 127.22.167.126:41675 was elected leader, sending a full tablet report...
I20260810 04:42:50.349138 10669 raft_consensus.cc:348] T 4ef91c0d6a9c4554af1e53ebec0bdb55 P 3b7f04a99439447e8dd21771e5ad2f2c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b7f04a99439447e8dd21771e5ad2f2c" member_type: VOTER last_known_addr { host: "127.22.167.67" port: 35955 } } peers { permanent_uuid: "7d82a56f77f0440987f48ef02149151e" member_type: VOTER last_known_addr { host: "127.22.167.66" port: 36695 } } peers { permanent_uuid: "e899545cb79146848bfadf7a0cdc441c" member_type: VOTER last_known_addr { host: "127.22.167.65" port: 32877 } }
I20260810 04:42:50.349352 10669 raft_consensus.cc:374] T 4ef91c0d6a9c4554af1e53ebec0bdb55 P 3b7f04a99439447e8dd21771e5ad2f2c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 04:42:50.349407 10669 raft_consensus.cc:729] T 4ef91c0d6a9c4554af1e53ebec0bdb55 P 3b7f04a99439447e8dd21771e5ad2f2c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3b7f04a99439447e8dd21771e5ad2f2c, State: Initialized, Role: FOLLOWER
I20260810 04:42:50.349534 10669 consensus_queue.cc:260] T 4ef91c0d6a9c4554af1e53ebec0bdb55 P 3b7f04a99439447e8dd21771e5ad2f2c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b7f04a99439447e8dd21771e5ad2f2c" member_type: VOTER last_known_addr { host: "127.22.167.67" port: 35955 } } peers { permanent_uuid: "7d82a56f77f0440987f48ef02149151e" member_type: VOTER last_known_addr { host: "127.22.167.66" port: 36695 } } peers { permanent_uuid: "e899545cb79146848bfadf7a0cdc441c" member_type: VOTER last_known_addr { host: "127.22.167.65" port: 32877 } }
I20260810 04:42:50.349924 10669 ts_tablet_manager.cc:1428] T 4ef91c0d6a9c4554af1e53ebec0bdb55 P 3b7f04a99439447e8dd21771e5ad2f2c: Time spent starting tablet: real 0.020s user 0.001s sys 0.004s
I20260810 04:42:50.350029 10636 heartbeater.cc:503] Master 127.22.167.126:41675 was elected leader, sending a full tablet report...
W20260810 04:42:50.430204 10177 tablet.cc:2369] T 4ef91c0d6a9c4554af1e53ebec0bdb55 P e899545cb79146848bfadf7a0cdc441c: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 10 04:42:50 dist-test-slave-bqcl krb5kdc[9055](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786336969, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.167.126@KRBTEST.COM for HTTP/127.22.167.84@KRBTEST.COM
W20260810 04:42:50.492238 10402 tablet.cc:2369] T 4ef91c0d6a9c4554af1e53ebec0bdb55 P 7d82a56f77f0440987f48ef02149151e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 04:42:50.500391 10683 raft_consensus.cc:482] T 4ef91c0d6a9c4554af1e53ebec0bdb55 P 3b7f04a99439447e8dd21771e5ad2f2c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 04:42:50.500538 10683 raft_consensus.cc:504] T 4ef91c0d6a9c4554af1e53ebec0bdb55 P 3b7f04a99439447e8dd21771e5ad2f2c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b7f04a99439447e8dd21771e5ad2f2c" member_type: VOTER last_known_addr { host: "127.22.167.67" port: 35955 } } peers { permanent_uuid: "7d82a56f77f0440987f48ef02149151e" member_type: VOTER last_known_addr { host: "127.22.167.66" port: 36695 } } peers { permanent_uuid: "e899545cb79146848bfadf7a0cdc441c" member_type: VOTER last_known_addr { host: "127.22.167.65" port: 32877 } }
I20260810 04:42:50.501092 10683 leader_election.cc:288] T 4ef91c0d6a9c4554af1e53ebec0bdb55 P 3b7f04a99439447e8dd21771e5ad2f2c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7d82a56f77f0440987f48ef02149151e (127.22.167.66:36695), e899545cb79146848bfadf7a0cdc441c (127.22.167.65:32877)
I20260810 04:42:50.505010 10638 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:42:50.501299 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.66:36695 (local address 127.22.167.67:35781)
0810 04:42:50.501395 (+ 96us) negotiation.cc:107] Waiting for socket to connect
0810 04:42:50.501407 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0810 04:42:50.501452 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 04:42:50.501691 (+ 239us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 04:42:50.501692 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 04:42:50.501694 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 04:42:50.501878 (+ 184us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:42:50.501881 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:42:50.502940 (+ 1059us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:42:50.502941 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:42:50.504050 (+ 1109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:42:50.504054 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:42:50.504770 (+ 716us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:42:50.504771 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:42:50.504852 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:42:50.504856 (+ 4us) client_negotiation.cc:769] Sending connection context
0810 04:42:50.504896 (+ 40us) client_negotiation.cc:240] Negotiation successful
0810 04:42:50.504935 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":74}
I20260810 04:42:50.505659 10667 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:42:50.501351 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.167.67:35781 (local address 127.22.167.66:36695)
0810 04:42:50.501532 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0810 04:42:50.501534 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 04:42:50.501542 (+ 8us) server_negotiation.cc:407] Connection header received
0810 04:42:50.501570 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:42:50.501571 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:42:50.501583 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:42:50.501633 (+ 50us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 04:42:50.501970 (+ 337us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:42:50.502757 (+ 787us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:42:50.504156 (+ 1399us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:42:50.504661 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:42:50.504727 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:42:50.504736 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0810 04:42:50.505542 (+ 806us) server_negotiation.cc:299] Negotiation successful
0810 04:42:50.505594 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":158}
I20260810 04:42:50.505940 10332 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4ef91c0d6a9c4554af1e53ebec0bdb55" candidate_uuid: "3b7f04a99439447e8dd21771e5ad2f2c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7d82a56f77f0440987f48ef02149151e" is_pre_election: true
I20260810 04:42:50.506075 10332 raft_consensus.cc:2454] T 4ef91c0d6a9c4554af1e53ebec0bdb55 P 7d82a56f77f0440987f48ef02149151e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b7f04a99439447e8dd21771e5ad2f2c in term 0.
I20260810 04:42:50.506646 10444 leader_election.cc:302] T 4ef91c0d6a9c4554af1e53ebec0bdb55 P 3b7f04a99439447e8dd21771e5ad2f2c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b7f04a99439447e8dd21771e5ad2f2c, 7d82a56f77f0440987f48ef02149151e; no voters:
I20260810 04:42:50.506940 10683 raft_consensus.cc:2781] T 4ef91c0d6a9c4554af1e53ebec0bdb55 P 3b7f04a99439447e8dd21771e5ad2f2c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 04:42:50.507000 10683 raft_consensus.cc:482] T 4ef91c0d6a9c4554af1e53ebec0bdb55 P 3b7f04a99439447e8dd21771e5ad2f2c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 04:42:50.507020 10683 raft_consensus.cc:3037] T 4ef91c0d6a9c4554af1e53ebec0bdb55 P 3b7f04a99439447e8dd21771e5ad2f2c [term 0 FOLLOWER]: Advancing to term 1
I20260810 04:42:50.508184 10683 raft_consensus.cc:504] T 4ef91c0d6a9c4554af1e53ebec0bdb55 P 3b7f04a99439447e8dd21771e5ad2f2c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b7f04a99439447e8dd21771e5ad2f2c" member_type: VOTER last_known_addr { host: "127.22.167.67" port: 35955 } } peers { permanent_uuid: "7d82a56f77f0440987f48ef02149151e" member_type: VOTER last_known_addr { host: "127.22.167.66" port: 36695 } } peers { permanent_uuid: "e899545cb79146848bfadf7a0cdc441c" member_type: VOTER last_known_addr { host: "127.22.167.65" port: 32877 } }
I20260810 04:42:50.508301 10683 leader_election.cc:288] T 4ef91c0d6a9c4554af1e53ebec0bdb55 P 3b7f04a99439447e8dd21771e5ad2f2c [CANDIDATE]: Term 1 election: Requested vote from peers 7d82a56f77f0440987f48ef02149151e (127.22.167.66:36695), e899545cb79146848bfadf7a0cdc441c (127.22.167.65:32877)
I20260810 04:42:50.508512 10332 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4ef91c0d6a9c4554af1e53ebec0bdb55" candidate_uuid: "3b7f04a99439447e8dd21771e5ad2f2c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7d82a56f77f0440987f48ef02149151e"
I20260810 04:42:50.508607 10332 raft_consensus.cc:3037] T 4ef91c0d6a9c4554af1e53ebec0bdb55 P 7d82a56f77f0440987f48ef02149151e [term 0 FOLLOWER]: Advancing to term 1
I20260810 04:42:50.509106 10658 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:42:50.501982 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.167.67:55163 (local address 127.22.167.65:32877)
0810 04:42:50.502976 (+ 994us) server_negotiation.cc:206] Beginning negotiation
0810 04:42:50.502978 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 04:42:50.502985 (+ 7us) server_negotiation.cc:407] Connection header received
0810 04:42:50.503008 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:42:50.503009 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:42:50.503017 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:42:50.503061 (+ 44us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 04:42:50.505344 (+ 2283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:42:50.506048 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:42:50.507865 (+ 1817us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:42:50.508647 (+ 782us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:42:50.508693 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:42:50.508702 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0810 04:42:50.508986 (+ 284us) server_negotiation.cc:299] Negotiation successful
0810 04:42:50.509032 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":966}
I20260810 04:42:50.509462 10685 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:42:50.501500 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.65:32877 (local address 127.22.167.67:55163)
0810 04:42:50.502267 (+ 767us) negotiation.cc:107] Waiting for socket to connect
0810 04:42:50.502278 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0810 04:42:50.502554 (+ 276us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 04:42:50.504930 (+ 2376us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 04:42:50.504931 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 04:42:50.504933 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 04:42:50.505256 (+ 323us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:42:50.505259 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:42:50.506180 (+ 921us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:42:50.506181 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:42:50.507766 (+ 1585us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:42:50.507768 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:42:50.508762 (+ 994us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:42:50.508763 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:42:50.508894 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:42:50.508899 (+ 5us) client_negotiation.cc:769] Sending connection context
0810 04:42:50.509177 (+ 278us) client_negotiation.cc:240] Negotiation successful
0810 04:42:50.509222 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":725,"thread_start_us":369,"threads_started":1}
I20260810 04:42:50.509517 10332 raft_consensus.cc:2454] T 4ef91c0d6a9c4554af1e53ebec0bdb55 P 7d82a56f77f0440987f48ef02149151e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3b7f04a99439447e8dd21771e5ad2f2c in term 1.
I20260810 04:42:50.509675 10444 leader_election.cc:302] T 4ef91c0d6a9c4554af1e53ebec0bdb55 P 3b7f04a99439447e8dd21771e5ad2f2c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b7f04a99439447e8dd21771e5ad2f2c, 7d82a56f77f0440987f48ef02149151e; no voters:
I20260810 04:42:50.509869 10093 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4ef91c0d6a9c4554af1e53ebec0bdb55" candidate_uuid: "3b7f04a99439447e8dd21771e5ad2f2c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e899545cb79146848bfadf7a0cdc441c"
I20260810 04:42:50.509994 10093 raft_consensus.cc:3037] T 4ef91c0d6a9c4554af1e53ebec0bdb55 P e899545cb79146848bfadf7a0cdc441c [term 0 FOLLOWER]: Advancing to term 1
I20260810 04:42:50.509951 10096 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4ef91c0d6a9c4554af1e53ebec0bdb55" candidate_uuid: "3b7f04a99439447e8dd21771e5ad2f2c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e899545cb79146848bfadf7a0cdc441c" is_pre_election: true
I20260810 04:42:50.511054 10093 raft_consensus.cc:2454] T 4ef91c0d6a9c4554af1e53ebec0bdb55 P e899545cb79146848bfadf7a0cdc441c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3b7f04a99439447e8dd21771e5ad2f2c in term 1.
I20260810 04:42:50.511708 10683 raft_consensus.cc:2781] T 4ef91c0d6a9c4554af1e53ebec0bdb55 P 3b7f04a99439447e8dd21771e5ad2f2c [term 1 FOLLOWER]: Leader election won for term 1
I20260810 04:42:50.511838 10683 raft_consensus.cc:686] T 4ef91c0d6a9c4554af1e53ebec0bdb55 P 3b7f04a99439447e8dd21771e5ad2f2c [term 1 LEADER]: Becoming Leader. State: Replica: 3b7f04a99439447e8dd21771e5ad2f2c, State: Running, Role: LEADER
I20260810 04:42:50.511914 10683 consensus_queue.cc:237] T 4ef91c0d6a9c4554af1e53ebec0bdb55 P 3b7f04a99439447e8dd21771e5ad2f2c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b7f04a99439447e8dd21771e5ad2f2c" member_type: VOTER last_known_addr { host: "127.22.167.67" port: 35955 } } peers { permanent_uuid: "7d82a56f77f0440987f48ef02149151e" member_type: VOTER last_known_addr { host: "127.22.167.66" port: 36695 } } peers { permanent_uuid: "e899545cb79146848bfadf7a0cdc441c" member_type: VOTER last_known_addr { host: "127.22.167.65" port: 32877 } }
I20260810 04:42:50.512730 9821 catalog_manager.cc:5495] T 4ef91c0d6a9c4554af1e53ebec0bdb55 P 3b7f04a99439447e8dd21771e5ad2f2c reported cstate change: term changed from 0 to 1, leader changed from <none> to 3b7f04a99439447e8dd21771e5ad2f2c (127.22.167.67). New cstate: current_term: 1 leader_uuid: "3b7f04a99439447e8dd21771e5ad2f2c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b7f04a99439447e8dd21771e5ad2f2c" member_type: VOTER last_known_addr { host: "127.22.167.67" port: 35955 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7d82a56f77f0440987f48ef02149151e" member_type: VOTER last_known_addr { host: "127.22.167.66" port: 36695 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e899545cb79146848bfadf7a0cdc441c" member_type: VOTER last_known_addr { host: "127.22.167.65" port: 32877 } health_report { overall_health: UNKNOWN } } }
W20260810 04:42:50.523981 10637 tablet.cc:2369] T 4ef91c0d6a9c4554af1e53ebec0bdb55 P 3b7f04a99439447e8dd21771e5ad2f2c: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 04:42:50.540666 10654 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:42:50.538342 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53070 (local address 127.22.167.67:35955)
0810 04:42:50.538459 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0810 04:42:50.538461 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 04:42:50.538466 (+ 5us) server_negotiation.cc:407] Connection header received
0810 04:42:50.538494 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:42:50.538495 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:42:50.538511 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:42:50.538568 (+ 57us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 04:42:50.538754 (+ 186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:42:50.539222 (+ 468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:42:50.539946 (+ 724us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:42:50.540151 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:42:50.540206 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:42:50.540354 (+ 148us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 04:42:50.540454 (+ 100us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 04:42:50.540525 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0810 04:42:50.540563 (+ 38us) server_negotiation.cc:299] Negotiation successful
0810 04:42:50.540594 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":99}
I20260810 04:42:50.542723 10093 raft_consensus.cc:1261] T 4ef91c0d6a9c4554af1e53ebec0bdb55 P e899545cb79146848bfadf7a0cdc441c [term 1 FOLLOWER]: Refusing update from remote peer 3b7f04a99439447e8dd21771e5ad2f2c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 04:42:50.542977 10332 raft_consensus.cc:1261] T 4ef91c0d6a9c4554af1e53ebec0bdb55 P 7d82a56f77f0440987f48ef02149151e [term 1 FOLLOWER]: Refusing update from remote peer 3b7f04a99439447e8dd21771e5ad2f2c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 04:42:50.543396 10686 consensus_queue.cc:1035] T 4ef91c0d6a9c4554af1e53ebec0bdb55 P 3b7f04a99439447e8dd21771e5ad2f2c [LEADER]: Connected to new peer: Peer: permanent_uuid: "e899545cb79146848bfadf7a0cdc441c" member_type: VOTER last_known_addr { host: "127.22.167.65" port: 32877 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 04:42:50.543509 10683 consensus_queue.cc:1035] T 4ef91c0d6a9c4554af1e53ebec0bdb55 P 3b7f04a99439447e8dd21771e5ad2f2c [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d82a56f77f0440987f48ef02149151e" member_type: VOTER last_known_addr { host: "127.22.167.66" port: 36695 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 04:42:50.547109 10696 mvcc.cc:205] Tried to move back new op lower bound from 7316836231340335104 to 7316836231217348608. Current Snapshot: MvccSnapshot[applied={T|T < 7316836231340335104}]
I20260810 04:42:50.549733 9821 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50816:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 04:42:50.549822 9821 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50816:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 04:42:50.550668 9821 catalog_manager.cc:5782] T 00000000000000000000000000000000 P ecf86695fb764288b300a8048b21bba1: Sending DeleteTablet for 3 replicas of tablet 4ef91c0d6a9c4554af1e53ebec0bdb55
I20260810 04:42:50.551070 10057 tablet_service.cc:1514] Processing DeleteTablet for tablet 4ef91c0d6a9c4554af1e53ebec0bdb55 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 04:42:50 UTC) from {username='kudu', principal='kudu/127.22.167.126@KRBTEST.COM'} at 127.0.0.1:56882
I20260810 04:42:50.551762 10520 tablet_service.cc:1514] Processing DeleteTablet for tablet 4ef91c0d6a9c4554af1e53ebec0bdb55 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 04:42:50 UTC) from {username='kudu', principal='kudu/127.22.167.126@KRBTEST.COM'} at 127.0.0.1:53060
I20260810 04:42:50.551776 10266 tablet_service.cc:1514] Processing DeleteTablet for tablet 4ef91c0d6a9c4554af1e53ebec0bdb55 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 04:42:50 UTC) from {username='kudu', principal='kudu/127.22.167.126@KRBTEST.COM'} at 127.0.0.1:60126
I20260810 04:42:50.552229 23197 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskemv_AM/build/release/bin/kudu with pid 9957
I20260810 04:42:50.569371 10698 mvcc.cc:205] Tried to move back new op lower bound from 7316836231340335104 to 7316836231217348608. Current Snapshot: MvccSnapshot[applied={T|T < 7316836231340335104}]
I20260810 04:42:50.570328 10713 tablet_replica.cc:330] T 4ef91c0d6a9c4554af1e53ebec0bdb55 P 7d82a56f77f0440987f48ef02149151e: stopping tablet replica
I20260810 04:42:50.570426 10713 raft_consensus.cc:2229] T 4ef91c0d6a9c4554af1e53ebec0bdb55 P 7d82a56f77f0440987f48ef02149151e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 04:42:50.570480 10713 raft_consensus.cc:2258] T 4ef91c0d6a9c4554af1e53ebec0bdb55 P 7d82a56f77f0440987f48ef02149151e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 04:42:50.577176 10713 ts_tablet_manager.cc:1905] T 4ef91c0d6a9c4554af1e53ebec0bdb55 P 7d82a56f77f0440987f48ef02149151e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 04:42:50.579655 10713 ts_tablet_manager.cc:1918] T 4ef91c0d6a9c4554af1e53ebec0bdb55 P 7d82a56f77f0440987f48ef02149151e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 04:42:50.579715 10713 log.cc:1196] T 4ef91c0d6a9c4554af1e53ebec0bdb55 P 7d82a56f77f0440987f48ef02149151e: Deleting WAL directory at /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786336863388938-23197-0/minicluster-data/ts-1/wal/wals/4ef91c0d6a9c4554af1e53ebec0bdb55
I20260810 04:42:50.579907 10713 ts_tablet_manager.cc:1939] T 4ef91c0d6a9c4554af1e53ebec0bdb55 P 7d82a56f77f0440987f48ef02149151e: Deleting consensus metadata
I20260810 04:42:50.580258 9788 catalog_manager.cc:4841] TS 7d82a56f77f0440987f48ef02149151e (127.22.167.66:36695): tablet 4ef91c0d6a9c4554af1e53ebec0bdb55 (table test-table [id=eced672c58f1412cbdb3fad823b2e595]) successfully deleted
I20260810 04:42:50.588677 10712 tablet_replica.cc:330] T 4ef91c0d6a9c4554af1e53ebec0bdb55 P 3b7f04a99439447e8dd21771e5ad2f2c: stopping tablet replica
I20260810 04:42:50.588797 10712 raft_consensus.cc:2229] T 4ef91c0d6a9c4554af1e53ebec0bdb55 P 3b7f04a99439447e8dd21771e5ad2f2c [term 1 LEADER]: Raft consensus shutting down.
I20260810 04:42:50.588891 10712 raft_consensus.cc:2258] T 4ef91c0d6a9c4554af1e53ebec0bdb55 P 3b7f04a99439447e8dd21771e5ad2f2c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 04:42:50.589352 10712 ts_tablet_manager.cc:1905] T 4ef91c0d6a9c4554af1e53ebec0bdb55 P 3b7f04a99439447e8dd21771e5ad2f2c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 04:42:50.591061 10712 ts_tablet_manager.cc:1918] T 4ef91c0d6a9c4554af1e53ebec0bdb55 P 3b7f04a99439447e8dd21771e5ad2f2c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 04:42:50.591146 10712 log.cc:1196] T 4ef91c0d6a9c4554af1e53ebec0bdb55 P 3b7f04a99439447e8dd21771e5ad2f2c: Deleting WAL directory at /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786336863388938-23197-0/minicluster-data/ts-2/wal/wals/4ef91c0d6a9c4554af1e53ebec0bdb55
I20260810 04:42:50.591413 10712 ts_tablet_manager.cc:1939] T 4ef91c0d6a9c4554af1e53ebec0bdb55 P 3b7f04a99439447e8dd21771e5ad2f2c: Deleting consensus metadata
I20260810 04:42:50.591847 9788 catalog_manager.cc:4841] TS 3b7f04a99439447e8dd21771e5ad2f2c (127.22.167.67:35955): tablet 4ef91c0d6a9c4554af1e53ebec0bdb55 (table test-table [id=eced672c58f1412cbdb3fad823b2e595]) successfully deleted
W20260810 04:42:50.596820 10450 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.22.167.65:32877 (error 108)
W20260810 04:42:50.596863 9792 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.22.167.65:32877 (error 108)
I20260810 04:42:50.597541 23197 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskemv_AM/build/release/bin/kudu with pid 10185
I20260810 04:42:50.598177 10659 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:42:50.597656 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.65:32877 (local address 127.0.0.1:56890)
0810 04:42:50.598058 (+ 402us) negotiation.cc:107] Waiting for socket to connect
0810 04:42:50.598094 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.167.65:32877: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":385}
W20260810 04:42:50.599583 9792 catalog_manager.cc:4584] TS e899545cb79146848bfadf7a0cdc441c (127.22.167.65:32877): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 4ef91c0d6a9c4554af1e53ebec0bdb55: Network error: Client connection negotiation failed: client connection to 127.22.167.65:32877: connect: Connection refused (error 111)
I20260810 04:42:50.623792 23197 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskemv_AM/build/release/bin/kudu with pid 10413
I20260810 04:42:50.632304 23197 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskemv_AM/build/release/bin/kudu with pid 9764
I20260810 04:42:50.643782 23197 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260810 04:42:50.986937 23197 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260810 04:42:50.987048 23197 mini_ranger.cc:66] Stopping Ranger...
I20260810 04:42:51.330303 23197 mini_ranger.cc:68] Stopped Ranger
2026-08-10 04:42:51.330 UTC [9106] LOG: received smart shutdown request
2026-08-10 04:42:51.333 UTC [9106] LOG: background worker "logical replication launcher" (PID 9118) exited with exit code 1
2026-08-10 04:42:51.334 UTC [9110] LOG: shutting down
2026-08-10 04:42:51.442 UTC [9106] LOG: database system is shut down
2026-08-10T04:42:51Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegration (47922 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers
Loading random data
Initializing database '/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786336863388938-23197-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 04:42:51 dist-test-slave-bqcl krb5kdc[10734](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 04:42:51 dist-test-slave-bqcl krb5kdc[10734](info): set up 2 sockets
Aug 10 04:42:51 dist-test-slave-bqcl krb5kdc[10734](info): commencing operation
krb5kdc: starting...
W20260810 04:42:52.715814 23197 mini_kdc.cc:121] Time spent starting KDC: real 1.163s user 0.001s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786336863388938-23197-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786336863388938-23197-0/krb5kdc/test-user.keytab.
Aug 10 04:42:52 dist-test-slave-bqcl krb5kdc[10734](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786336972, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T04:42:52Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T04:42:52Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.22.167.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.22.167.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.22.167.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786336863388938-23197-0/krb5kdc/rangeradmin_127.22.167.84@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.22.167.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786336863388938-23197-0/krb5kdc/rangeradmin_127.22.167.84@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.22.167.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.22.167.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.22.167.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786336863388938-23197-0/krb5kdc/rangerlookup_127.22.167.84@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.22.167.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786336863388938-23197-0/krb5kdc/rangerlookup_127.22.167.84@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.22.167.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.22.167.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.22.167.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786336863388938-23197-0/krb5kdc/HTTP_127.22.167.84@KRBTEST.COM.keytab.
Entry for principal HTTP/127.22.167.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786336863388938-23197-0/krb5kdc/HTTP_127.22.167.84@KRBTEST.COM.keytab.
I20260810 04:42:52.872244 23197 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786336863388938-23197-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskemv_AM/build/release/bin/postgres/pg_ctl -D /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786336863388938-23197-0/minicluster-data/postgres -l logfile start
2026-08-10 04:42:55.076 UTC [10783] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-10 04:42:55.076 UTC [10783] LOG: listening on IPv4 address "127.22.167.84", port 57725
2026-08-10 04:42:55.079 UTC [10783] LOG: listening on Unix socket "/tmp/.s.PGSQL.57725"
2026-08-10 04:42:55.103 UTC [10788] LOG: database system was shut down at 2026-08-10 04:42:53 UTC
2026-08-10 04:42:55.107 UTC [10783] LOG: database system is ready to accept connections
I20260810 04:42:56.172956 23197 mini_postgres.cc:96] Postgres bound to 57725
2026-08-10 04:42:56.179 UTC [10808] FATAL: database "slave" does not exist
127.22.167.84:57725 - accepting connections
I20260810 04:42:56.180514 23197 mini_ranger.cc:161] Starting Ranger...
I20260810 04:42:56.195430 23197 mini_ranger.cc:84] Created miniranger Postgres user
I20260810 04:42:56.599509 23197 mini_ranger.cc:87] Created ranger Postgres database
I20260810 04:42:56.599583 23197 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786336863388938-23197-0/minicluster-data/ranger-admin
2026-08-10 04:42:56,877 [I] DB FLAVOR :POSTGRES
2026-08-10 04:42:56,877 [I] --------- Verifying Ranger DB connection ---------
2026-08-10 04:42:56,877 [I] Checking connection..
2026-08-10 04:42:56,877 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskemv_AM/build/release/bin/postgresql.jar:/tmp/dist-test-taskemv_AM/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.167.84:57725/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-10 04:42:57,134 [I] Checking connection passed.
2026-08-10 04:42:57,134 [I] --------- Verifying version history table ---------
2026-08-10 04:42:57,134 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskemv_AM/build/release/bin/postgresql.jar:/tmp/dist-test-taskemv_AM/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.167.84:57725/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-10 04:42:57,391 [I] Table x_db_version_h does not exist in database ranger
2026-08-10 04:42:57,392 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskemv_AM/build/release/bin/postgresql.jar:/tmp/dist-test-taskemv_AM/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.167.84:57725/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-10 04:42:57,650 [I] Table x_db_version_h does not exist in database ranger
2026-08-10 04:42:57,650 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-10 04:42:57,650 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskemv_AM/build/release/bin/postgresql.jar:/tmp/dist-test-taskemv_AM/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.167.84:57725/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskemv_AM/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-10 04:42:57.879 UTC [10925] WARNING: there is no transaction in progress
2026-08-10 04:42:57,887 [I] create_dbversion_catalog.sql file imported successfully
2026-08-10 04:42:57,887 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskemv_AM/build/release/bin/postgresql.jar:/tmp/dist-test-taskemv_AM/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.167.84:57725/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-10 04:42:58,143 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-10 04:42:58,143 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-10 04:42:58,143 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskemv_AM/build/release/bin/postgresql.jar:/tmp/dist-test-taskemv_AM/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.167.84:57725/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-10 04:42:58,397 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskemv_AM/build/release/bin/postgresql.jar:/tmp/dist-test-taskemv_AM/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.167.84:57725/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-10 04:42:58,652 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskemv_AM/build/release/bin/postgresql.jar:/tmp/dist-test-taskemv_AM/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.167.84:57725/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-bqcl.c.gcp-upstream.internal','N') ;"
2026-08-10 04:42:58,888 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskemv_AM/build/release/bin/postgresql.jar:/tmp/dist-test-taskemv_AM/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.167.84:57725/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-10 04:42:59,124 [I] Table x_portal_user does not exist in database ranger
2026-08-10 04:42:59,124 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskemv_AM/build/release/bin/postgresql.jar:/tmp/dist-test-taskemv_AM/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.167.84:57725/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-10 04:42:59,367 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-10 04:42:59,367 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskemv_AM/build/release/bin/postgresql.jar:/tmp/dist-test-taskemv_AM/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.167.84:57725/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-10 04:42:59,595 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskemv_AM/build/release/bin/postgresql.jar:/tmp/dist-test-taskemv_AM/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.167.84:57725/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-10 04:42:59,829 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-10 04:42:59,829 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskemv_AM/build/release/bin/postgresql.jar:/tmp/dist-test-taskemv_AM/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.167.84:57725/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskemv_AM/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-10 04:43:00.755 UTC [11093] WARNING: there is no transaction in progress
2026-08-10 04:43:00.770 UTC [11093] WARNING: there is no transaction in progress
2026-08-10 04:43:00.785 UTC [11093] WARNING: there is no transaction in progress
2026-08-10 04:43:00.799 UTC [11093] WARNING: there is no transaction in progress
2026-08-10 04:43:00.813 UTC [11093] WARNING: there is no transaction in progress
2026-08-10 04:43:00.829 UTC [11093] WARNING: there is no transaction in progress
2026-08-10 04:43:00.889 UTC [11093] WARNING: there is no transaction in progress
2026-08-10 04:43:00.896 UTC [11093] WARNING: there is no transaction in progress
2026-08-10 04:43:00.909 UTC [11093] WARNING: there is no transaction in progress
2026-08-10 04:43:00.919 UTC [11093] WARNING: there is no transaction in progress
2026-08-10 04:43:00.928 UTC [11093] WARNING: there is no transaction in progress
2026-08-10 04:43:00.940 UTC [11093] WARNING: there is no transaction in progress
2026-08-10 04:43:00.949 UTC [11093] WARNING: there is no transaction in progress
2026-08-10 04:43:00.955 UTC [11093] WARNING: there is no transaction in progress
2026-08-10 04:43:00.961 UTC [11093] WARNING: there is no transaction in progress
2026-08-10 04:43:00.967 UTC [11093] WARNING: there is no transaction in progress
2026-08-10 04:43:01,560 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-10 04:43:01,560 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskemv_AM/build/release/bin/postgresql.jar:/tmp/dist-test-taskemv_AM/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.167.84:57725/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-10 04:43:01,810 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-10 04:43:01,811 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskemv_AM/build/release/bin/postgresql.jar:/tmp/dist-test-taskemv_AM/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.167.84:57725/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-10 04:43:02,069 [I] Table x_portal_user already exists in database 'ranger'
2026-08-10 04:43:02,069 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskemv_AM/build/release/bin/postgresql.jar:/tmp/dist-test-taskemv_AM/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.167.84:57725/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-10 04:43:02,325 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-10 04:43:02,325 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskemv_AM/build/release/bin/postgresql.jar:/tmp/dist-test-taskemv_AM/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.167.84:57725/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-10 04:43:02,579 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskemv_AM/build/release/bin/postgresql.jar:/tmp/dist-test-taskemv_AM/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.167.84:57725/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-10 04:43:02,826 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskemv_AM/build/release/bin/postgresql.jar:/tmp/dist-test-taskemv_AM/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.167.84:57725/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-bqcl.c.gcp-upstream.internal';"
2026-08-10 04:43:03,076 [I] CORE_DB_SCHEMA import status has been updated
2026-08-10 04:43:03,076 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskemv_AM/build/release/bin/postgresql.jar:/tmp/dist-test-taskemv_AM/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.167.84:57725/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-10 04:43:03,309 [I] DB_PATCHES have already been applied
I20260810 04:43:03.315178 23197 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786336863388938-23197-0/minicluster-data/ranger-admin:/tmp/dist-test-taskemv_AM/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskemv_AM/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskemv_AM/build/release/bin/postgresql.jar:/tmp/dist-test-taskemv_AM/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260810 04:43:03.315255 23197 mini_ranger.cc:193] Using host: 127.22.167.84
I20260810 04:43:03.319259 23197 mini_ranger.cc:227] Ranger admin URL: http://127.22.167.84:43119
Aug 10, 2026 4:43:03 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 10, 2026 4:43:03 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 10, 2026 4:43:03 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786336863388938-23197-0/minicluster-data/ranger-admin/ews/webapp
Aug 10, 2026 4:43:03 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786336863388938-23197-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 10, 2026 4:43:03 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786336863388938-23197-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 10, 2026 4:43:04 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786336863388938-23197-0/minicluster-data/ranger-admin/ews/webapp].
Aug 10, 2026 4:43:04 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260810 04:43:04.537882 23197 mini_ranger.cc:160] Time spent starting Ranger: real 8.357s user 0.002s sys 0.007s
Aug 10, 2026 4:43:23 AM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 10, 2026 4:43:23 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
Aug 10, 2026 4:43:23 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 10, 2026 4:43:23 AM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 10, 2026 4:43:23 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 10, 2026 4:43:23 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 10, 2026 4:43:23 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 10, 2026 4:43:23 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 10, 2026 4:43:23 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 10, 2026 4:43:23 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 10, 2026 4:43:23 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 10, 2026 4:43:23 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 10, 2026 4:43:23 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 10, 2026 4:43:23 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 10, 2026 4:43:23 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 10, 2026 4:43:23 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 10, 2026 4:43:23 AM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 10, 2026 4:43:24 AM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260810 04:43:25.387910 23197 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.22.167.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.22.167.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.22.167.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786336863388938-23197-0/krb5kdc/rangerkms_127.22.167.84@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.22.167.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786336863388938-23197-0/krb5kdc/rangerkms_127.22.167.84@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.22.167.84@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.22.167.84@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.22.167.84@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786336863388938-23197-0/krb5kdc/HTTP_127.22.167.84@KRBTEST.COM.keytab.
Entry for principal HTTP/127.22.167.84@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786336863388938-23197-0/krb5kdc/HTTP_127.22.167.84@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 10 04:43:25 dist-test-slave-bqcl krb5kdc[10734](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786337005, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260810 04:43:25.486521 23197 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260810 04:43:25.497141 23197 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260810 04:43:26.234905 23197 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260810 04:43:26.234978 23197 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786336863388938-23197-0/minicluster-data/ranger-kms
I20260810 04:43:26.234997 23197 mini_ranger_kms.cc:227] Using postgres at 127.22.167.84:57725
2026-08-10 04:43:26,379 [I] DB FLAVOR :POSTGRES
2026-08-10 04:43:26,380 [I] --------- Verifying Ranger DB connection ---------
2026-08-10 04:43:26,380 [I] Checking connection
2026-08-10 04:43:26,380 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskemv_AM/build/release/bin/postgresql.jar:/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786336863388938-23197-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.167.84:57725/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-10 04:43:26,610 [I] connection success
2026-08-10 04:43:26,610 [I] --------- Verifying Ranger DB tables ---------
2026-08-10 04:43:26,610 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-10 04:43:26,611 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskemv_AM/build/release/bin/postgresql.jar:/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786336863388938-23197-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.167.84:57725/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-10 04:43:26,853 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-10 04:43:26,853 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-10 04:43:26,853 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-10 04:43:26,853 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskemv_AM/build/release/bin/postgresql.jar:/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786336863388938-23197-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.167.84:57725/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786336863388938-23197-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-10 04:43:27,100 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260810 04:43:27.537353 23197 mini_ranger_kms.cc:310] Created kms service
I20260810 04:43:27.699965 23197 mini_ranger_kms.cc:326] Created kudu user
I20260810 04:43:27.740365 23197 mini_ranger_kms.cc:343] Created rangerkms user
Aug 10 04:43:27 dist-test-slave-bqcl krb5kdc[10734](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786337005, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.22.167.84@KRBTEST.COM
I20260810 04:43:28.141093 23197 mini_ranger_kms.cc:384] Added ranger policy
I20260810 04:43:28.141209 23197 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786336863388938-23197-0/minicluster-data/ranger-kms:/tmp/dist-test-taskemv_AM/build/release/bin/postgresql.jar:/tmp/dist-test-taskemv_AM/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskemv_AM/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260810 04:43:28.141227 23197 mini_ranger_kms.cc:240] Using host: 127.22.167.84
I20260810 04:43:28.146562 23197 mini_ranger_kms.cc:276] Ranger KMS PID: 11380
I20260810 04:43:28.146636 23197 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.22.167.84:39545
Aug 10, 2026 4:43:29 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 10, 2026 4:43:29 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 10, 2026 4:43:29 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786336863388938-23197-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 10, 2026 4:43:29 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786336863388938-23197-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 10, 2026 4:43:29 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786336863388938-23197-0/minicluster-data/ranger-kms/ews/webapp].
Aug 10, 2026 4:43:29 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
Aug 10 04:43:30 dist-test-slave-bqcl krb5kdc[10734](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786337010, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.22.167.84@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 04:43:30.677296 23197 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 5.191s user 0.004s sys 0.125s
I20260810 04:43:30.677412 23197 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260810 04:43:30.677454 23197 mini_ranger_kms.cc:401] 127.22.167.84:39545/kms/v1/keys
Aug 10 04:43:30 dist-test-slave-bqcl krb5kdc[10734](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786337010, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.22.167.84@KRBTEST.COM for HTTP/127.22.167.84@KRBTEST.COM
Aug 10 04:43:31 dist-test-slave-bqcl krb5kdc[10734](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786337010, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.22.167.84@KRBTEST.COM for HTTP/127.22.167.84@KRBTEST.COM
Aug 10 04:43:33 dist-test-slave-bqcl krb5kdc[10734](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786337013, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.22.167.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.167.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.167.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786336863388938-23197-0/krb5kdc/kudu_127.22.167.126.keytab.
Entry for principal kudu/127.22.167.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786336863388938-23197-0/krb5kdc/kudu_127.22.167.126.keytab.
I20260810 04:43:33.525826 23197 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskemv_AM/build/release/bin/kudu
/tmp/dist-test-taskemv_AM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786336863388938-23197-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786336863388938-23197-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786336863388938-23197-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786336863388938-23197-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.167.126:46725
--webserver_interface=127.22.167.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786336863388938-23197-0/krb5kdc/kudu_127.22.167.126.keytab
--principal=kudu/127.22.167.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.167.84:34261
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.167.126:46725
--ranger_config_path=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786336863388938-23197-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.22.167.84:39545/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786336863388938-23197-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786336863388938-23197-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786336863388938-23197-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 04:43:33.643945 11470 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 04:43:33.644238 11470 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 04:43:33.644272 11470 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 04:43:33.645973 11470 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 04:43:33.646035 11470 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 04:43:33.646056 11470 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 04:43:33.646073 11470 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 04:43:33.646097 11470 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 04:43:33.648445 11470 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.167.84:34261
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786336863388938-23197-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786336863388938-23197-0/minicluster-data/master-0/wal
--ranger_kms_url=127.22.167.84:39545/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.22.167.126:46725
--ranger_config_path=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786336863388938-23197-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786336863388938-23197-0/krb5kdc/kudu_127.22.167.126.keytab
--principal=kudu/127.22.167.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.167.126:46725
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786336863388938-23197-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.167.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11470
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786336863388938-23197-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 03:54:01 UTC on ccycloud.quasar-icijmt.root.comops.site
build id 4577009
I20260810 04:43:33.648710 11470 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 04:43:33.648977 11470 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 04:43:33.654249 11480 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 04:43:33.654233 11477 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 04:43:33.654273 11470 server_base.cc:1034] running on GCE node
W20260810 04:43:33.654220 11483 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 04:43:33.654672 11470 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 04:43:33.655009 11470 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 04:43:33.656198 11470 hybrid_clock.cc:648] HybridClock initialized: now 1786337013656177 us; error 34 us; skew 500 ppm
Aug 10 04:43:33 dist-test-slave-bqcl krb5kdc[10734](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786337013, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.167.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 04:43:33.659324 11470 init.cc:377] Logged in from keytab as kudu/127.22.167.126@KRBTEST.COM (short username kudu)
I20260810 04:43:33.661182 11470 webserver.cc:460] Webserver started at http://127.22.167.126:35963/ using document root <none> and password file <none>
I20260810 04:43:33.661405 11470 fs_manager.cc:359] Metadata directory not provided
I20260810 04:43:33.661443 11470 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 04:43:33.661566 11470 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 10 04:43:33 dist-test-slave-bqcl krb5kdc[10734](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786337013, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.167.126@KRBTEST.COM for HTTP/127.22.167.84@KRBTEST.COM
I20260810 04:43:33.715327 11470 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786336863388938-23197-0/minicluster-data/master-0/data/instance:
uuid: "4b4fa2bf44ae4e55a8947ac66f5e2e64"
format_stamp: "Formatted at 2026-08-10 04:43:33 on dist-test-slave-bqcl"
server_key: "134a1b3bb20dcc4e07b6960c6b9fe412"
server_key_iv: "5edc92e73b22df31e24700992247615e"
server_key_version: "kuduclusterkey@0"
I20260810 04:43:33.715847 11470 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786336863388938-23197-0/minicluster-data/master-0/wal/instance:
uuid: "4b4fa2bf44ae4e55a8947ac66f5e2e64"
format_stamp: "Formatted at 2026-08-10 04:43:33 on dist-test-slave-bqcl"
server_key: "134a1b3bb20dcc4e07b6960c6b9fe412"
server_key_iv: "5edc92e73b22df31e24700992247615e"
server_key_version: "kuduclusterkey@0"
I20260810 04:43:33.717617 11470 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.003s sys 0.000s
I20260810 04:43:33.728912 11495 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 04:43:33.729107 11470 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260810 04:43:33.729200 11470 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786336863388938-23197-0/minicluster-data/master-0/data,/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786336863388938-23197-0/minicluster-data/master-0/wal
uuid: "4b4fa2bf44ae4e55a8947ac66f5e2e64"
format_stamp: "Formatted at 2026-08-10 04:43:33 on dist-test-slave-bqcl"
server_key: "134a1b3bb20dcc4e07b6960c6b9fe412"
server_key_iv: "5edc92e73b22df31e24700992247615e"
server_key_version: "kuduclusterkey@0"
I20260810 04:43:33.729290 11470 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786336863388938-23197-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786336863388938-23197-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786336863388938-23197-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 04:43:33.743216 11470 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 04:43:33.745048 11470 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 04:43:33.745190 11470 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 04:43:33.749567 11470 rpc_server.cc:307] RPC server started. Bound to: 127.22.167.126:46725
I20260810 04:43:33.749645 11580 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.167.126:46725 every 8 connection(s)
I20260810 04:43:33.749971 11470 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786336863388938-23197-0/minicluster-data/master-0/data/info.pb
I20260810 04:43:33.750771 11584 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 04:43:33.752466 23197 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskemv_AM/build/release/bin/kudu as pid 11470
I20260810 04:43:33.752574 23197 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786336863388938-23197-0/minicluster-data/master-0/wal/instance
I20260810 04:43:33.752755 23197 external_mini_cluster.cc:1428] Setting key 396031119827e6642d9cbc2641b5ce38
I20260810 04:43:33.753858 11584 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4b4fa2bf44ae4e55a8947ac66f5e2e64: Bootstrap starting.
I20260810 04:43:33.754616 11584 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4b4fa2bf44ae4e55a8947ac66f5e2e64: Neither blocks nor log segments found. Creating new log.
I20260810 04:43:33.754992 11584 log.cc:824] T 00000000000000000000000000000000 P 4b4fa2bf44ae4e55a8947ac66f5e2e64: Log is configured to *not* fsync() on all Append() calls
I20260810 04:43:33.755787 11584 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4b4fa2bf44ae4e55a8947ac66f5e2e64: No bootstrap required, opened a new log
I20260810 04:43:33.758231 11584 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4b4fa2bf44ae4e55a8947ac66f5e2e64 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4b4fa2bf44ae4e55a8947ac66f5e2e64" member_type: VOTER last_known_addr { host: "127.22.167.126" port: 46725 } }
I20260810 04:43:33.758427 11584 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4b4fa2bf44ae4e55a8947ac66f5e2e64 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 04:43:33.758510 11584 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4b4fa2bf44ae4e55a8947ac66f5e2e64 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4b4fa2bf44ae4e55a8947ac66f5e2e64, State: Initialized, Role: FOLLOWER
I20260810 04:43:33.758651 11584 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4b4fa2bf44ae4e55a8947ac66f5e2e64 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4b4fa2bf44ae4e55a8947ac66f5e2e64" member_type: VOTER last_known_addr { host: "127.22.167.126" port: 46725 } }
I20260810 04:43:33.758730 11584 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4b4fa2bf44ae4e55a8947ac66f5e2e64 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
Aug 10 04:43:33 dist-test-slave-bqcl krb5kdc[10734](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786337013, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.167.126@KRBTEST.COM
I20260810 04:43:33.758790 11584 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4b4fa2bf44ae4e55a8947ac66f5e2e64 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 04:43:33.758836 11584 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4b4fa2bf44ae4e55a8947ac66f5e2e64 [term 0 FOLLOWER]: Advancing to term 1
I20260810 04:43:33.759473 11584 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4b4fa2bf44ae4e55a8947ac66f5e2e64 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4b4fa2bf44ae4e55a8947ac66f5e2e64" member_type: VOTER last_known_addr { host: "127.22.167.126" port: 46725 } }
I20260810 04:43:33.759584 11584 leader_election.cc:302] T 00000000000000000000000000000000 P 4b4fa2bf44ae4e55a8947ac66f5e2e64 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4b4fa2bf44ae4e55a8947ac66f5e2e64; no voters:
I20260810 04:43:33.759822 11584 leader_election.cc:288] T 00000000000000000000000000000000 P 4b4fa2bf44ae4e55a8947ac66f5e2e64 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 04:43:33.759877 11590 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4b4fa2bf44ae4e55a8947ac66f5e2e64 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 04:43:33.760140 11584 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4b4fa2bf44ae4e55a8947ac66f5e2e64 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 04:43:33.760314 11590 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4b4fa2bf44ae4e55a8947ac66f5e2e64 [term 1 LEADER]: Becoming Leader. State: Replica: 4b4fa2bf44ae4e55a8947ac66f5e2e64, State: Running, Role: LEADER
I20260810 04:43:33.760448 11590 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4b4fa2bf44ae4e55a8947ac66f5e2e64 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4b4fa2bf44ae4e55a8947ac66f5e2e64" member_type: VOTER last_known_addr { host: "127.22.167.126" port: 46725 } }
I20260810 04:43:33.760807 11590 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4b4fa2bf44ae4e55a8947ac66f5e2e64 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4b4fa2bf44ae4e55a8947ac66f5e2e64. Latest consensus state: current_term: 1 leader_uuid: "4b4fa2bf44ae4e55a8947ac66f5e2e64" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4b4fa2bf44ae4e55a8947ac66f5e2e64" member_type: VOTER last_known_addr { host: "127.22.167.126" port: 46725 } } }
I20260810 04:43:33.760946 11590 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4b4fa2bf44ae4e55a8947ac66f5e2e64 [sys.catalog]: This master's current role is: LEADER
I20260810 04:43:33.761605 11595 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4b4fa2bf44ae4e55a8947ac66f5e2e64 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4b4fa2bf44ae4e55a8947ac66f5e2e64" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4b4fa2bf44ae4e55a8947ac66f5e2e64" member_type: VOTER last_known_addr { host: "127.22.167.126" port: 46725 } } }
I20260810 04:43:33.761735 11595 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4b4fa2bf44ae4e55a8947ac66f5e2e64 [sys.catalog]: This master's current role is: LEADER
I20260810 04:43:33.761772 11584 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786336863388938-23197-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260810 04:43:33.762401 11627 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 04:43:33.765856 11627 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 04:43:33.766054 11588 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:43:33.753236 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50784 (local address 127.22.167.126:46725)
0810 04:43:33.753460 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0810 04:43:33.753468 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 04:43:33.753491 (+ 23us) server_negotiation.cc:407] Connection header received
0810 04:43:33.753873 (+ 382us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:43:33.753879 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:43:33.753928 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:43:33.754074 (+ 146us) server_negotiation.cc:226] Negotiated authn=SASL
0810 04:43:33.754692 (+ 618us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:43:33.755763 (+ 1071us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:43:33.756505 (+ 742us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:43:33.756906 (+ 401us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:43:33.757009 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:43:33.759258 (+ 2249us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 04:43:33.759262 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 04:43:33.759263 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 04:43:33.759274 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0810 04:43:33.761047 (+ 1773us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 04:43:33.761428 (+ 381us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 04:43:33.761431 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 04:43:33.761431 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 04:43:33.761534 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 04:43:33.761729 (+ 195us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 04:43:33.761730 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 04:43:33.761730 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 04:43:33.762069 (+ 339us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 04:43:33.762278 (+ 209us) server_negotiation.cc:1035] Waiting for connection context
0810 04:43:33.762785 (+ 507us) server_negotiation.cc:299] Negotiation successful
0810 04:43:33.765543 (+ 2758us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":182,"thread_start_us":73,"threads_started":1}
I20260810 04:43:33.768522 11627 catalog_manager.cc:1347] Generated new cluster ID: 846005b1b71747f4a630e9867e6d21db
I20260810 04:43:33.768576 11627 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 04:43:33.781692 11627 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 04:43:33.782536 11627 catalog_manager.cc:1504] Loading token signing keys...
I20260810 04:43:33.791920 11627 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4b4fa2bf44ae4e55a8947ac66f5e2e64: Generated new TSK 0
I20260810 04:43:33.792181 11627 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.22.167.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.167.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.167.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786336863388938-23197-0/krb5kdc/kudu_127.22.167.65.keytab.
Entry for principal kudu/127.22.167.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786336863388938-23197-0/krb5kdc/kudu_127.22.167.65.keytab.
I20260810 04:43:34.450093 23197 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskemv_AM/build/release/bin/kudu
/tmp/dist-test-taskemv_AM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786336863388938-23197-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786336863388938-23197-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786336863388938-23197-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786336863388938-23197-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.22.167.84:39545/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.167.65:0
--local_ip_for_outbound_sockets=127.22.167.65
--webserver_interface=127.22.167.65
--webserver_port=0
--tserver_master_addrs=127.22.167.126:46725
--keytab_file=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786336863388938-23197-0/krb5kdc/kudu_127.22.167.65.keytab
--principal=kudu/127.22.167.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.167.84:34261
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786336863388938-23197-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786336863388938-23197-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786336863388938-23197-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 04:43:34.563958 11672 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 04:43:34.564224 11672 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 04:43:34.564258 11672 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 04:43:34.565876 11672 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 04:43:34.565934 11672 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 04:43:34.565973 11672 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.167.65
I20260810 04:43:34.567451 11672 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.167.84:34261
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786336863388938-23197-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786336863388938-23197-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.22.167.84:39545/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786336863388938-23197-0/krb5kdc/kudu_127.22.167.65.keytab
--principal=kudu/127.22.167.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.167.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786336863388938-23197-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.22.167.65
--webserver_port=0
--tserver_master_addrs=127.22.167.126:46725
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11672
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.167.65
--log_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786336863388938-23197-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 03:54:01 UTC on ccycloud.quasar-icijmt.root.comops.site
build id 4577009
I20260810 04:43:34.567658 11672 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 04:43:34.567876 11672 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 04:43:34.572618 11678 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 04:43:34.572624 11684 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 04:43:34.572819 11681 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 04:43:34.573294 11672 server_base.cc:1034] running on GCE node
I20260810 04:43:34.573585 11672 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 04:43:34.573927 11672 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 04:43:34.575238 11672 hybrid_clock.cc:648] HybridClock initialized: now 1786337014575226 us; error 41 us; skew 500 ppm
Aug 10 04:43:34 dist-test-slave-bqcl krb5kdc[10734](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786337014, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.167.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 04:43:34.578297 11672 init.cc:377] Logged in from keytab as kudu/127.22.167.65@KRBTEST.COM (short username kudu)
I20260810 04:43:34.578845 11672 webserver.cc:460] Webserver started at http://127.22.167.65:34455/ using document root <none> and password file <none>
I20260810 04:43:34.579097 11672 fs_manager.cc:359] Metadata directory not provided
I20260810 04:43:34.579159 11672 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 04:43:34.579288 11672 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 10 04:43:34 dist-test-slave-bqcl krb5kdc[10734](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786337014, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.167.65@KRBTEST.COM for HTTP/127.22.167.84@KRBTEST.COM
I20260810 04:43:34.592212 11672 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786336863388938-23197-0/minicluster-data/ts-0/data/instance:
uuid: "3611eb1a11f9449db2274c36c1d30340"
format_stamp: "Formatted at 2026-08-10 04:43:34 on dist-test-slave-bqcl"
server_key: "929bf3687adebac484874fb914b31ccd"
server_key_iv: "0ba015526d8779b0a8a4626ac4795fd0"
server_key_version: "kuduclusterkey@0"
I20260810 04:43:34.592653 11672 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786336863388938-23197-0/minicluster-data/ts-0/wal/instance:
uuid: "3611eb1a11f9449db2274c36c1d30340"
format_stamp: "Formatted at 2026-08-10 04:43:34 on dist-test-slave-bqcl"
server_key: "929bf3687adebac484874fb914b31ccd"
server_key_iv: "0ba015526d8779b0a8a4626ac4795fd0"
server_key_version: "kuduclusterkey@0"
I20260810 04:43:34.595666 11672 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.001s sys 0.000s
I20260810 04:43:34.608683 11697 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 04:43:34.608886 11672 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 04:43:34.608991 11672 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786336863388938-23197-0/minicluster-data/ts-0/data,/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786336863388938-23197-0/minicluster-data/ts-0/wal
uuid: "3611eb1a11f9449db2274c36c1d30340"
format_stamp: "Formatted at 2026-08-10 04:43:34 on dist-test-slave-bqcl"
server_key: "929bf3687adebac484874fb914b31ccd"
server_key_iv: "0ba015526d8779b0a8a4626ac4795fd0"
server_key_version: "kuduclusterkey@0"
I20260810 04:43:34.609078 11672 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786336863388938-23197-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786336863388938-23197-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786336863388938-23197-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 04:43:34.622915 11672 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 04:43:34.624847 11672 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 04:43:34.625007 11672 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 04:43:34.625281 11672 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 04:43:34.625649 11672 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 04:43:34.625698 11672 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 04:43:34.625733 11672 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 04:43:34.625756 11672 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 04:43:34.632970 11672 rpc_server.cc:307] RPC server started. Bound to: 127.22.167.65:34665
I20260810 04:43:34.633040 11888 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.167.65:34665 every 8 connection(s)
I20260810 04:43:34.633388 11672 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786336863388938-23197-0/minicluster-data/ts-0/data/info.pb
I20260810 04:43:34.637173 23197 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskemv_AM/build/release/bin/kudu as pid 11672
I20260810 04:43:34.637296 23197 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786336863388938-23197-0/minicluster-data/ts-0/wal/instance
I20260810 04:43:34.637504 23197 external_mini_cluster.cc:1428] Setting key b8b1d94250f490eeaead65933e9936e7
Aug 10 04:43:34 dist-test-slave-bqcl krb5kdc[10734](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786337014, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.167.65@KRBTEST.COM for kudu/127.22.167.126@KRBTEST.COM
I20260810 04:43:34.644685 11896 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:43:34.634272 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.167.65:56645 (local address 127.22.167.126:46725)
0810 04:43:34.634407 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0810 04:43:34.634409 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 04:43:34.634650 (+ 241us) server_negotiation.cc:407] Connection header received
0810 04:43:34.635306 (+ 656us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:43:34.635308 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:43:34.635322 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:43:34.635410 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0810 04:43:34.635992 (+ 582us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:43:34.636798 (+ 806us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:43:34.638245 (+ 1447us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:43:34.638563 (+ 318us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:43:34.638610 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:43:34.641074 (+ 2464us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 04:43:34.641076 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 04:43:34.641076 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 04:43:34.641094 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0810 04:43:34.643021 (+ 1927us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 04:43:34.643529 (+ 508us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 04:43:34.643531 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 04:43:34.643531 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 04:43:34.643603 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 04:43:34.643909 (+ 306us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 04:43:34.643910 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 04:43:34.643911 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 04:43:34.644052 (+ 141us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 04:43:34.644096 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0810 04:43:34.644496 (+ 400us) server_negotiation.cc:299] Negotiation successful
0810 04:43:34.644585 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":111,"thread_start_us":80,"threads_started":1}
I20260810 04:43:34.645159 11895 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:43:34.634236 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.126:46725 (local address 127.22.167.65:56645)
0810 04:43:34.634497 (+ 261us) negotiation.cc:107] Waiting for socket to connect
0810 04:43:34.634550 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0810 04:43:34.635177 (+ 627us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 04:43:34.635498 (+ 321us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 04:43:34.635502 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 04:43:34.635713 (+ 211us) client_negotiation.cc:189] Negotiated authn=SASL
0810 04:43:34.635895 (+ 182us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:43:34.635901 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:43:34.636880 (+ 979us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:43:34.636881 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:43:34.638115 (+ 1234us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:43:34.638119 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:43:34.638720 (+ 601us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:43:34.638721 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:43:34.638858 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:43:34.639407 (+ 549us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 04:43:34.639417 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0810 04:43:34.640966 (+ 1549us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 04:43:34.643118 (+ 2152us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 04:43:34.643125 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 04:43:34.643127 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 04:43:34.643433 (+ 306us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 04:43:34.643679 (+ 246us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 04:43:34.643680 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 04:43:34.643680 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 04:43:34.643822 (+ 142us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 04:43:34.644147 (+ 325us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 04:43:34.644155 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 04:43:34.644330 (+ 175us) client_negotiation.cc:769] Sending connection context
0810 04:43:34.644779 (+ 449us) client_negotiation.cc:240] Negotiation successful
0810 04:43:34.644941 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":214,"thread_start_us":82,"threads_started":1}
I20260810 04:43:34.645641 11891 heartbeater.cc:347] Connected to a master server at 127.22.167.126:46725
I20260810 04:43:34.645725 11891 heartbeater.cc:464] Registering TS with master...
I20260810 04:43:34.645892 11891 heartbeater.cc:511] Master 127.22.167.126:46725 requested a full tablet report, sending...
I20260810 04:43:34.646652 11520 ts_manager.cc:194] Registered new tserver with Master: 3611eb1a11f9449db2274c36c1d30340 (127.22.167.65:34665)
I20260810 04:43:34.647570 11520 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.167.65@KRBTEST.COM'} at 127.22.167.65:56645
WARNING: no policy specified for kudu/127.22.167.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.167.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.167.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786336863388938-23197-0/krb5kdc/kudu_127.22.167.66.keytab.
Entry for principal kudu/127.22.167.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786336863388938-23197-0/krb5kdc/kudu_127.22.167.66.keytab.
I20260810 04:43:34.668491 23197 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskemv_AM/build/release/bin/kudu
/tmp/dist-test-taskemv_AM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786336863388938-23197-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786336863388938-23197-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786336863388938-23197-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786336863388938-23197-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.22.167.84:39545/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.167.66:0
--local_ip_for_outbound_sockets=127.22.167.66
--webserver_interface=127.22.167.66
--webserver_port=0
--tserver_master_addrs=127.22.167.126:46725
--keytab_file=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786336863388938-23197-0/krb5kdc/kudu_127.22.167.66.keytab
--principal=kudu/127.22.167.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.167.84:34261
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786336863388938-23197-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786336863388938-23197-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786336863388938-23197-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 04:43:34.787563 11901 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 04:43:34.787828 11901 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 04:43:34.787868 11901 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 04:43:34.789402 11901 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 04:43:34.789450 11901 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
Aug 10 04:43:34 dist-test-slave-bqcl krb5kdc[10734](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786337014, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.167.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260810 04:43:34.789487 11901 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.167.66
I20260810 04:43:34.791563 11901 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.167.84:34261
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786336863388938-23197-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786336863388938-23197-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.22.167.84:39545/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786336863388938-23197-0/krb5kdc/kudu_127.22.167.66.keytab
--principal=kudu/127.22.167.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.167.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786336863388938-23197-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.22.167.66
--webserver_port=0
--tserver_master_addrs=127.22.167.126:46725
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11901
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.167.66
--log_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786336863388938-23197-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 03:54:01 UTC on ccycloud.quasar-icijmt.root.comops.site
build id 4577009
I20260810 04:43:34.791812 11901 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 04:43:34.792136 11901 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 04:43:34.798409 11913 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 04:43:34.798488 11910 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 04:43:34.798625 11911 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 04:43:34.799006 11901 server_base.cc:1034] running on GCE node
I20260810 04:43:34.799206 11901 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 04:43:34.799440 11901 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 04:43:34.800575 11901 hybrid_clock.cc:648] HybridClock initialized: now 1786337014800560 us; error 31 us; skew 500 ppm
Aug 10 04:43:34 dist-test-slave-bqcl krb5kdc[10734](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786337014, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.167.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 04:43:34.805469 11901 init.cc:377] Logged in from keytab as kudu/127.22.167.66@KRBTEST.COM (short username kudu)
I20260810 04:43:34.805974 11901 webserver.cc:460] Webserver started at http://127.22.167.66:46499/ using document root <none> and password file <none>
I20260810 04:43:34.806284 11901 fs_manager.cc:359] Metadata directory not provided
I20260810 04:43:34.806350 11901 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 04:43:34.806476 11901 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 10 04:43:34 dist-test-slave-bqcl krb5kdc[10734](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786337014, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.167.66@KRBTEST.COM for HTTP/127.22.167.84@KRBTEST.COM
I20260810 04:43:34.824347 11901 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786336863388938-23197-0/minicluster-data/ts-1/data/instance:
uuid: "71640ad9a6c844c795f61bdb42f171b7"
format_stamp: "Formatted at 2026-08-10 04:43:34 on dist-test-slave-bqcl"
server_key: "1577e26ac69520715284a459210786d9"
server_key_iv: "5aede5d5520ea819d4b161f7ff5a098f"
server_key_version: "kuduclusterkey@0"
I20260810 04:43:34.824708 11901 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786336863388938-23197-0/minicluster-data/ts-1/wal/instance:
uuid: "71640ad9a6c844c795f61bdb42f171b7"
format_stamp: "Formatted at 2026-08-10 04:43:34 on dist-test-slave-bqcl"
server_key: "1577e26ac69520715284a459210786d9"
server_key_iv: "5aede5d5520ea819d4b161f7ff5a098f"
server_key_version: "kuduclusterkey@0"
I20260810 04:43:34.826437 11901 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.002s sys 0.000s
I20260810 04:43:34.842224 11927 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 04:43:34.842430 11901 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260810 04:43:34.842541 11901 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786336863388938-23197-0/minicluster-data/ts-1/data,/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786336863388938-23197-0/minicluster-data/ts-1/wal
uuid: "71640ad9a6c844c795f61bdb42f171b7"
format_stamp: "Formatted at 2026-08-10 04:43:34 on dist-test-slave-bqcl"
server_key: "1577e26ac69520715284a459210786d9"
server_key_iv: "5aede5d5520ea819d4b161f7ff5a098f"
server_key_version: "kuduclusterkey@0"
I20260810 04:43:34.842628 11901 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786336863388938-23197-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786336863388938-23197-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786336863388938-23197-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 04:43:34.865152 11901 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 04:43:34.867798 11901 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 04:43:34.868009 11901 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 04:43:34.868330 11901 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 04:43:34.868747 11901 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 04:43:34.868844 11901 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 04:43:34.868917 11901 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 04:43:34.868978 11901 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 04:43:34.875960 11901 rpc_server.cc:307] RPC server started. Bound to: 127.22.167.66:44815
I20260810 04:43:34.876470 11901 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786336863388938-23197-0/minicluster-data/ts-1/data/info.pb
I20260810 04:43:34.876700 23197 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskemv_AM/build/release/bin/kudu as pid 11901
I20260810 04:43:34.876796 23197 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786336863388938-23197-0/minicluster-data/ts-1/wal/instance
I20260810 04:43:34.876996 23197 external_mini_cluster.cc:1428] Setting key 3f5dc840ecbf0a5b78ae8e730b2dacf3
I20260810 04:43:34.882854 12119 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.167.66:44815 every 8 connection(s)
WARNING: no policy specified for kudu/127.22.167.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.167.67@KRBTEST.COM" created.
Aug 10 04:43:34 dist-test-slave-bqcl krb5kdc[10734](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786337014, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.167.66@KRBTEST.COM for kudu/127.22.167.126@KRBTEST.COM
I20260810 04:43:34.905947 11896 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:43:34.883045 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.167.66:42431 (local address 127.22.167.126:46725)
0810 04:43:34.883093 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0810 04:43:34.883096 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 04:43:34.894409 (+ 11313us) server_negotiation.cc:407] Connection header received
0810 04:43:34.895151 (+ 742us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:43:34.895152 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:43:34.895177 (+ 25us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:43:34.895229 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0810 04:43:34.896057 (+ 828us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:43:34.896838 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:43:34.898451 (+ 1613us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:43:34.898734 (+ 283us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:43:34.898783 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:43:34.902276 (+ 3493us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 04:43:34.902279 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 04:43:34.902279 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 04:43:34.902287 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0810 04:43:34.904052 (+ 1765us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 04:43:34.904632 (+ 580us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 04:43:34.904634 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 04:43:34.904634 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 04:43:34.904710 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 04:43:34.905081 (+ 371us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 04:43:34.905082 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 04:43:34.905083 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 04:43:34.905263 (+ 180us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 04:43:34.905327 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0810 04:43:34.905738 (+ 411us) server_negotiation.cc:299] Negotiation successful
0810 04:43:34.905840 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260810 04:43:34.906807 12127 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:43:34.882771 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.126:46725 (local address 127.22.167.66:42431)
0810 04:43:34.894236 (+ 11465us) negotiation.cc:107] Waiting for socket to connect
0810 04:43:34.894302 (+ 66us) client_negotiation.cc:174] Beginning negotiation
0810 04:43:34.895002 (+ 700us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 04:43:34.895372 (+ 370us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 04:43:34.895377 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 04:43:34.895619 (+ 242us) client_negotiation.cc:189] Negotiated authn=SASL
0810 04:43:34.895914 (+ 295us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:43:34.895921 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:43:34.896965 (+ 1044us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:43:34.896967 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:43:34.898343 (+ 1376us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:43:34.898347 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:43:34.898901 (+ 554us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:43:34.898902 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:43:34.899061 (+ 159us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:43:34.900399 (+ 1338us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 04:43:34.900410 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0810 04:43:34.902117 (+ 1707us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 04:43:34.904176 (+ 2059us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 04:43:34.904184 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 04:43:34.904187 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 04:43:34.904513 (+ 326us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 04:43:34.904800 (+ 287us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 04:43:34.904800 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 04:43:34.904801 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 04:43:34.904972 (+ 171us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 04:43:34.905365 (+ 393us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 04:43:34.905373 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 04:43:34.905569 (+ 196us) client_negotiation.cc:769] Sending connection context
0810 04:43:34.906351 (+ 782us) client_negotiation.cc:240] Negotiation successful
0810 04:43:34.906549 (+ 198us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11401,"thread_start_us":87,"threads_started":1}
I20260810 04:43:34.907693 12122 heartbeater.cc:347] Connected to a master server at 127.22.167.126:46725
I20260810 04:43:34.907805 12122 heartbeater.cc:464] Registering TS with master...
I20260810 04:43:34.908022 12122 heartbeater.cc:511] Master 127.22.167.126:46725 requested a full tablet report, sending...
I20260810 04:43:34.908485 11520 ts_manager.cc:194] Registered new tserver with Master: 71640ad9a6c844c795f61bdb42f171b7 (127.22.167.66:44815)
I20260810 04:43:34.909009 11520 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.167.66@KRBTEST.COM'} at 127.22.167.66:42431
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.167.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786336863388938-23197-0/krb5kdc/kudu_127.22.167.67.keytab.
Entry for principal kudu/127.22.167.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786336863388938-23197-0/krb5kdc/kudu_127.22.167.67.keytab.
I20260810 04:43:34.917107 23197 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskemv_AM/build/release/bin/kudu
/tmp/dist-test-taskemv_AM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786336863388938-23197-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786336863388938-23197-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786336863388938-23197-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786336863388938-23197-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.22.167.84:39545/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.167.67:0
--local_ip_for_outbound_sockets=127.22.167.67
--webserver_interface=127.22.167.67
--webserver_port=0
--tserver_master_addrs=127.22.167.126:46725
--keytab_file=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786336863388938-23197-0/krb5kdc/kudu_127.22.167.67.keytab
--principal=kudu/127.22.167.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.167.84:34261
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786336863388938-23197-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786336863388938-23197-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786336863388938-23197-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 04:43:35.042908 12130 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 04:43:35.043131 12130 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 04:43:35.043174 12130 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 04:43:35.044340 12130 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 04:43:35.044381 12130 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 04:43:35.044415 12130 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.167.67
I20260810 04:43:35.045958 12130 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.167.84:34261
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786336863388938-23197-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786336863388938-23197-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.22.167.84:39545/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786336863388938-23197-0/krb5kdc/kudu_127.22.167.67.keytab
--principal=kudu/127.22.167.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.167.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786336863388938-23197-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.22.167.67
--webserver_port=0
--tserver_master_addrs=127.22.167.126:46725
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12130
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.167.67
--log_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786336863388938-23197-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 03:54:01 UTC on ccycloud.quasar-icijmt.root.comops.site
build id 4577009
I20260810 04:43:35.046228 12130 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 04:43:35.046444 12130 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 04:43:35.051362 12138 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 04:43:35.051391 12143 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 04:43:35.051483 12139 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 04:43:35.052117 12130 server_base.cc:1034] running on GCE node
I20260810 04:43:35.052349 12130 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 04:43:35.052611 12130 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 04:43:35.053759 12130 hybrid_clock.cc:648] HybridClock initialized: now 1786337015053748 us; error 29 us; skew 500 ppm
Aug 10 04:43:35 dist-test-slave-bqcl krb5kdc[10734](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786337015, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.167.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 04:43:35.056358 12130 init.cc:377] Logged in from keytab as kudu/127.22.167.67@KRBTEST.COM (short username kudu)
I20260810 04:43:35.056903 12130 webserver.cc:460] Webserver started at http://127.22.167.67:44167/ using document root <none> and password file <none>
I20260810 04:43:35.057164 12130 fs_manager.cc:359] Metadata directory not provided
I20260810 04:43:35.057228 12130 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 04:43:35.057364 12130 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 10 04:43:35 dist-test-slave-bqcl krb5kdc[10734](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786337015, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.167.67@KRBTEST.COM for HTTP/127.22.167.84@KRBTEST.COM
I20260810 04:43:35.069779 12130 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786336863388938-23197-0/minicluster-data/ts-2/data/instance:
uuid: "806d9b6ce4884202822fe4e31271bae4"
format_stamp: "Formatted at 2026-08-10 04:43:35 on dist-test-slave-bqcl"
server_key: "1e94362acd1081ecf517736875d95776"
server_key_iv: "cccc87283916ffcdd67900c88384623a"
server_key_version: "kuduclusterkey@0"
I20260810 04:43:35.070343 12130 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786336863388938-23197-0/minicluster-data/ts-2/wal/instance:
uuid: "806d9b6ce4884202822fe4e31271bae4"
format_stamp: "Formatted at 2026-08-10 04:43:35 on dist-test-slave-bqcl"
server_key: "1e94362acd1081ecf517736875d95776"
server_key_iv: "cccc87283916ffcdd67900c88384623a"
server_key_version: "kuduclusterkey@0"
I20260810 04:43:35.073000 12130 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.000s sys 0.001s
I20260810 04:43:35.093971 12152 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 04:43:35.094178 12130 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.001s sys 0.000s
I20260810 04:43:35.094282 12130 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786336863388938-23197-0/minicluster-data/ts-2/data,/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786336863388938-23197-0/minicluster-data/ts-2/wal
uuid: "806d9b6ce4884202822fe4e31271bae4"
format_stamp: "Formatted at 2026-08-10 04:43:35 on dist-test-slave-bqcl"
server_key: "1e94362acd1081ecf517736875d95776"
server_key_iv: "cccc87283916ffcdd67900c88384623a"
server_key_version: "kuduclusterkey@0"
I20260810 04:43:35.094380 12130 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786336863388938-23197-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786336863388938-23197-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786336863388938-23197-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 04:43:35.105870 12130 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 04:43:35.109308 12130 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 04:43:35.109457 12130 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 04:43:35.109685 12130 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 04:43:35.110059 12130 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 04:43:35.110110 12130 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 04:43:35.110175 12130 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 04:43:35.110198 12130 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 04:43:35.117836 12130 rpc_server.cc:307] RPC server started. Bound to: 127.22.167.67:42283
I20260810 04:43:35.118307 12320 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.167.67:42283 every 8 connection(s)
I20260810 04:43:35.118415 12130 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786336863388938-23197-0/minicluster-data/ts-2/data/info.pb
I20260810 04:43:35.124363 23197 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskemv_AM/build/release/bin/kudu as pid 12130
I20260810 04:43:35.124464 23197 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786336863388938-23197-0/minicluster-data/ts-2/wal/instance
I20260810 04:43:35.124678 23197 external_mini_cluster.cc:1428] Setting key 34be1c00e73aabc6df3d59425ff37d5c
Aug 10 04:43:35 dist-test-slave-bqcl krb5kdc[10734](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786337015, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.167.67@KRBTEST.COM for kudu/127.22.167.126@KRBTEST.COM
I20260810 04:43:35.139598 11896 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:43:35.119926 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.167.67:43051 (local address 127.22.167.126:46725)
0810 04:43:35.119970 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0810 04:43:35.119972 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 04:43:35.120141 (+ 169us) server_negotiation.cc:407] Connection header received
0810 04:43:35.120813 (+ 672us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:43:35.120815 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:43:35.120827 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:43:35.120896 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0810 04:43:35.121852 (+ 956us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:43:35.122694 (+ 842us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:43:35.124410 (+ 1716us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:43:35.124675 (+ 265us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:43:35.124715 (+ 40us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:43:35.135883 (+ 11168us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 04:43:35.135885 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 04:43:35.135885 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 04:43:35.135893 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0810 04:43:35.137813 (+ 1920us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 04:43:35.138385 (+ 572us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 04:43:35.138386 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 04:43:35.138387 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 04:43:35.138450 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 04:43:35.138742 (+ 292us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 04:43:35.138743 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 04:43:35.138744 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 04:43:35.138911 (+ 167us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 04:43:35.138989 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0810 04:43:35.139395 (+ 406us) server_negotiation.cc:299] Negotiation successful
0810 04:43:35.139496 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260810 04:43:35.140012 12323 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:43:35.119484 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.126:46725 (local address 127.22.167.67:43051)
0810 04:43:35.119806 (+ 322us) negotiation.cc:107] Waiting for socket to connect
0810 04:43:35.119854 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0810 04:43:35.120693 (+ 839us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 04:43:35.120972 (+ 279us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 04:43:35.120977 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 04:43:35.121307 (+ 330us) client_negotiation.cc:189] Negotiated authn=SASL
0810 04:43:35.121756 (+ 449us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:43:35.121763 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:43:35.122790 (+ 1027us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:43:35.122791 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:43:35.124311 (+ 1520us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:43:35.124314 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:43:35.124803 (+ 489us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:43:35.124804 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:43:35.125067 (+ 263us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:43:35.125733 (+ 666us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 04:43:35.125743 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0810 04:43:35.135746 (+ 10003us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 04:43:35.137913 (+ 2167us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 04:43:35.137923 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 04:43:35.137926 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 04:43:35.138291 (+ 365us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 04:43:35.138520 (+ 229us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 04:43:35.138521 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 04:43:35.138521 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 04:43:35.138658 (+ 137us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 04:43:35.139020 (+ 362us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 04:43:35.139028 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 04:43:35.139221 (+ 193us) client_negotiation.cc:769] Sending connection context
0810 04:43:35.139353 (+ 132us) client_negotiation.cc:240] Negotiation successful
0810 04:43:35.139539 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":275,"thread_start_us":231,"threads_started":1}
I20260810 04:43:35.140722 12321 heartbeater.cc:347] Connected to a master server at 127.22.167.126:46725
I20260810 04:43:35.140839 12321 heartbeater.cc:464] Registering TS with master...
I20260810 04:43:35.141045 12321 heartbeater.cc:511] Master 127.22.167.126:46725 requested a full tablet report, sending...
I20260810 04:43:35.141492 11520 ts_manager.cc:194] Registered new tserver with Master: 806d9b6ce4884202822fe4e31271bae4 (127.22.167.67:42283)
I20260810 04:43:35.141986 11520 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.167.67@KRBTEST.COM'} at 127.22.167.67:43051
I20260810 04:43:35.149897 23197 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260810 04:43:35.150645 23197 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskemv_AM/build/release/bin/kudu with pid 11672
I20260810 04:43:35.212862 23197 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskemv_AM/build/release/bin/kudu with pid 11901
I20260810 04:43:35.235143 23197 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskemv_AM/build/release/bin/kudu with pid 12130
I20260810 04:43:35.247584 23197 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskemv_AM/build/release/bin/kudu with pid 11470
I20260810 04:43:35.263386 23197 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskemv_AM/build/release/bin/kudu
/tmp/dist-test-taskemv_AM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786336863388938-23197-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786336863388938-23197-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786336863388938-23197-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786336863388938-23197-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.167.126:46725
--webserver_interface=127.22.167.126
--webserver_port=35963
--keytab_file=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786336863388938-23197-0/krb5kdc/kudu_127.22.167.126.keytab
--principal=kudu/127.22.167.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.167.84:34261
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.167.126:46725
--ranger_config_path=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786336863388938-23197-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.22.167.84:39545/kms
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.22.167.84:39545/kms with env {KRB5CCNAME=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786336863388938-23197-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786336863388938-23197-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786336863388938-23197-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 04:43:35.348235 12325 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 04:43:35.348433 12325 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 04:43:35.348454 12325 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 04:43:35.349565 12325 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 04:43:35.349609 12325 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 04:43:35.349622 12325 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 04:43:35.349634 12325 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 04:43:35.349658 12325 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 04:43:35.351229 12325 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.167.84:34261
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786336863388938-23197-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786336863388938-23197-0/minicluster-data/master-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.22.167.84:39545/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.22.167.126:46725
--ranger_config_path=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786336863388938-23197-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786336863388938-23197-0/krb5kdc/kudu_127.22.167.126.keytab
--principal=kudu/127.22.167.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.167.126:46725
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786336863388938-23197-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.167.126
--webserver_port=35963
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12325
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786336863388938-23197-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 03:54:01 UTC on ccycloud.quasar-icijmt.root.comops.site
build id 4577009
I20260810 04:43:35.351447 12325 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 04:43:35.351651 12325 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 04:43:35.356143 12332 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 04:43:35.356192 12336 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 04:43:35.356187 12333 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 04:43:35.356232 12325 server_base.cc:1034] running on GCE node
I20260810 04:43:35.356648 12325 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 04:43:35.356901 12325 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 04:43:35.358081 12325 hybrid_clock.cc:648] HybridClock initialized: now 1786337015358026 us; error 28 us; skew 500 ppm
Aug 10 04:43:35 dist-test-slave-bqcl krb5kdc[10734](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786337015, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.167.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 04:43:35.360579 12325 init.cc:377] Logged in from keytab as kudu/127.22.167.126@KRBTEST.COM (short username kudu)
I20260810 04:43:35.361028 12325 webserver.cc:460] Webserver started at http://127.22.167.126:35963/ using document root <none> and password file <none>
I20260810 04:43:35.361217 12325 fs_manager.cc:359] Metadata directory not provided
I20260810 04:43:35.361276 12325 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 10 04:43:35 dist-test-slave-bqcl krb5kdc[10734](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786337015, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.167.126@KRBTEST.COM for HTTP/127.22.167.84@KRBTEST.COM
I20260810 04:43:35.398666 12325 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260810 04:43:35.399343 12349 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 04:43:35.399528 12325 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 04:43:35.399644 12325 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786336863388938-23197-0/minicluster-data/master-0/data,/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786336863388938-23197-0/minicluster-data/master-0/wal
uuid: "4b4fa2bf44ae4e55a8947ac66f5e2e64"
format_stamp: "Formatted at 2026-08-10 04:43:33 on dist-test-slave-bqcl"
server_key: "134a1b3bb20dcc4e07b6960c6b9fe412"
server_key_iv: "5edc92e73b22df31e24700992247615e"
server_key_version: "kuduclusterkey@0"
I20260810 04:43:35.399997 12325 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786336863388938-23197-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786336863388938-23197-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786336863388938-23197-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 04:43:35.413075 12325 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 04:43:35.414865 12325 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 04:43:35.414992 12325 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 04:43:35.419500 12325 rpc_server.cc:307] RPC server started. Bound to: 127.22.167.126:46725
I20260810 04:43:35.419569 12437 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.167.126:46725 every 8 connection(s)
I20260810 04:43:35.419898 12325 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786336863388938-23197-0/minicluster-data/master-0/data/info.pb
I20260810 04:43:35.421065 12438 sys_catalog.cc:263] Verifying existing consensus state
I20260810 04:43:35.421701 12438 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4b4fa2bf44ae4e55a8947ac66f5e2e64: Bootstrap starting.
I20260810 04:43:35.429590 12438 log.cc:824] T 00000000000000000000000000000000 P 4b4fa2bf44ae4e55a8947ac66f5e2e64: Log is configured to *not* fsync() on all Append() calls
I20260810 04:43:35.430047 23197 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskemv_AM/build/release/bin/kudu as pid 12325
I20260810 04:43:35.430367 23197 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskemv_AM/build/release/bin/kudu
/tmp/dist-test-taskemv_AM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786336863388938-23197-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786336863388938-23197-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786336863388938-23197-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786336863388938-23197-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.22.167.84:39545/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.167.65:34665
--local_ip_for_outbound_sockets=127.22.167.65
--tserver_master_addrs=127.22.167.126:46725
--webserver_port=34455
--webserver_interface=127.22.167.65
--keytab_file=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786336863388938-23197-0/krb5kdc/kudu_127.22.167.65.keytab
--principal=kudu/127.22.167.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.167.84:34261
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.22.167.84:39545/kms with env {KRB5CCNAME=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786336863388938-23197-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786336863388938-23197-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786336863388938-23197-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 04:43:35.431607 12438 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4b4fa2bf44ae4e55a8947ac66f5e2e64: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260810 04:43:35.431878 12438 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4b4fa2bf44ae4e55a8947ac66f5e2e64: Bootstrap complete.
I20260810 04:43:35.434190 12438 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4b4fa2bf44ae4e55a8947ac66f5e2e64 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4b4fa2bf44ae4e55a8947ac66f5e2e64" member_type: VOTER last_known_addr { host: "127.22.167.126" port: 46725 } }
I20260810 04:43:35.434459 12438 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4b4fa2bf44ae4e55a8947ac66f5e2e64 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4b4fa2bf44ae4e55a8947ac66f5e2e64, State: Initialized, Role: FOLLOWER
I20260810 04:43:35.434623 12438 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4b4fa2bf44ae4e55a8947ac66f5e2e64 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4b4fa2bf44ae4e55a8947ac66f5e2e64" member_type: VOTER last_known_addr { host: "127.22.167.126" port: 46725 } }
I20260810 04:43:35.434698 12438 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4b4fa2bf44ae4e55a8947ac66f5e2e64 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 04:43:35.434746 12438 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4b4fa2bf44ae4e55a8947ac66f5e2e64 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 04:43:35.434792 12438 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4b4fa2bf44ae4e55a8947ac66f5e2e64 [term 1 FOLLOWER]: Advancing to term 2
I20260810 04:43:35.435727 12438 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4b4fa2bf44ae4e55a8947ac66f5e2e64 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4b4fa2bf44ae4e55a8947ac66f5e2e64" member_type: VOTER last_known_addr { host: "127.22.167.126" port: 46725 } }
I20260810 04:43:35.435822 12438 leader_election.cc:302] T 00000000000000000000000000000000 P 4b4fa2bf44ae4e55a8947ac66f5e2e64 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4b4fa2bf44ae4e55a8947ac66f5e2e64; no voters:
I20260810 04:43:35.435976 12438 leader_election.cc:288] T 00000000000000000000000000000000 P 4b4fa2bf44ae4e55a8947ac66f5e2e64 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260810 04:43:35.436026 12443 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4b4fa2bf44ae4e55a8947ac66f5e2e64 [term 2 FOLLOWER]: Leader election won for term 2
I20260810 04:43:35.436139 12443 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4b4fa2bf44ae4e55a8947ac66f5e2e64 [term 2 LEADER]: Becoming Leader. State: Replica: 4b4fa2bf44ae4e55a8947ac66f5e2e64, State: Running, Role: LEADER
I20260810 04:43:35.436245 12438 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4b4fa2bf44ae4e55a8947ac66f5e2e64 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 04:43:35.436232 12443 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4b4fa2bf44ae4e55a8947ac66f5e2e64 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4b4fa2bf44ae4e55a8947ac66f5e2e64" member_type: VOTER last_known_addr { host: "127.22.167.126" port: 46725 } }
I20260810 04:43:35.436508 12444 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4b4fa2bf44ae4e55a8947ac66f5e2e64 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "4b4fa2bf44ae4e55a8947ac66f5e2e64" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4b4fa2bf44ae4e55a8947ac66f5e2e64" member_type: VOTER last_known_addr { host: "127.22.167.126" port: 46725 } } }
I20260810 04:43:35.436594 12444 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4b4fa2bf44ae4e55a8947ac66f5e2e64 [sys.catalog]: This master's current role is: LEADER
I20260810 04:43:35.436828 12447 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4b4fa2bf44ae4e55a8947ac66f5e2e64 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4b4fa2bf44ae4e55a8947ac66f5e2e64. Latest consensus state: current_term: 2 leader_uuid: "4b4fa2bf44ae4e55a8947ac66f5e2e64" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4b4fa2bf44ae4e55a8947ac66f5e2e64" member_type: VOTER last_known_addr { host: "127.22.167.126" port: 46725 } } }
I20260810 04:43:35.436939 12447 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4b4fa2bf44ae4e55a8947ac66f5e2e64 [sys.catalog]: This master's current role is: LEADER
I20260810 04:43:35.437065 12453 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 04:43:35.437556 12453 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 04:43:35.437690 12453 catalog_manager.cc:1259] Loaded cluster ID: 846005b1b71747f4a630e9867e6d21db
I20260810 04:43:35.437728 12453 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 04:43:35.437961 12438 ranger_client.cc:318] Using overwritten properties file: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786336863388938-23197-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260810 04:43:35.441444 12453 catalog_manager.cc:1504] Loading token signing keys...
I20260810 04:43:35.442008 12453 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 4b4fa2bf44ae4e55a8947ac66f5e2e64: Loaded TSK: 0
I20260810 04:43:35.442341 12453 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260810 04:43:35.517040 12441 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 04:43:35.517235 12441 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 04:43:35.517266 12441 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 04:43:35.518491 12441 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 04:43:35.518532 12441 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 04:43:35.518566 12441 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.167.65
I20260810 04:43:35.519973 12441 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.167.84:34261
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786336863388938-23197-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786336863388938-23197-0/minicluster-data/ts-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.22.167.84:39545/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786336863388938-23197-0/krb5kdc/kudu_127.22.167.65.keytab
--principal=kudu/127.22.167.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.167.65:34665
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786336863388938-23197-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.22.167.65
--webserver_port=34455
--tserver_master_addrs=127.22.167.126:46725
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12441
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.167.65
--log_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786336863388938-23197-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 03:54:01 UTC on ccycloud.quasar-icijmt.root.comops.site
build id 4577009
I20260810 04:43:35.520193 12441 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 04:43:35.520393 12441 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 04:43:35.526080 12499 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 04:43:35.526278 12502 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 04:43:35.526360 12498 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 04:43:35.526369 12441 server_base.cc:1034] running on GCE node
I20260810 04:43:35.526644 12441 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 04:43:35.526878 12441 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 04:43:35.528023 12441 hybrid_clock.cc:648] HybridClock initialized: now 1786337015528011 us; error 25 us; skew 500 ppm
Aug 10 04:43:35 dist-test-slave-bqcl krb5kdc[10734](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786337015, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.167.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 04:43:35.530844 12441 init.cc:377] Logged in from keytab as kudu/127.22.167.65@KRBTEST.COM (short username kudu)
I20260810 04:43:35.531260 12441 webserver.cc:460] Webserver started at http://127.22.167.65:34455/ using document root <none> and password file <none>
I20260810 04:43:35.531469 12441 fs_manager.cc:359] Metadata directory not provided
I20260810 04:43:35.531530 12441 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 10 04:43:35 dist-test-slave-bqcl krb5kdc[10734](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786337015, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.167.65@KRBTEST.COM for HTTP/127.22.167.84@KRBTEST.COM
I20260810 04:43:35.553673 12441 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260810 04:43:35.554811 12513 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 04:43:35.555033 12441 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260810 04:43:35.555128 12441 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786336863388938-23197-0/minicluster-data/ts-0/data,/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786336863388938-23197-0/minicluster-data/ts-0/wal
uuid: "3611eb1a11f9449db2274c36c1d30340"
format_stamp: "Formatted at 2026-08-10 04:43:34 on dist-test-slave-bqcl"
server_key: "929bf3687adebac484874fb914b31ccd"
server_key_iv: "0ba015526d8779b0a8a4626ac4795fd0"
server_key_version: "kuduclusterkey@0"
I20260810 04:43:35.555545 12441 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786336863388938-23197-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786336863388938-23197-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786336863388938-23197-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 04:43:35.573338 12441 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 04:43:35.575598 12441 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 04:43:35.575758 12441 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 04:43:35.575995 12441 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 04:43:35.576356 12441 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 04:43:35.576412 12441 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 04:43:35.576447 12441 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 04:43:35.576470 12441 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 04:43:35.586274 12441 rpc_server.cc:307] RPC server started. Bound to: 127.22.167.65:34665
I20260810 04:43:35.586673 12441 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786336863388938-23197-0/minicluster-data/ts-0/data/info.pb
I20260810 04:43:35.587311 12688 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.167.65:34665 every 8 connection(s)
I20260810 04:43:35.587554 23197 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskemv_AM/build/release/bin/kudu as pid 12441
I20260810 04:43:35.587826 23197 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskemv_AM/build/release/bin/kudu
/tmp/dist-test-taskemv_AM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786336863388938-23197-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786336863388938-23197-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786336863388938-23197-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786336863388938-23197-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.22.167.84:39545/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.167.66:44815
--local_ip_for_outbound_sockets=127.22.167.66
--tserver_master_addrs=127.22.167.126:46725
--webserver_port=46499
--webserver_interface=127.22.167.66
--keytab_file=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786336863388938-23197-0/krb5kdc/kudu_127.22.167.66.keytab
--principal=kudu/127.22.167.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.167.84:34261
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.22.167.84:39545/kms with env {KRB5CCNAME=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786336863388938-23197-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786336863388938-23197-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786336863388938-23197-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 10 04:43:35 dist-test-slave-bqcl krb5kdc[10734](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786337015, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.167.65@KRBTEST.COM for kudu/127.22.167.126@KRBTEST.COM
I20260810 04:43:35.602169 12693 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:43:35.590147 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.126:46725 (local address 127.22.167.65:60721)
0810 04:43:35.590710 (+ 563us) negotiation.cc:107] Waiting for socket to connect
0810 04:43:35.590755 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0810 04:43:35.591272 (+ 517us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 04:43:35.591800 (+ 528us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 04:43:35.591804 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 04:43:35.592007 (+ 203us) client_negotiation.cc:189] Negotiated authn=SASL
0810 04:43:35.592260 (+ 253us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:43:35.592266 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:43:35.593832 (+ 1566us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:43:35.593833 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:43:35.595106 (+ 1273us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:43:35.595109 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:43:35.596042 (+ 933us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:43:35.596043 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:43:35.596183 (+ 140us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:43:35.596687 (+ 504us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 04:43:35.596697 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0810 04:43:35.598060 (+ 1363us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 04:43:35.600250 (+ 2190us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 04:43:35.600257 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 04:43:35.600259 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 04:43:35.600544 (+ 285us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 04:43:35.600848 (+ 304us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 04:43:35.600849 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 04:43:35.600849 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 04:43:35.600984 (+ 135us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 04:43:35.601472 (+ 488us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 04:43:35.601479 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 04:43:35.601647 (+ 168us) client_negotiation.cc:769] Sending connection context
0810 04:43:35.601759 (+ 112us) client_negotiation.cc:240] Negotiation successful
0810 04:43:35.601913 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":516,"thread_start_us":74,"threads_started":1}
I20260810 04:43:35.602804 12695 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:43:35.590500 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.167.65:60721 (local address 127.22.167.126:46725)
0810 04:43:35.591435 (+ 935us) server_negotiation.cc:206] Beginning negotiation
0810 04:43:35.591443 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 04:43:35.591465 (+ 22us) server_negotiation.cc:407] Connection header received
0810 04:43:35.591588 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:43:35.591592 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:43:35.591631 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:43:35.591738 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0810 04:43:35.592388 (+ 650us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:43:35.593710 (+ 1322us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:43:35.595205 (+ 1495us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:43:35.595824 (+ 619us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:43:35.595928 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:43:35.598222 (+ 2294us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 04:43:35.598227 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 04:43:35.598227 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 04:43:35.598240 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0810 04:43:35.600157 (+ 1917us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 04:43:35.600659 (+ 502us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 04:43:35.600662 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 04:43:35.600663 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 04:43:35.600770 (+ 107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 04:43:35.601059 (+ 289us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 04:43:35.601060 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 04:43:35.601061 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 04:43:35.601359 (+ 298us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 04:43:35.601440 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0810 04:43:35.602453 (+ 1013us) server_negotiation.cc:299] Negotiation successful
0810 04:43:35.602575 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":885,"thread_start_us":85,"threads_started":1}
I20260810 04:43:35.603658 12689 heartbeater.cc:347] Connected to a master server at 127.22.167.126:46725
I20260810 04:43:35.603765 12689 heartbeater.cc:464] Registering TS with master...
I20260810 04:43:35.603948 12689 heartbeater.cc:511] Master 127.22.167.126:46725 requested a full tablet report, sending...
W20260810 04:43:35.604815 12689 heartbeater.cc:644] Failed to heartbeat to 127.22.167.126:46725 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260810 04:43:35.604879 12689 heartbeater.cc:464] Registering TS with master...
I20260810 04:43:35.604943 12689 heartbeater.cc:511] Master 127.22.167.126:46725 requested a full tablet report, sending...
I20260810 04:43:35.605273 12689 heartbeater.cc:464] Registering TS with master...
I20260810 04:43:35.605351 12689 heartbeater.cc:511] Master 127.22.167.126:46725 requested a full tablet report, sending...
W20260810 04:43:35.605652 12689 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260810 04:43:35.691844 12691 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 04:43:35.692090 12691 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 04:43:35.692124 12691 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 04:43:35.693737 12691 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 04:43:35.693782 12691 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 04:43:35.693817 12691 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.167.66
I20260810 04:43:35.695885 12691 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.167.84:34261
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786336863388938-23197-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786336863388938-23197-0/minicluster-data/ts-1/wal
--ranger_kms_url=invalid.host:1234/kms,127.22.167.84:39545/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786336863388938-23197-0/krb5kdc/kudu_127.22.167.66.keytab
--principal=kudu/127.22.167.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.167.66:44815
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786336863388938-23197-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.22.167.66
--webserver_port=46499
--tserver_master_addrs=127.22.167.126:46725
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12691
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.167.66
--log_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786336863388938-23197-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 03:54:01 UTC on ccycloud.quasar-icijmt.root.comops.site
build id 4577009
I20260810 04:43:35.696130 12691 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 04:43:35.696378 12691 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 04:43:35.701687 12704 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 04:43:35.701793 12703 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 04:43:35.701702 12708 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 04:43:35.702077 12691 server_base.cc:1034] running on GCE node
I20260810 04:43:35.702298 12691 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 04:43:35.702555 12691 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 04:43:35.703713 12691 hybrid_clock.cc:648] HybridClock initialized: now 1786337015703703 us; error 42 us; skew 500 ppm
Aug 10 04:43:35 dist-test-slave-bqcl krb5kdc[10734](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786337015, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.167.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 04:43:35.707172 12691 init.cc:377] Logged in from keytab as kudu/127.22.167.66@KRBTEST.COM (short username kudu)
I20260810 04:43:35.707672 12691 webserver.cc:460] Webserver started at http://127.22.167.66:46499/ using document root <none> and password file <none>
I20260810 04:43:35.707932 12691 fs_manager.cc:359] Metadata directory not provided
I20260810 04:43:35.708005 12691 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 10 04:43:35 dist-test-slave-bqcl krb5kdc[10734](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786337015, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.167.66@KRBTEST.COM for HTTP/127.22.167.84@KRBTEST.COM
I20260810 04:43:35.720371 12691 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260810 04:43:35.721318 12718 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 04:43:35.721474 12691 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 04:43:35.721570 12691 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786336863388938-23197-0/minicluster-data/ts-1/data,/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786336863388938-23197-0/minicluster-data/ts-1/wal
uuid: "71640ad9a6c844c795f61bdb42f171b7"
format_stamp: "Formatted at 2026-08-10 04:43:34 on dist-test-slave-bqcl"
server_key: "1577e26ac69520715284a459210786d9"
server_key_iv: "5aede5d5520ea819d4b161f7ff5a098f"
server_key_version: "kuduclusterkey@0"
I20260810 04:43:35.722016 12691 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786336863388938-23197-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786336863388938-23197-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786336863388938-23197-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 04:43:35.783598 12691 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 04:43:35.788353 12691 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 04:43:35.788532 12691 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 04:43:35.788763 12691 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 04:43:35.789172 12691 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 04:43:35.789227 12691 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 04:43:35.789269 12691 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 04:43:35.789286 12691 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 04:43:35.796693 12691 rpc_server.cc:307] RPC server started. Bound to: 127.22.167.66:44815
I20260810 04:43:35.796758 12901 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.167.66:44815 every 8 connection(s)
I20260810 04:43:35.797184 12691 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786336863388938-23197-0/minicluster-data/ts-1/data/info.pb
I20260810 04:43:35.799357 23197 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskemv_AM/build/release/bin/kudu as pid 12691
I20260810 04:43:35.799661 23197 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskemv_AM/build/release/bin/kudu
/tmp/dist-test-taskemv_AM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786336863388938-23197-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786336863388938-23197-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786336863388938-23197-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786336863388938-23197-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.22.167.84:39545/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.167.67:42283
--local_ip_for_outbound_sockets=127.22.167.67
--tserver_master_addrs=127.22.167.126:46725
--webserver_port=44167
--webserver_interface=127.22.167.67
--keytab_file=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786336863388938-23197-0/krb5kdc/kudu_127.22.167.67.keytab
--principal=kudu/127.22.167.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.167.84:34261
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.22.167.84:39545/kms with env {KRB5CCNAME=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786336863388938-23197-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786336863388938-23197-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786336863388938-23197-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 10 04:43:35 dist-test-slave-bqcl krb5kdc[10734](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786337015, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.167.66@KRBTEST.COM for kudu/127.22.167.126@KRBTEST.COM
I20260810 04:43:35.810806 12695 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:43:35.797979 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.167.66:33803 (local address 127.22.167.126:46725)
0810 04:43:35.798030 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0810 04:43:35.798032 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 04:43:35.798312 (+ 280us) server_negotiation.cc:407] Connection header received
0810 04:43:35.798967 (+ 655us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:43:35.798969 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:43:35.798982 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:43:35.799014 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0810 04:43:35.799807 (+ 793us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:43:35.800369 (+ 562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:43:35.801416 (+ 1047us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:43:35.801713 (+ 297us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:43:35.801752 (+ 39us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:43:35.804746 (+ 2994us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 04:43:35.804748 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 04:43:35.804748 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 04:43:35.804756 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0810 04:43:35.809111 (+ 4355us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 04:43:35.809624 (+ 513us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 04:43:35.809626 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 04:43:35.809626 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 04:43:35.809697 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 04:43:35.809996 (+ 299us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 04:43:35.809997 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 04:43:35.809997 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 04:43:35.810173 (+ 176us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 04:43:35.810221 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0810 04:43:35.810620 (+ 399us) server_negotiation.cc:299] Negotiation successful
0810 04:43:35.810710 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260810 04:43:35.811299 12904 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:43:35.797996 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.126:46725 (local address 127.22.167.66:33803)
0810 04:43:35.798189 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0810 04:43:35.798232 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0810 04:43:35.798835 (+ 603us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 04:43:35.799224 (+ 389us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 04:43:35.799228 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 04:43:35.799401 (+ 173us) client_negotiation.cc:189] Negotiated authn=SASL
0810 04:43:35.799686 (+ 285us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:43:35.799693 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:43:35.800462 (+ 769us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:43:35.800463 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:43:35.801309 (+ 846us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:43:35.801312 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:43:35.801857 (+ 545us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:43:35.801858 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:43:35.801990 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:43:35.802997 (+ 1007us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 04:43:35.803006 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0810 04:43:35.804638 (+ 1632us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 04:43:35.809204 (+ 4566us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 04:43:35.809211 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 04:43:35.809213 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 04:43:35.809514 (+ 301us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 04:43:35.809775 (+ 261us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 04:43:35.809775 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 04:43:35.809776 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 04:43:35.809911 (+ 135us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 04:43:35.810269 (+ 358us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 04:43:35.810275 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 04:43:35.810434 (+ 159us) client_negotiation.cc:769] Sending connection context
0810 04:43:35.810909 (+ 475us) client_negotiation.cc:240] Negotiation successful
0810 04:43:35.811075 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":149,"thread_start_us":92,"threads_started":1}
I20260810 04:43:35.811758 12902 heartbeater.cc:347] Connected to a master server at 127.22.167.126:46725
I20260810 04:43:35.811858 12902 heartbeater.cc:464] Registering TS with master...
I20260810 04:43:35.812054 12902 heartbeater.cc:511] Master 127.22.167.126:46725 requested a full tablet report, sending...
W20260810 04:43:35.812637 12902 heartbeater.cc:644] Failed to heartbeat to 127.22.167.126:46725 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260810 04:43:35.812711 12902 heartbeater.cc:464] Registering TS with master...
I20260810 04:43:35.812773 12902 heartbeater.cc:511] Master 127.22.167.126:46725 requested a full tablet report, sending...
I20260810 04:43:35.813100 12902 heartbeater.cc:464] Registering TS with master...
I20260810 04:43:35.813180 12902 heartbeater.cc:511] Master 127.22.167.126:46725 requested a full tablet report, sending...
W20260810 04:43:35.813477 12902 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260810 04:43:35.905591 12905 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 04:43:35.905841 12905 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 04:43:35.905874 12905 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 04:43:35.907526 12905 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 04:43:35.907585 12905 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 04:43:35.907624 12905 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.167.67
I20260810 04:43:35.909763 12905 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.167.84:34261
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786336863388938-23197-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786336863388938-23197-0/minicluster-data/ts-2/wal
--ranger_kms_url=invalid.host:1234/kms,127.22.167.84:39545/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786336863388938-23197-0/krb5kdc/kudu_127.22.167.67.keytab
--principal=kudu/127.22.167.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.167.67:42283
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786336863388938-23197-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.22.167.67
--webserver_port=44167
--tserver_master_addrs=127.22.167.126:46725
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12905
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.167.67
--log_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786336863388938-23197-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 03:54:01 UTC on ccycloud.quasar-icijmt.root.comops.site
build id 4577009
I20260810 04:43:35.909998 12905 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 04:43:35.910269 12905 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 04:43:35.916571 12916 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 04:43:35.916803 12915 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 04:43:35.917143 12905 server_base.cc:1034] running on GCE node
W20260810 04:43:35.918259 12918 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 04:43:35.918499 12905 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 04:43:35.918776 12905 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 04:43:35.919948 12905 hybrid_clock.cc:648] HybridClock initialized: now 1786337015919917 us; error 45 us; skew 500 ppm
Aug 10 04:43:35 dist-test-slave-bqcl krb5kdc[10734](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786337015, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.167.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 04:43:35.923277 12905 init.cc:377] Logged in from keytab as kudu/127.22.167.67@KRBTEST.COM (short username kudu)
I20260810 04:43:35.923833 12905 webserver.cc:460] Webserver started at http://127.22.167.67:44167/ using document root <none> and password file <none>
I20260810 04:43:35.924151 12905 fs_manager.cc:359] Metadata directory not provided
I20260810 04:43:35.924276 12905 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 10 04:43:35 dist-test-slave-bqcl krb5kdc[10734](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786337015, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.167.67@KRBTEST.COM for HTTP/127.22.167.84@KRBTEST.COM
I20260810 04:43:35.947014 12905 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.001s sys 0.000s
I20260810 04:43:35.947912 12936 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 04:43:35.948133 12905 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260810 04:43:35.948279 12905 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786336863388938-23197-0/minicluster-data/ts-2/data,/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786336863388938-23197-0/minicluster-data/ts-2/wal
uuid: "806d9b6ce4884202822fe4e31271bae4"
format_stamp: "Formatted at 2026-08-10 04:43:35 on dist-test-slave-bqcl"
server_key: "1e94362acd1081ecf517736875d95776"
server_key_iv: "cccc87283916ffcdd67900c88384623a"
server_key_version: "kuduclusterkey@0"
I20260810 04:43:35.948843 12905 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786336863388938-23197-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786336863388938-23197-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786336863388938-23197-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 04:43:35.983479 12905 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 04:43:35.985814 12905 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 04:43:35.985970 12905 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 04:43:35.986233 12905 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 04:43:35.986617 12905 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 04:43:35.986663 12905 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 04:43:35.986694 12905 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 04:43:35.986714 12905 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 04:43:35.993381 12905 rpc_server.cc:307] RPC server started. Bound to: 127.22.167.67:42283
I20260810 04:43:35.993868 12905 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786336863388938-23197-0/minicluster-data/ts-2/data/info.pb
I20260810 04:43:35.994351 13149 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.167.67:42283 every 8 connection(s)
I20260810 04:43:35.996776 23197 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskemv_AM/build/release/bin/kudu as pid 12905
Aug 10 04:43:35 dist-test-slave-bqcl krb5kdc[10734](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786337015, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.167.67@KRBTEST.COM for kudu/127.22.167.126@KRBTEST.COM
I20260810 04:43:36.005472 13158 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:43:35.997770 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58010 (local address 127.22.167.126:46725)
0810 04:43:35.997899 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0810 04:43:35.997901 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 04:43:35.997909 (+ 8us) server_negotiation.cc:407] Connection header received
0810 04:43:35.997928 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:43:35.997928 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:43:35.997938 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:43:35.998023 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0810 04:43:35.999204 (+ 1181us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:43:35.999798 (+ 594us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:43:36.000890 (+ 1092us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:43:36.001094 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:43:36.001140 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:43:36.002516 (+ 1376us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 04:43:36.002518 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 04:43:36.002518 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 04:43:36.002525 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 04:43:36.004250 (+ 1725us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 04:43:36.004687 (+ 437us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 04:43:36.004688 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 04:43:36.004688 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 04:43:36.004747 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 04:43:36.004938 (+ 191us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 04:43:36.004939 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 04:43:36.004939 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 04:43:36.005068 (+ 129us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 04:43:36.005230 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0810 04:43:36.005286 (+ 56us) server_negotiation.cc:299] Negotiation successful
0810 04:43:36.005373 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":111,"thread_start_us":79,"threads_started":1}
I20260810 04:43:36.008410 12695 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:43:35.995311 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.167.67:58877 (local address 127.22.167.126:46725)
0810 04:43:35.995357 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0810 04:43:35.995359 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 04:43:35.995371 (+ 12us) server_negotiation.cc:407] Connection header received
0810 04:43:35.996025 (+ 654us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:43:35.996027 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:43:35.996040 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:43:35.996134 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0810 04:43:35.996762 (+ 628us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:43:35.997582 (+ 820us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:43:35.999085 (+ 1503us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:43:35.999427 (+ 342us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:43:35.999503 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:43:36.001678 (+ 2175us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 04:43:36.001680 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 04:43:36.001680 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 04:43:36.001687 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 04:43:36.006853 (+ 5166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 04:43:36.007349 (+ 496us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 04:43:36.007350 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 04:43:36.007350 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 04:43:36.007414 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 04:43:36.007694 (+ 280us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 04:43:36.007695 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 04:43:36.007695 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 04:43:36.007824 (+ 129us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 04:43:36.007890 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0810 04:43:36.008239 (+ 349us) server_negotiation.cc:299] Negotiation successful
0810 04:43:36.008319 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":24,"server-negotiator.queue_time_us":22}
I20260810 04:43:36.008607 13154 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:43:35.995003 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.126:46725 (local address 127.22.167.67:58877)
0810 04:43:35.995207 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0810 04:43:35.995259 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0810 04:43:35.995893 (+ 634us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 04:43:35.996205 (+ 312us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 04:43:35.996210 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 04:43:35.996422 (+ 212us) client_negotiation.cc:189] Negotiated authn=SASL
0810 04:43:35.996658 (+ 236us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:43:35.996666 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:43:35.997698 (+ 1032us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:43:35.997699 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:43:35.998973 (+ 1274us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:43:35.998976 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:43:35.999541 (+ 565us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:43:35.999542 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:43:35.999714 (+ 172us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:43:36.000232 (+ 518us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 04:43:36.000240 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0810 04:43:36.001593 (+ 1353us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 04:43:36.006947 (+ 5354us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 04:43:36.006954 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 04:43:36.006956 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 04:43:36.007266 (+ 310us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 04:43:36.007485 (+ 219us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 04:43:36.007485 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 04:43:36.007485 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 04:43:36.007631 (+ 146us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 04:43:36.007893 (+ 262us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 04:43:36.007899 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 04:43:36.008067 (+ 168us) client_negotiation.cc:769] Sending connection context
0810 04:43:36.008202 (+ 135us) client_negotiation.cc:240] Negotiation successful
0810 04:43:36.008366 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":153,"thread_start_us":103,"threads_started":1}
I20260810 04:43:36.009184 13151 heartbeater.cc:347] Connected to a master server at 127.22.167.126:46725
I20260810 04:43:36.009290 13151 heartbeater.cc:464] Registering TS with master...
I20260810 04:43:36.009490 13151 heartbeater.cc:511] Master 127.22.167.126:46725 requested a full tablet report, sending...
W20260810 04:43:36.010277 13151 heartbeater.cc:644] Failed to heartbeat to 127.22.167.126:46725 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260810 04:43:36.010349 13151 heartbeater.cc:464] Registering TS with master...
I20260810 04:43:36.010409 13151 heartbeater.cc:511] Master 127.22.167.126:46725 requested a full tablet report, sending...
I20260810 04:43:36.010829 13151 heartbeater.cc:464] Registering TS with master...
I20260810 04:43:36.010915 13151 heartbeater.cc:511] Master 127.22.167.126:46725 requested a full tablet report, sending...
W20260810 04:43:36.011307 13151 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
Aug 10 04:43:36 dist-test-slave-bqcl krb5kdc[10734](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786337016, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.167.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 04:43:36.605819 12689 heartbeater.cc:464] Registering TS with master...
I20260810 04:43:36.605967 12689 heartbeater.cc:511] Master 127.22.167.126:46725 requested a full tablet report, sending...
I20260810 04:43:36.606372 12380 ts_manager.cc:194] Registered new tserver with Master: 3611eb1a11f9449db2274c36c1d30340 (127.22.167.65:34665)
I20260810 04:43:36.607363 12380 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.167.65@KRBTEST.COM'} at 127.22.167.65:60721
I20260810 04:43:36.813685 12902 heartbeater.cc:464] Registering TS with master...
I20260810 04:43:36.813855 12902 heartbeater.cc:511] Master 127.22.167.126:46725 requested a full tablet report, sending...
I20260810 04:43:36.814198 12380 ts_manager.cc:194] Registered new tserver with Master: 71640ad9a6c844c795f61bdb42f171b7 (127.22.167.66:44815)
I20260810 04:43:36.814760 12380 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.167.66@KRBTEST.COM'} at 127.22.167.66:33803
Aug 10 04:43:36 dist-test-slave-bqcl krb5kdc[10734](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786337016, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.167.126@KRBTEST.COM for HTTP/127.22.167.84@KRBTEST.COM
I20260810 04:43:37.011511 13151 heartbeater.cc:464] Registering TS with master...
I20260810 04:43:37.011693 13151 heartbeater.cc:511] Master 127.22.167.126:46725 requested a full tablet report, sending...
I20260810 04:43:37.011941 12380 ts_manager.cc:194] Registered new tserver with Master: 806d9b6ce4884202822fe4e31271bae4 (127.22.167.67:42283)
I20260810 04:43:37.012444 12380 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.167.67@KRBTEST.COM'} at 127.22.167.67:58877
I20260810 04:43:37.021431 23197 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260810 04:43:37.033241 13185 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:43:37.025051 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58022 (local address 127.22.167.126:46725)
0810 04:43:37.025195 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0810 04:43:37.025197 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 04:43:37.025407 (+ 210us) server_negotiation.cc:407] Connection header received
0810 04:43:37.025447 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:43:37.025448 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:43:37.025462 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:43:37.025546 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0810 04:43:37.026383 (+ 837us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:43:37.027220 (+ 837us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:43:37.028259 (+ 1039us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:43:37.028559 (+ 300us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:43:37.028649 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:43:37.030176 (+ 1527us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 04:43:37.030178 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 04:43:37.030178 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 04:43:37.030185 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 04:43:37.032013 (+ 1828us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 04:43:37.032441 (+ 428us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 04:43:37.032442 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 04:43:37.032442 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 04:43:37.032513 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 04:43:37.032703 (+ 190us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 04:43:37.032703 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 04:43:37.032704 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 04:43:37.032846 (+ 142us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 04:43:37.032950 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0810 04:43:37.033048 (+ 98us) server_negotiation.cc:299] Negotiation successful
0810 04:43:37.033136 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":123,"thread_start_us":81,"threads_started":1}
I20260810 04:43:37.035576 12380 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58022:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 04:43:37.037968 12380 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 04:43:37.047624 13206 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:43:37.041533 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.65:34665 (local address 127.0.0.1:33212)
0810 04:43:37.042404 (+ 871us) negotiation.cc:107] Waiting for socket to connect
0810 04:43:37.042413 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0810 04:43:37.042457 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 04:43:37.042860 (+ 403us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 04:43:37.042861 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 04:43:37.042863 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 04:43:37.042914 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:43:37.042916 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:43:37.044492 (+ 1576us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:43:37.044493 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:43:37.045585 (+ 1092us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:43:37.045588 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:43:37.047316 (+ 1728us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:43:37.047317 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:43:37.047400 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:43:37.047414 (+ 14us) client_negotiation.cc:769] Sending connection context
0810 04:43:37.047485 (+ 71us) client_negotiation.cc:240] Negotiation successful
0810 04:43:37.047537 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":830,"mutex_wait_us":1,"thread_start_us":414,"threads_started":1}
I20260810 04:43:37.047851 13200 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:43:37.041677 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33212 (local address 127.22.167.65:34665)
0810 04:43:37.042450 (+ 773us) server_negotiation.cc:206] Beginning negotiation
0810 04:43:37.042455 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 04:43:37.042467 (+ 12us) server_negotiation.cc:407] Connection header received
0810 04:43:37.042528 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:43:37.042530 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:43:37.042547 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:43:37.042642 (+ 95us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 04:43:37.043010 (+ 368us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:43:37.044395 (+ 1385us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:43:37.045698 (+ 1303us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:43:37.047070 (+ 1372us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:43:37.047208 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:43:37.047277 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0810 04:43:37.047732 (+ 455us) server_negotiation.cc:299] Negotiation successful
0810 04:43:37.047776 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":735,"thread_start_us":59,"threads_started":1}
I20260810 04:43:37.048547 12563 tablet_service.cc:1467] Processing CreateTablet for tablet 3c24322fbdf04d2183cb515d6c0eb034 (DEFAULT_TABLE table=test-table [id=56dafcd2660c47219c72058a74563fd2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 04:43:37.048908 12563 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3c24322fbdf04d2183cb515d6c0eb034. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 04:43:37.049028 13202 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:43:37.041180 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.67:42283 (local address 127.0.0.1:44756)
0810 04:43:37.042084 (+ 904us) negotiation.cc:107] Waiting for socket to connect
0810 04:43:37.042152 (+ 68us) client_negotiation.cc:174] Beginning negotiation
0810 04:43:37.042222 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 04:43:37.042826 (+ 604us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 04:43:37.042830 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 04:43:37.042832 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 04:43:37.042899 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:43:37.042901 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:43:37.044105 (+ 1204us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:43:37.044106 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:43:37.045329 (+ 1223us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:43:37.045332 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:43:37.048791 (+ 3459us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:43:37.048792 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:43:37.048856 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:43:37.048860 (+ 4us) client_negotiation.cc:769] Sending connection context
0810 04:43:37.048917 (+ 57us) client_negotiation.cc:240] Negotiation successful
0810 04:43:37.048958 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":853,"thread_start_us":647,"threads_started":1}
I20260810 04:43:37.050422 13195 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:43:37.041264 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44756 (local address 127.22.167.67:42283)
0810 04:43:37.041371 (+ 107us) server_negotiation.cc:206] Beginning negotiation
0810 04:43:37.041375 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 04:43:37.042537 (+ 1162us) server_negotiation.cc:407] Connection header received
0810 04:43:37.042591 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:43:37.042594 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:43:37.042610 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:43:37.042680 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 04:43:37.042992 (+ 312us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:43:37.044017 (+ 1025us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:43:37.047161 (+ 3144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:43:37.048581 (+ 1420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:43:37.048651 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:43:37.048711 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0810 04:43:37.050260 (+ 1549us) server_negotiation.cc:299] Negotiation successful
0810 04:43:37.050304 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80,"thread_start_us":57,"threads_started":1}
I20260810 04:43:37.051833 13196 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:43:37.041187 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.66:44815 (local address 127.0.0.1:46788)
0810 04:43:37.042267 (+ 1080us) negotiation.cc:107] Waiting for socket to connect
0810 04:43:37.042279 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0810 04:43:37.042334 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 04:43:37.045618 (+ 3284us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 04:43:37.045619 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 04:43:37.045621 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 04:43:37.045748 (+ 127us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:43:37.045750 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:43:37.049199 (+ 3449us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:43:37.049200 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:43:37.050321 (+ 1121us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:43:37.050324 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:43:37.051542 (+ 1218us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:43:37.051543 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:43:37.051681 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:43:37.051685 (+ 4us) client_negotiation.cc:769] Sending connection context
0810 04:43:37.051719 (+ 34us) client_negotiation.cc:240] Negotiation successful
0810 04:43:37.051759 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1033,"thread_start_us":187,"threads_started":1}
I20260810 04:43:37.052268 13198 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:43:37.041619 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46788 (local address 127.22.167.66:44815)
0810 04:43:37.045459 (+ 3840us) server_negotiation.cc:206] Beginning negotiation
0810 04:43:37.045464 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 04:43:37.045472 (+ 8us) server_negotiation.cc:407] Connection header received
0810 04:43:37.045503 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:43:37.045505 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:43:37.045519 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:43:37.045832 (+ 313us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 04:43:37.046057 (+ 225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:43:37.047067 (+ 1010us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:43:37.050422 (+ 3355us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:43:37.051388 (+ 966us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:43:37.052010 (+ 622us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:43:37.052085 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0810 04:43:37.052113 (+ 28us) server_negotiation.cc:299] Negotiation successful
0810 04:43:37.052156 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3808,"thread_start_us":65,"threads_started":1}
I20260810 04:43:37.053005 12790 tablet_service.cc:1467] Processing CreateTablet for tablet 3c24322fbdf04d2183cb515d6c0eb034 (DEFAULT_TABLE table=test-table [id=56dafcd2660c47219c72058a74563fd2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 04:43:37.053375 12790 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3c24322fbdf04d2183cb515d6c0eb034. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 04:43:37.054920 13012 tablet_service.cc:1467] Processing CreateTablet for tablet 3c24322fbdf04d2183cb515d6c0eb034 (DEFAULT_TABLE table=test-table [id=56dafcd2660c47219c72058a74563fd2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 04:43:37.055302 13012 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3c24322fbdf04d2183cb515d6c0eb034. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 04:43:37.058765 13207 tablet_bootstrap.cc:492] T 3c24322fbdf04d2183cb515d6c0eb034 P 3611eb1a11f9449db2274c36c1d30340: Bootstrap starting.
I20260810 04:43:37.059640 13208 tablet_bootstrap.cc:492] T 3c24322fbdf04d2183cb515d6c0eb034 P 71640ad9a6c844c795f61bdb42f171b7: Bootstrap starting.
I20260810 04:43:37.061089 13208 tablet_bootstrap.cc:654] T 3c24322fbdf04d2183cb515d6c0eb034 P 71640ad9a6c844c795f61bdb42f171b7: Neither blocks nor log segments found. Creating new log.
I20260810 04:43:37.061980 13208 log.cc:824] T 3c24322fbdf04d2183cb515d6c0eb034 P 71640ad9a6c844c795f61bdb42f171b7: Log is configured to *not* fsync() on all Append() calls
I20260810 04:43:37.062801 13207 tablet_bootstrap.cc:654] T 3c24322fbdf04d2183cb515d6c0eb034 P 3611eb1a11f9449db2274c36c1d30340: Neither blocks nor log segments found. Creating new log.
I20260810 04:43:37.062920 13208 tablet_bootstrap.cc:492] T 3c24322fbdf04d2183cb515d6c0eb034 P 71640ad9a6c844c795f61bdb42f171b7: No bootstrap required, opened a new log
I20260810 04:43:37.062990 13208 ts_tablet_manager.cc:1397] T 3c24322fbdf04d2183cb515d6c0eb034 P 71640ad9a6c844c795f61bdb42f171b7: Time spent bootstrapping tablet: real 0.003s user 0.003s sys 0.000s
I20260810 04:43:37.064307 13207 log.cc:824] T 3c24322fbdf04d2183cb515d6c0eb034 P 3611eb1a11f9449db2274c36c1d30340: Log is configured to *not* fsync() on all Append() calls
I20260810 04:43:37.064836 13209 tablet_bootstrap.cc:492] T 3c24322fbdf04d2183cb515d6c0eb034 P 806d9b6ce4884202822fe4e31271bae4: Bootstrap starting.
I20260810 04:43:37.066331 13209 tablet_bootstrap.cc:654] T 3c24322fbdf04d2183cb515d6c0eb034 P 806d9b6ce4884202822fe4e31271bae4: Neither blocks nor log segments found. Creating new log.
I20260810 04:43:37.067358 13209 log.cc:824] T 3c24322fbdf04d2183cb515d6c0eb034 P 806d9b6ce4884202822fe4e31271bae4: Log is configured to *not* fsync() on all Append() calls
I20260810 04:43:37.067996 13208 raft_consensus.cc:348] T 3c24322fbdf04d2183cb515d6c0eb034 P 71640ad9a6c844c795f61bdb42f171b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71640ad9a6c844c795f61bdb42f171b7" member_type: VOTER last_known_addr { host: "127.22.167.66" port: 44815 } } peers { permanent_uuid: "3611eb1a11f9449db2274c36c1d30340" member_type: VOTER last_known_addr { host: "127.22.167.65" port: 34665 } } peers { permanent_uuid: "806d9b6ce4884202822fe4e31271bae4" member_type: VOTER last_known_addr { host: "127.22.167.67" port: 42283 } }
I20260810 04:43:37.068135 13208 raft_consensus.cc:374] T 3c24322fbdf04d2183cb515d6c0eb034 P 71640ad9a6c844c795f61bdb42f171b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 04:43:37.068168 13208 raft_consensus.cc:729] T 3c24322fbdf04d2183cb515d6c0eb034 P 71640ad9a6c844c795f61bdb42f171b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 71640ad9a6c844c795f61bdb42f171b7, State: Initialized, Role: FOLLOWER
I20260810 04:43:37.068300 13208 consensus_queue.cc:260] T 3c24322fbdf04d2183cb515d6c0eb034 P 71640ad9a6c844c795f61bdb42f171b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71640ad9a6c844c795f61bdb42f171b7" member_type: VOTER last_known_addr { host: "127.22.167.66" port: 44815 } } peers { permanent_uuid: "3611eb1a11f9449db2274c36c1d30340" member_type: VOTER last_known_addr { host: "127.22.167.65" port: 34665 } } peers { permanent_uuid: "806d9b6ce4884202822fe4e31271bae4" member_type: VOTER last_known_addr { host: "127.22.167.67" port: 42283 } }
I20260810 04:43:37.068619 13208 ts_tablet_manager.cc:1428] T 3c24322fbdf04d2183cb515d6c0eb034 P 71640ad9a6c844c795f61bdb42f171b7: Time spent starting tablet: real 0.006s user 0.002s sys 0.004s
I20260810 04:43:37.068750 12902 heartbeater.cc:503] Master 127.22.167.126:46725 was elected leader, sending a full tablet report...
I20260810 04:43:37.069612 13207 tablet_bootstrap.cc:492] T 3c24322fbdf04d2183cb515d6c0eb034 P 3611eb1a11f9449db2274c36c1d30340: No bootstrap required, opened a new log
I20260810 04:43:37.069680 13207 ts_tablet_manager.cc:1397] T 3c24322fbdf04d2183cb515d6c0eb034 P 3611eb1a11f9449db2274c36c1d30340: Time spent bootstrapping tablet: real 0.011s user 0.003s sys 0.000s
I20260810 04:43:37.073637 13207 raft_consensus.cc:348] T 3c24322fbdf04d2183cb515d6c0eb034 P 3611eb1a11f9449db2274c36c1d30340 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71640ad9a6c844c795f61bdb42f171b7" member_type: VOTER last_known_addr { host: "127.22.167.66" port: 44815 } } peers { permanent_uuid: "3611eb1a11f9449db2274c36c1d30340" member_type: VOTER last_known_addr { host: "127.22.167.65" port: 34665 } } peers { permanent_uuid: "806d9b6ce4884202822fe4e31271bae4" member_type: VOTER last_known_addr { host: "127.22.167.67" port: 42283 } }
I20260810 04:43:37.073766 13207 raft_consensus.cc:374] T 3c24322fbdf04d2183cb515d6c0eb034 P 3611eb1a11f9449db2274c36c1d30340 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 04:43:37.073797 13207 raft_consensus.cc:729] T 3c24322fbdf04d2183cb515d6c0eb034 P 3611eb1a11f9449db2274c36c1d30340 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3611eb1a11f9449db2274c36c1d30340, State: Initialized, Role: FOLLOWER
I20260810 04:43:37.073906 13207 consensus_queue.cc:260] T 3c24322fbdf04d2183cb515d6c0eb034 P 3611eb1a11f9449db2274c36c1d30340 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71640ad9a6c844c795f61bdb42f171b7" member_type: VOTER last_known_addr { host: "127.22.167.66" port: 44815 } } peers { permanent_uuid: "3611eb1a11f9449db2274c36c1d30340" member_type: VOTER last_known_addr { host: "127.22.167.65" port: 34665 } } peers { permanent_uuid: "806d9b6ce4884202822fe4e31271bae4" member_type: VOTER last_known_addr { host: "127.22.167.67" port: 42283 } }
I20260810 04:43:37.074249 13207 ts_tablet_manager.cc:1428] T 3c24322fbdf04d2183cb515d6c0eb034 P 3611eb1a11f9449db2274c36c1d30340: Time spent starting tablet: real 0.005s user 0.004s sys 0.000s
I20260810 04:43:37.074385 12689 heartbeater.cc:503] Master 127.22.167.126:46725 was elected leader, sending a full tablet report...
I20260810 04:43:37.075510 13209 tablet_bootstrap.cc:492] T 3c24322fbdf04d2183cb515d6c0eb034 P 806d9b6ce4884202822fe4e31271bae4: No bootstrap required, opened a new log
I20260810 04:43:37.075578 13209 ts_tablet_manager.cc:1397] T 3c24322fbdf04d2183cb515d6c0eb034 P 806d9b6ce4884202822fe4e31271bae4: Time spent bootstrapping tablet: real 0.011s user 0.003s sys 0.000s
I20260810 04:43:37.079479 13209 raft_consensus.cc:348] T 3c24322fbdf04d2183cb515d6c0eb034 P 806d9b6ce4884202822fe4e31271bae4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71640ad9a6c844c795f61bdb42f171b7" member_type: VOTER last_known_addr { host: "127.22.167.66" port: 44815 } } peers { permanent_uuid: "3611eb1a11f9449db2274c36c1d30340" member_type: VOTER last_known_addr { host: "127.22.167.65" port: 34665 } } peers { permanent_uuid: "806d9b6ce4884202822fe4e31271bae4" member_type: VOTER last_known_addr { host: "127.22.167.67" port: 42283 } }
I20260810 04:43:37.079605 13209 raft_consensus.cc:374] T 3c24322fbdf04d2183cb515d6c0eb034 P 806d9b6ce4884202822fe4e31271bae4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 04:43:37.079638 13209 raft_consensus.cc:729] T 3c24322fbdf04d2183cb515d6c0eb034 P 806d9b6ce4884202822fe4e31271bae4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 806d9b6ce4884202822fe4e31271bae4, State: Initialized, Role: FOLLOWER
I20260810 04:43:37.079761 13209 consensus_queue.cc:260] T 3c24322fbdf04d2183cb515d6c0eb034 P 806d9b6ce4884202822fe4e31271bae4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71640ad9a6c844c795f61bdb42f171b7" member_type: VOTER last_known_addr { host: "127.22.167.66" port: 44815 } } peers { permanent_uuid: "3611eb1a11f9449db2274c36c1d30340" member_type: VOTER last_known_addr { host: "127.22.167.65" port: 34665 } } peers { permanent_uuid: "806d9b6ce4884202822fe4e31271bae4" member_type: VOTER last_known_addr { host: "127.22.167.67" port: 42283 } }
I20260810 04:43:37.080087 13209 ts_tablet_manager.cc:1428] T 3c24322fbdf04d2183cb515d6c0eb034 P 806d9b6ce4884202822fe4e31271bae4: Time spent starting tablet: real 0.004s user 0.001s sys 0.004s
I20260810 04:43:37.080237 13151 heartbeater.cc:503] Master 127.22.167.126:46725 was elected leader, sending a full tablet report...
W20260810 04:43:37.088114 12690 tablet.cc:2369] T 3c24322fbdf04d2183cb515d6c0eb034 P 3611eb1a11f9449db2274c36c1d30340: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 04:43:37.176385 13218 raft_consensus.cc:482] T 3c24322fbdf04d2183cb515d6c0eb034 P 3611eb1a11f9449db2274c36c1d30340 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 04:43:37.176548 13218 raft_consensus.cc:504] T 3c24322fbdf04d2183cb515d6c0eb034 P 3611eb1a11f9449db2274c36c1d30340 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71640ad9a6c844c795f61bdb42f171b7" member_type: VOTER last_known_addr { host: "127.22.167.66" port: 44815 } } peers { permanent_uuid: "3611eb1a11f9449db2274c36c1d30340" member_type: VOTER last_known_addr { host: "127.22.167.65" port: 34665 } } peers { permanent_uuid: "806d9b6ce4884202822fe4e31271bae4" member_type: VOTER last_known_addr { host: "127.22.167.67" port: 42283 } }
I20260810 04:43:37.177023 13218 leader_election.cc:288] T 3c24322fbdf04d2183cb515d6c0eb034 P 3611eb1a11f9449db2274c36c1d30340 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 71640ad9a6c844c795f61bdb42f171b7 (127.22.167.66:44815), 806d9b6ce4884202822fe4e31271bae4 (127.22.167.67:42283)
Aug 10 04:43:37 dist-test-slave-bqcl krb5kdc[10734](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786337016, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.167.126@KRBTEST.COM for HTTP/127.22.167.84@KRBTEST.COM
I20260810 04:43:37.182331 13195 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:43:37.177501 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.167.65:44937 (local address 127.22.167.67:42283)
0810 04:43:37.177542 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0810 04:43:37.177544 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 04:43:37.178058 (+ 514us) server_negotiation.cc:407] Connection header received
0810 04:43:37.178191 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:43:37.178192 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:43:37.178204 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:43:37.178234 (+ 30us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 04:43:37.178445 (+ 211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:43:37.179152 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:43:37.181334 (+ 2182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:43:37.181913 (+ 579us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:43:37.181961 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:43:37.181969 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0810 04:43:37.182215 (+ 246us) server_negotiation.cc:299] Negotiation successful
0810 04:43:37.182259 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260810 04:43:37.182761 13224 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:43:37.177231 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.67:42283 (local address 127.22.167.65:44937)
0810 04:43:37.178000 (+ 769us) negotiation.cc:107] Waiting for socket to connect
0810 04:43:37.178010 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0810 04:43:37.178084 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 04:43:37.178301 (+ 217us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 04:43:37.178301 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 04:43:37.178303 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 04:43:37.178368 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:43:37.178369 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:43:37.180089 (+ 1720us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:43:37.180090 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:43:37.181228 (+ 1138us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:43:37.181231 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:43:37.182045 (+ 814us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:43:37.182046 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:43:37.182116 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:43:37.182138 (+ 22us) client_negotiation.cc:769] Sending connection context
0810 04:43:37.182622 (+ 484us) client_negotiation.cc:240] Negotiation successful
0810 04:43:37.182670 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":754,"thread_start_us":110,"threads_started":1}
I20260810 04:43:37.183223 13063 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3c24322fbdf04d2183cb515d6c0eb034" candidate_uuid: "3611eb1a11f9449db2274c36c1d30340" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "806d9b6ce4884202822fe4e31271bae4" is_pre_election: true
I20260810 04:43:37.183384 13063 raft_consensus.cc:2454] T 3c24322fbdf04d2183cb515d6c0eb034 P 806d9b6ce4884202822fe4e31271bae4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3611eb1a11f9449db2274c36c1d30340 in term 0.
I20260810 04:43:37.184234 13198 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:43:37.177386 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.167.65:42131 (local address 127.22.167.66:44815)
0810 04:43:37.177430 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0810 04:43:37.177432 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 04:43:37.177554 (+ 122us) server_negotiation.cc:407] Connection header received
0810 04:43:37.177688 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:43:37.177690 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:43:37.177702 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:43:37.177735 (+ 33us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 04:43:37.177981 (+ 246us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:43:37.178744 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:43:37.179738 (+ 994us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:43:37.180353 (+ 615us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:43:37.180427 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:43:37.180437 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0810 04:43:37.184115 (+ 3678us) server_negotiation.cc:299] Negotiation successful
0810 04:43:37.184165 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260810 04:43:37.184509 13223 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:43:37.177175 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.66:44815 (local address 127.22.167.65:42131)
0810 04:43:37.177479 (+ 304us) negotiation.cc:107] Waiting for socket to connect
0810 04:43:37.177491 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0810 04:43:37.177589 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 04:43:37.177814 (+ 225us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 04:43:37.177815 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 04:43:37.177817 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 04:43:37.177896 (+ 79us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:43:37.177898 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:43:37.178848 (+ 950us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:43:37.178849 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:43:37.179653 (+ 804us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:43:37.179656 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:43:37.183913 (+ 4257us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:43:37.183915 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:43:37.184014 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:43:37.184019 (+ 5us) client_negotiation.cc:769] Sending connection context
0810 04:43:37.184324 (+ 305us) client_negotiation.cc:240] Negotiation successful
0810 04:43:37.184368 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":282,"thread_start_us":99,"threads_started":1}
I20260810 04:43:37.185134 12829 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3c24322fbdf04d2183cb515d6c0eb034" candidate_uuid: "3611eb1a11f9449db2274c36c1d30340" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "71640ad9a6c844c795f61bdb42f171b7" is_pre_election: true
I20260810 04:43:37.185369 12829 raft_consensus.cc:2454] T 3c24322fbdf04d2183cb515d6c0eb034 P 71640ad9a6c844c795f61bdb42f171b7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3611eb1a11f9449db2274c36c1d30340 in term 0.
I20260810 04:43:37.185676 12515 leader_election.cc:302] T 3c24322fbdf04d2183cb515d6c0eb034 P 3611eb1a11f9449db2274c36c1d30340 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3611eb1a11f9449db2274c36c1d30340, 71640ad9a6c844c795f61bdb42f171b7; no voters:
I20260810 04:43:37.187675 13218 raft_consensus.cc:2781] T 3c24322fbdf04d2183cb515d6c0eb034 P 3611eb1a11f9449db2274c36c1d30340 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 04:43:37.187744 13218 raft_consensus.cc:482] T 3c24322fbdf04d2183cb515d6c0eb034 P 3611eb1a11f9449db2274c36c1d30340 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 04:43:37.187764 13218 raft_consensus.cc:3037] T 3c24322fbdf04d2183cb515d6c0eb034 P 3611eb1a11f9449db2274c36c1d30340 [term 0 FOLLOWER]: Advancing to term 1
I20260810 04:43:37.188613 13218 raft_consensus.cc:504] T 3c24322fbdf04d2183cb515d6c0eb034 P 3611eb1a11f9449db2274c36c1d30340 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71640ad9a6c844c795f61bdb42f171b7" member_type: VOTER last_known_addr { host: "127.22.167.66" port: 44815 } } peers { permanent_uuid: "3611eb1a11f9449db2274c36c1d30340" member_type: VOTER last_known_addr { host: "127.22.167.65" port: 34665 } } peers { permanent_uuid: "806d9b6ce4884202822fe4e31271bae4" member_type: VOTER last_known_addr { host: "127.22.167.67" port: 42283 } }
I20260810 04:43:37.188735 13218 leader_election.cc:288] T 3c24322fbdf04d2183cb515d6c0eb034 P 3611eb1a11f9449db2274c36c1d30340 [CANDIDATE]: Term 1 election: Requested vote from peers 71640ad9a6c844c795f61bdb42f171b7 (127.22.167.66:44815), 806d9b6ce4884202822fe4e31271bae4 (127.22.167.67:42283)
I20260810 04:43:37.189623 13063 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3c24322fbdf04d2183cb515d6c0eb034" candidate_uuid: "3611eb1a11f9449db2274c36c1d30340" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "806d9b6ce4884202822fe4e31271bae4"
I20260810 04:43:37.189714 13063 raft_consensus.cc:3037] T 3c24322fbdf04d2183cb515d6c0eb034 P 806d9b6ce4884202822fe4e31271bae4 [term 0 FOLLOWER]: Advancing to term 1
I20260810 04:43:37.190557 13063 raft_consensus.cc:2454] T 3c24322fbdf04d2183cb515d6c0eb034 P 806d9b6ce4884202822fe4e31271bae4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3611eb1a11f9449db2274c36c1d30340 in term 1.
I20260810 04:43:37.190691 12829 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3c24322fbdf04d2183cb515d6c0eb034" candidate_uuid: "3611eb1a11f9449db2274c36c1d30340" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "71640ad9a6c844c795f61bdb42f171b7"
I20260810 04:43:37.190773 12829 raft_consensus.cc:3037] T 3c24322fbdf04d2183cb515d6c0eb034 P 71640ad9a6c844c795f61bdb42f171b7 [term 0 FOLLOWER]: Advancing to term 1
I20260810 04:43:37.190800 12517 leader_election.cc:302] T 3c24322fbdf04d2183cb515d6c0eb034 P 3611eb1a11f9449db2274c36c1d30340 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3611eb1a11f9449db2274c36c1d30340, 806d9b6ce4884202822fe4e31271bae4; no voters:
I20260810 04:43:37.191562 12829 raft_consensus.cc:2454] T 3c24322fbdf04d2183cb515d6c0eb034 P 71640ad9a6c844c795f61bdb42f171b7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3611eb1a11f9449db2274c36c1d30340 in term 1.
I20260810 04:43:37.191668 13218 raft_consensus.cc:2781] T 3c24322fbdf04d2183cb515d6c0eb034 P 3611eb1a11f9449db2274c36c1d30340 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 04:43:37.191830 13218 raft_consensus.cc:686] T 3c24322fbdf04d2183cb515d6c0eb034 P 3611eb1a11f9449db2274c36c1d30340 [term 1 LEADER]: Becoming Leader. State: Replica: 3611eb1a11f9449db2274c36c1d30340, State: Running, Role: LEADER
I20260810 04:43:37.191922 13218 consensus_queue.cc:237] T 3c24322fbdf04d2183cb515d6c0eb034 P 3611eb1a11f9449db2274c36c1d30340 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71640ad9a6c844c795f61bdb42f171b7" member_type: VOTER last_known_addr { host: "127.22.167.66" port: 44815 } } peers { permanent_uuid: "3611eb1a11f9449db2274c36c1d30340" member_type: VOTER last_known_addr { host: "127.22.167.65" port: 34665 } } peers { permanent_uuid: "806d9b6ce4884202822fe4e31271bae4" member_type: VOTER last_known_addr { host: "127.22.167.67" port: 42283 } }
I20260810 04:43:37.192780 12379 catalog_manager.cc:5495] T 3c24322fbdf04d2183cb515d6c0eb034 P 3611eb1a11f9449db2274c36c1d30340 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3611eb1a11f9449db2274c36c1d30340 (127.22.167.65). New cstate: current_term: 1 leader_uuid: "3611eb1a11f9449db2274c36c1d30340" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71640ad9a6c844c795f61bdb42f171b7" member_type: VOTER last_known_addr { host: "127.22.167.66" port: 44815 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3611eb1a11f9449db2274c36c1d30340" member_type: VOTER last_known_addr { host: "127.22.167.65" port: 34665 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "806d9b6ce4884202822fe4e31271bae4" member_type: VOTER last_known_addr { host: "127.22.167.67" port: 42283 } health_report { overall_health: UNKNOWN } } }
I20260810 04:43:37.228680 13200 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:43:37.224936 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33216 (local address 127.22.167.65:34665)
0810 04:43:37.224978 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0810 04:43:37.224980 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 04:43:37.224989 (+ 9us) server_negotiation.cc:407] Connection header received
0810 04:43:37.225035 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:43:37.225036 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:43:37.225059 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:43:37.225133 (+ 74us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 04:43:37.225348 (+ 215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:43:37.226103 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:43:37.227406 (+ 1303us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:43:37.227824 (+ 418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:43:37.227895 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:43:37.228111 (+ 216us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 04:43:37.228248 (+ 137us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 04:43:37.228322 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0810 04:43:37.228576 (+ 254us) server_negotiation.cc:299] Negotiation successful
0810 04:43:37.228597 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260810 04:43:37.231350 13063 raft_consensus.cc:1261] T 3c24322fbdf04d2183cb515d6c0eb034 P 806d9b6ce4884202822fe4e31271bae4 [term 1 FOLLOWER]: Refusing update from remote peer 3611eb1a11f9449db2274c36c1d30340: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 04:43:37.231460 12829 raft_consensus.cc:1261] T 3c24322fbdf04d2183cb515d6c0eb034 P 71640ad9a6c844c795f61bdb42f171b7 [term 1 FOLLOWER]: Refusing update from remote peer 3611eb1a11f9449db2274c36c1d30340: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 04:43:37.231745 13218 consensus_queue.cc:1035] T 3c24322fbdf04d2183cb515d6c0eb034 P 3611eb1a11f9449db2274c36c1d30340 [LEADER]: Connected to new peer: Peer: permanent_uuid: "806d9b6ce4884202822fe4e31271bae4" member_type: VOTER last_known_addr { host: "127.22.167.67" port: 42283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 04:43:37.232088 13218 consensus_queue.cc:1035] T 3c24322fbdf04d2183cb515d6c0eb034 P 3611eb1a11f9449db2274c36c1d30340 [LEADER]: Connected to new peer: Peer: permanent_uuid: "71640ad9a6c844c795f61bdb42f171b7" member_type: VOTER last_known_addr { host: "127.22.167.66" port: 44815 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 04:43:37.235713 13233 mvcc.cc:205] Tried to move back new op lower bound from 7316836422576779264 to 7316836422418714624. Current Snapshot: MvccSnapshot[applied={T|T < 7316836422576779264}]
I20260810 04:43:37.239480 12379 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58022:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 04:43:37.239584 12379 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58022:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 04:43:37.240567 13235 mvcc.cc:205] Tried to move back new op lower bound from 7316836422576779264 to 7316836422418714624. Current Snapshot: MvccSnapshot[applied={T|T < 7316836422576779264}]
I20260810 04:43:37.242555 13234 mvcc.cc:205] Tried to move back new op lower bound from 7316836422576779264 to 7316836422418714624. Current Snapshot: MvccSnapshot[applied={T|T < 7316836422576779264}]
I20260810 04:43:37.242758 12379 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 4b4fa2bf44ae4e55a8947ac66f5e2e64: Sending DeleteTablet for 3 replicas of tablet 3c24322fbdf04d2183cb515d6c0eb034
I20260810 04:43:37.243538 12563 tablet_service.cc:1514] Processing DeleteTablet for tablet 3c24322fbdf04d2183cb515d6c0eb034 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 04:43:37 UTC) from {username='kudu', principal='kudu/127.22.167.126@KRBTEST.COM'} at 127.0.0.1:33212
I20260810 04:43:37.244410 13012 tablet_service.cc:1514] Processing DeleteTablet for tablet 3c24322fbdf04d2183cb515d6c0eb034 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 04:43:37 UTC) from {username='kudu', principal='kudu/127.22.167.126@KRBTEST.COM'} at 127.0.0.1:44756
I20260810 04:43:37.244895 23197 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskemv_AM/build/release/bin/kudu with pid 12441
I20260810 04:43:37.255335 12790 tablet_service.cc:1514] Processing DeleteTablet for tablet 3c24322fbdf04d2183cb515d6c0eb034 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 04:43:37 UTC) from {username='kudu', principal='kudu/127.22.167.126@KRBTEST.COM'} at 127.0.0.1:46788
I20260810 04:43:37.255613 13247 tablet_replica.cc:330] T 3c24322fbdf04d2183cb515d6c0eb034 P 806d9b6ce4884202822fe4e31271bae4: stopping tablet replica
I20260810 04:43:37.255788 13247 raft_consensus.cc:2229] T 3c24322fbdf04d2183cb515d6c0eb034 P 806d9b6ce4884202822fe4e31271bae4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 04:43:37.256062 13247 raft_consensus.cc:2258] T 3c24322fbdf04d2183cb515d6c0eb034 P 806d9b6ce4884202822fe4e31271bae4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 04:43:37.256742 13247 ts_tablet_manager.cc:1905] T 3c24322fbdf04d2183cb515d6c0eb034 P 806d9b6ce4884202822fe4e31271bae4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 04:43:37.257789 13248 tablet_replica.cc:330] T 3c24322fbdf04d2183cb515d6c0eb034 P 71640ad9a6c844c795f61bdb42f171b7: stopping tablet replica
I20260810 04:43:37.257925 13248 raft_consensus.cc:2229] T 3c24322fbdf04d2183cb515d6c0eb034 P 71640ad9a6c844c795f61bdb42f171b7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 04:43:37.257998 13248 raft_consensus.cc:2258] T 3c24322fbdf04d2183cb515d6c0eb034 P 71640ad9a6c844c795f61bdb42f171b7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 04:43:37.259302 13247 ts_tablet_manager.cc:1918] T 3c24322fbdf04d2183cb515d6c0eb034 P 806d9b6ce4884202822fe4e31271bae4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 04:43:37.259446 13247 log.cc:1196] T 3c24322fbdf04d2183cb515d6c0eb034 P 806d9b6ce4884202822fe4e31271bae4: Deleting WAL directory at /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786336863388938-23197-0/minicluster-data/ts-2/wal/wals/3c24322fbdf04d2183cb515d6c0eb034
I20260810 04:43:37.259632 13248 ts_tablet_manager.cc:1905] T 3c24322fbdf04d2183cb515d6c0eb034 P 71640ad9a6c844c795f61bdb42f171b7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 04:43:37.259951 13247 ts_tablet_manager.cc:1939] T 3c24322fbdf04d2183cb515d6c0eb034 P 806d9b6ce4884202822fe4e31271bae4: Deleting consensus metadata
I20260810 04:43:37.260475 12354 catalog_manager.cc:4841] TS 806d9b6ce4884202822fe4e31271bae4 (127.22.167.67:42283): tablet 3c24322fbdf04d2183cb515d6c0eb034 (table test-table [id=56dafcd2660c47219c72058a74563fd2]) successfully deleted
I20260810 04:43:37.261474 13248 ts_tablet_manager.cc:1918] T 3c24322fbdf04d2183cb515d6c0eb034 P 71640ad9a6c844c795f61bdb42f171b7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 04:43:37.261543 13248 log.cc:1196] T 3c24322fbdf04d2183cb515d6c0eb034 P 71640ad9a6c844c795f61bdb42f171b7: Deleting WAL directory at /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786336863388938-23197-0/minicluster-data/ts-1/wal/wals/3c24322fbdf04d2183cb515d6c0eb034
I20260810 04:43:37.262785 13248 ts_tablet_manager.cc:1939] T 3c24322fbdf04d2183cb515d6c0eb034 P 71640ad9a6c844c795f61bdb42f171b7: Deleting consensus metadata
W20260810 04:43:37.262821 12352 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.22.167.65:34665 (error 108)
I20260810 04:43:37.263307 12352 catalog_manager.cc:4841] TS 71640ad9a6c844c795f61bdb42f171b7 (127.22.167.66:44815): tablet 3c24322fbdf04d2183cb515d6c0eb034 (table test-table [id=56dafcd2660c47219c72058a74563fd2]) successfully deleted
I20260810 04:43:37.263600 13196 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:43:37.263468 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.65:34665 (local address 127.0.0.1:33224)
0810 04:43:37.263526 (+ 58us) negotiation.cc:107] Waiting for socket to connect
0810 04:43:37.263568 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.167.65:34665: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
W20260810 04:43:37.263715 12352 catalog_manager.cc:4584] TS 3611eb1a11f9449db2274c36c1d30340 (127.22.167.65:34665): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 3c24322fbdf04d2183cb515d6c0eb034: Network error: Client connection negotiation failed: client connection to 127.22.167.65:34665: connect: Connection refused (error 111)
I20260810 04:43:37.263911 23197 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskemv_AM/build/release/bin/kudu with pid 12691
I20260810 04:43:37.276887 23197 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskemv_AM/build/release/bin/kudu with pid 12905
I20260810 04:43:37.315032 23197 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskemv_AM/build/release/bin/kudu with pid 12325
I20260810 04:43:37.327586 23197 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260810 04:43:37.672134 23197 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260810 04:43:37.672236 23197 mini_ranger.cc:66] Stopping Ranger...
I20260810 04:43:38.015648 23197 mini_ranger.cc:68] Stopped Ranger
2026-08-10 04:43:38.015 UTC [10783] LOG: received smart shutdown request
2026-08-10 04:43:38.019 UTC [10783] LOG: background worker "logical replication launcher" (PID 10805) exited with exit code 1
2026-08-10 04:43:38.020 UTC [10791] LOG: shutting down
2026-08-10 04:43:38.149 UTC [10783] LOG: database system is shut down
2026-08-10T04:43:38Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers (46708 ms)
[ RUN ] SecurityITest.IPKICACert
Loading random data
Initializing database '/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786336863388938-23197-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 04:43:38 dist-test-slave-bqcl krb5kdc[13266](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 04:43:38 dist-test-slave-bqcl krb5kdc[13266](info): set up 2 sockets
Aug 10 04:43:38 dist-test-slave-bqcl krb5kdc[13266](info): commencing operation
krb5kdc: starting...
W20260810 04:43:39.407449 23197 mini_kdc.cc:121] Time spent starting KDC: real 1.147s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786336863388938-23197-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786336863388938-23197-0/krb5kdc/test-user.keytab.
Aug 10 04:43:39 dist-test-slave-bqcl krb5kdc[13266](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786337019, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T04:43:39Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T04:43:39Z Disabled control of system clock
WARNING: no policy specified for kudu/127.22.167.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.167.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.167.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786336863388938-23197-0/krb5kdc/kudu_127.22.167.126.keytab.
Entry for principal kudu/127.22.167.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786336863388938-23197-0/krb5kdc/kudu_127.22.167.126.keytab.
I20260810 04:43:39.521632 23197 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskemv_AM/build/release/bin/kudu
/tmp/dist-test-taskemv_AM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786336863388938-23197-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786336863388938-23197-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786336863388938-23197-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786336863388938-23197-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.167.126:41211
--webserver_interface=127.22.167.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786336863388938-23197-0/krb5kdc/kudu_127.22.167.126.keytab
--principal=kudu/127.22.167.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.167.84:33893
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.167.126:41211,127.22.167.125:39737,127.22.167.124:34159
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786336863388938-23197-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786336863388938-23197-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786336863388938-23197-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 04:43:39.603118 13290 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 04:43:39.603302 13290 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 04:43:39.603322 13290 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 04:43:39.604364 13290 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 04:43:39.604398 13290 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 04:43:39.604410 13290 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 04:43:39.604421 13290 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 04:43:39.604434 13290 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 04:43:39.605871 13290 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.167.84:33893
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786336863388938-23197-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786336863388938-23197-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.22.167.126:41211,127.22.167.125:39737,127.22.167.124:34159
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786336863388938-23197-0/krb5kdc/kudu_127.22.167.126.keytab
--principal=kudu/127.22.167.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.167.126:41211
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786336863388938-23197-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.167.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13290
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786336863388938-23197-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 03:54:01 UTC on ccycloud.quasar-icijmt.root.comops.site
build id 4577009
I20260810 04:43:39.606056 13290 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 04:43:39.606276 13290 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 04:43:39.610651 13298 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 04:43:39.610706 13301 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 04:43:39.610713 13299 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 04:43:39.610893 13290 server_base.cc:1034] running on GCE node
I20260810 04:43:39.611166 13290 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 04:43:39.611420 13290 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 04:43:39.612555 13290 hybrid_clock.cc:648] HybridClock initialized: now 1786337019612536 us; error 35 us; skew 500 ppm
Aug 10 04:43:39 dist-test-slave-bqcl krb5kdc[13266](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786337019, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.167.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 04:43:39.614897 13290 init.cc:377] Logged in from keytab as kudu/127.22.167.126@KRBTEST.COM (short username kudu)
I20260810 04:43:39.615288 13290 webserver.cc:460] Webserver started at http://127.22.167.126:44087/ using document root <none> and password file <none>
I20260810 04:43:39.615490 13290 fs_manager.cc:359] Metadata directory not provided
I20260810 04:43:39.615533 13290 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 04:43:39.615651 13290 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 04:43:39.616607 13290 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786336863388938-23197-0/minicluster-data/master-0/data/instance:
uuid: "ced6bb2b892b4c0a9db68c3798caf0a4"
format_stamp: "Formatted at 2026-08-10 04:43:39 on dist-test-slave-bqcl"
server_key: "83ddbe3433f049189cc1387f8cc689ec"
server_key_iv: "44c44b33be654d51cfd7c04107d511b3"
server_key_version: "encryptionkey@0"
I20260810 04:43:39.616918 13290 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786336863388938-23197-0/minicluster-data/master-0/wal/instance:
uuid: "ced6bb2b892b4c0a9db68c3798caf0a4"
format_stamp: "Formatted at 2026-08-10 04:43:39 on dist-test-slave-bqcl"
server_key: "83ddbe3433f049189cc1387f8cc689ec"
server_key_iv: "44c44b33be654d51cfd7c04107d511b3"
server_key_version: "encryptionkey@0"
I20260810 04:43:39.618067 13290 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260810 04:43:39.618744 13311 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 04:43:39.618888 13290 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260810 04:43:39.618966 13290 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786336863388938-23197-0/minicluster-data/master-0/data,/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786336863388938-23197-0/minicluster-data/master-0/wal
uuid: "ced6bb2b892b4c0a9db68c3798caf0a4"
format_stamp: "Formatted at 2026-08-10 04:43:39 on dist-test-slave-bqcl"
server_key: "83ddbe3433f049189cc1387f8cc689ec"
server_key_iv: "44c44b33be654d51cfd7c04107d511b3"
server_key_version: "encryptionkey@0"
I20260810 04:43:39.619029 13290 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786336863388938-23197-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786336863388938-23197-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786336863388938-23197-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 04:43:39.628309 13290 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 04:43:39.629848 13290 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 04:43:39.629966 13290 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 04:43:39.633790 13290 rpc_server.cc:307] RPC server started. Bound to: 127.22.167.126:41211
I20260810 04:43:39.633811 13403 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.167.126:41211 every 8 connection(s)
I20260810 04:43:39.634119 13290 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786336863388938-23197-0/minicluster-data/master-0/data/info.pb
I20260810 04:43:39.634932 13407 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 04:43:39.636039 23197 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskemv_AM/build/release/bin/kudu as pid 13290
I20260810 04:43:39.636142 23197 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786336863388938-23197-0/minicluster-data/master-0/wal/instance
I20260810 04:43:39.636322 23197 external_mini_cluster.cc:1428] Setting key a9f7941e19da6332b6eb1255a6eca3c6
I20260810 04:43:39.636510 13407 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.22.167.126" port: 41211 } has no permanent_uuid. Determining permanent_uuid...
Aug 10 04:43:39 dist-test-slave-bqcl krb5kdc[13266](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786337019, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.167.126@KRBTEST.COM for kudu/127.22.167.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.22.167.125@KRBTEST.COM; defaulting to no policy
I20260810 04:43:39.646219 13411 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:43:39.637175 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.126:41211 (local address 127.0.0.1:48616)
0810 04:43:39.637344 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0810 04:43:39.637385 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0810 04:43:39.637911 (+ 526us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 04:43:39.638324 (+ 413us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 04:43:39.638327 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 04:43:39.638633 (+ 306us) client_negotiation.cc:189] Negotiated authn=SASL
0810 04:43:39.638839 (+ 206us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:43:39.638845 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:43:39.639818 (+ 973us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:43:39.639820 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:43:39.640866 (+ 1046us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:43:39.640869 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:43:39.641325 (+ 456us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:43:39.641326 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:43:39.641403 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:43:39.641835 (+ 432us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 04:43:39.641845 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0810 04:43:39.643335 (+ 1490us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 04:43:39.644881 (+ 1546us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 04:43:39.644887 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 04:43:39.644889 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 04:43:39.645090 (+ 201us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 04:43:39.645312 (+ 222us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 04:43:39.645313 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 04:43:39.645313 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 04:43:39.645395 (+ 82us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 04:43:39.645700 (+ 305us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 04:43:39.645705 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 04:43:39.645764 (+ 59us) client_negotiation.cc:769] Sending connection context
0810 04:43:39.645843 (+ 79us) client_negotiation.cc:240] Negotiation successful
0810 04:43:39.646047 (+ 204us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":126,"mutex_wait_us":78,"thread_start_us":83,"threads_started":1}
I20260810 04:43:39.646225 13413 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:43:39.637209 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48616 (local address 127.22.167.126:41211)
0810 04:43:39.638089 (+ 880us) server_negotiation.cc:206] Beginning negotiation
0810 04:43:39.638094 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 04:43:39.638109 (+ 15us) server_negotiation.cc:407] Connection header received
0810 04:43:39.638205 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:43:39.638206 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:43:39.638221 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:43:39.638286 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0810 04:43:39.638929 (+ 643us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:43:39.639726 (+ 797us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:43:39.640940 (+ 1214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:43:39.641247 (+ 307us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:43:39.641321 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:43:39.643412 (+ 2091us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 04:43:39.643419 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 04:43:39.643420 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 04:43:39.643426 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0810 04:43:39.644806 (+ 1380us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 04:43:39.645168 (+ 362us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 04:43:39.645170 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 04:43:39.645171 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 04:43:39.645254 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 04:43:39.645448 (+ 194us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 04:43:39.645449 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 04:43:39.645449 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 04:43:39.645639 (+ 190us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 04:43:39.645709 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0810 04:43:39.645875 (+ 166us) server_negotiation.cc:299] Negotiation successful
0810 04:43:39.645977 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":853,"thread_start_us":66,"threads_started":1}
I20260810 04:43:39.646919 13407 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.22.167.125" port: 39737 } has no permanent_uuid. Determining permanent_uuid...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.167.125@KRBTEST.COM" created.
I20260810 04:43:39.647222 13411 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:43:39.647108 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.125:39737 (local address 127.0.0.1:52054)
0810 04:43:39.647150 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0810 04:43:39.647197 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.167.125:39737: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
W20260810 04:43:39.647356 13321 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.22.167.125:39737: connect: Connection refused (error 111)
I20260810 04:43:39.647739 13411 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:43:39.647657 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.125:39737 (local address 127.0.0.1:52064)
0810 04:43:39.647699 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0810 04:43:39.647722 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.167.125:39737: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260810 04:43:39.647853 13407 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.22.167.125:39737: Network error: Client connection negotiation failed: client connection to 127.22.167.125:39737: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.167.125 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786336863388938-23197-0/krb5kdc/kudu_127.22.167.125.keytab.
Entry for principal kudu/127.22.167.125 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786336863388938-23197-0/krb5kdc/kudu_127.22.167.125.keytab.
I20260810 04:43:39.660099 23197 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskemv_AM/build/release/bin/kudu
/tmp/dist-test-taskemv_AM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786336863388938-23197-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786336863388938-23197-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786336863388938-23197-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786336863388938-23197-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.167.125:39737
--webserver_interface=127.22.167.125
--webserver_port=0
--keytab_file=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786336863388938-23197-0/krb5kdc/kudu_127.22.167.125.keytab
--principal=kudu/127.22.167.125
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.167.84:33893
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.167.126:41211,127.22.167.125:39737,127.22.167.124:34159
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786336863388938-23197-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786336863388938-23197-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786336863388938-23197-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 04:43:39.697039 13407 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.22.167.125" port: 39737 } attempt: 1
I20260810 04:43:39.697535 13411 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:43:39.697404 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.125:39737 (local address 127.0.0.1:52074)
0810 04:43:39.697465 (+ 61us) negotiation.cc:107] Waiting for socket to connect
0810 04:43:39.697505 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.167.125:39737: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
I20260810 04:43:39.697839 13411 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:43:39.697763 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.125:39737 (local address 127.0.0.1:52084)
0810 04:43:39.697796 (+ 33us) negotiation.cc:107] Waiting for socket to connect
0810 04:43:39.697820 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.167.125:39737: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":23}
W20260810 04:43:39.697934 13407 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.22.167.125:39737: Network error: Client connection negotiation failed: client connection to 127.22.167.125:39737: connect: Connection refused (error 111)
W20260810 04:43:39.741160 13419 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 04:43:39.741357 13419 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 04:43:39.741389 13419 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 04:43:39.742482 13419 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 04:43:39.742527 13419 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 04:43:39.742547 13419 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 04:43:39.742563 13419 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 04:43:39.742579 13419 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 04:43:39.744014 13419 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.167.84:33893
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786336863388938-23197-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786336863388938-23197-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.22.167.126:41211,127.22.167.125:39737,127.22.167.124:34159
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786336863388938-23197-0/krb5kdc/kudu_127.22.167.125.keytab
--principal=kudu/127.22.167.125
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.167.125:39737
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786336863388938-23197-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.22.167.125
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13419
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786336863388938-23197-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 03:54:01 UTC on ccycloud.quasar-icijmt.root.comops.site
build id 4577009
I20260810 04:43:39.744253 13419 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 04:43:39.744447 13419 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 04:43:39.748974 13437 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 04:43:39.748960 13431 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 04:43:39.748951 13427 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 04:43:39.749245 13419 server_base.cc:1034] running on GCE node
I20260810 04:43:39.749495 13419 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 04:43:39.749771 13419 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 04:43:39.750907 13419 hybrid_clock.cc:648] HybridClock initialized: now 1786337019750891 us; error 28 us; skew 500 ppm
Aug 10 04:43:39 dist-test-slave-bqcl krb5kdc[13266](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786337019, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.167.125@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 04:43:39.753182 13419 init.cc:377] Logged in from keytab as kudu/127.22.167.125@KRBTEST.COM (short username kudu)
I20260810 04:43:39.753566 13419 webserver.cc:460] Webserver started at http://127.22.167.125:39397/ using document root <none> and password file <none>
I20260810 04:43:39.753758 13419 fs_manager.cc:359] Metadata directory not provided
I20260810 04:43:39.753800 13419 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 04:43:39.753914 13419 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 04:43:39.754920 13419 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786336863388938-23197-0/minicluster-data/master-1/data/instance:
uuid: "80b7340e2c7645bdb5f15237334a54e0"
format_stamp: "Formatted at 2026-08-10 04:43:39 on dist-test-slave-bqcl"
server_key: "07c974cfbf6d881e86612c8e1988f6eb"
server_key_iv: "876f5aa4e0ed7978496b8050dfb7c593"
server_key_version: "encryptionkey@0"
I20260810 04:43:39.755322 13419 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786336863388938-23197-0/minicluster-data/master-1/wal/instance:
uuid: "80b7340e2c7645bdb5f15237334a54e0"
format_stamp: "Formatted at 2026-08-10 04:43:39 on dist-test-slave-bqcl"
server_key: "07c974cfbf6d881e86612c8e1988f6eb"
server_key_iv: "876f5aa4e0ed7978496b8050dfb7c593"
server_key_version: "encryptionkey@0"
I20260810 04:43:39.756506 13419 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260810 04:43:39.757143 13448 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 04:43:39.757301 13419 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260810 04:43:39.757385 13419 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786336863388938-23197-0/minicluster-data/master-1/data,/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786336863388938-23197-0/minicluster-data/master-1/wal
uuid: "80b7340e2c7645bdb5f15237334a54e0"
format_stamp: "Formatted at 2026-08-10 04:43:39 on dist-test-slave-bqcl"
server_key: "07c974cfbf6d881e86612c8e1988f6eb"
server_key_iv: "876f5aa4e0ed7978496b8050dfb7c593"
server_key_version: "encryptionkey@0"
I20260810 04:43:39.757437 13419 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786336863388938-23197-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786336863388938-23197-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786336863388938-23197-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 04:43:39.766067 13407 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.22.167.125" port: 39737 } attempt: 2
I20260810 04:43:39.766428 13411 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:43:39.766322 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.125:39737 (local address 127.0.0.1:52100)
0810 04:43:39.766373 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0810 04:43:39.766401 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.167.125:39737: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
I20260810 04:43:39.766762 13411 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:43:39.766679 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.125:39737 (local address 127.0.0.1:52104)
0810 04:43:39.766717 (+ 38us) negotiation.cc:107] Waiting for socket to connect
0810 04:43:39.766743 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.167.125:39737: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":25}
W20260810 04:43:39.766835 13407 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.22.167.125:39737: Network error: Client connection negotiation failed: client connection to 127.22.167.125:39737: connect: Connection refused (error 111)
I20260810 04:43:39.773564 13419 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 04:43:39.775398 13419 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 04:43:39.775525 13419 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 04:43:39.779326 13419 rpc_server.cc:307] RPC server started. Bound to: 127.22.167.125:39737
I20260810 04:43:39.779359 13526 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.167.125:39737 every 8 connection(s)
I20260810 04:43:39.779670 13419 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786336863388938-23197-0/minicluster-data/master-1/data/info.pb
I20260810 04:43:39.780467 13527 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 04:43:39.781913 13527 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.22.167.126" port: 41211 } has no permanent_uuid. Determining permanent_uuid...
I20260810 04:43:39.784678 23197 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskemv_AM/build/release/bin/kudu as pid 13419
I20260810 04:43:39.784765 23197 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786336863388938-23197-0/minicluster-data/master-1/wal/instance
I20260810 04:43:39.784909 23197 external_mini_cluster.cc:1428] Setting key 2de35ee59547a234ac4b06a433a2dcc1
Aug 10 04:43:39 dist-test-slave-bqcl krb5kdc[13266](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786337019, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.167.125@KRBTEST.COM for kudu/127.22.167.126@KRBTEST.COM
I20260810 04:43:39.790333 13413 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:43:39.782554 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48626 (local address 127.22.167.126:41211)
0810 04:43:39.782622 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0810 04:43:39.782624 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 04:43:39.782790 (+ 166us) server_negotiation.cc:407] Connection header received
0810 04:43:39.783230 (+ 440us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:43:39.783231 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:43:39.783241 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:43:39.783291 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0810 04:43:39.783836 (+ 545us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:43:39.784307 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:43:39.785335 (+ 1028us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:43:39.785607 (+ 272us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:43:39.785681 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:43:39.787702 (+ 2021us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 04:43:39.787704 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 04:43:39.787704 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 04:43:39.787713 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0810 04:43:39.788954 (+ 1241us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 04:43:39.789413 (+ 459us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 04:43:39.789414 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 04:43:39.789415 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 04:43:39.789466 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 04:43:39.789735 (+ 269us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 04:43:39.789735 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 04:43:39.789736 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 04:43:39.789826 (+ 90us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 04:43:39.789870 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0810 04:43:39.790209 (+ 339us) server_negotiation.cc:299] Negotiation successful
0810 04:43:39.790274 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260810 04:43:39.790576 13530 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:43:39.782528 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.126:41211 (local address 127.0.0.1:48626)
0810 04:43:39.782684 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0810 04:43:39.782719 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0810 04:43:39.783157 (+ 438us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 04:43:39.783348 (+ 191us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 04:43:39.783352 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 04:43:39.783591 (+ 239us) client_negotiation.cc:189] Negotiated authn=SASL
0810 04:43:39.783773 (+ 182us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:43:39.783777 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:43:39.784364 (+ 587us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:43:39.784364 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:43:39.785259 (+ 895us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:43:39.785261 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:43:39.785678 (+ 417us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:43:39.785679 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:43:39.785772 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:43:39.786203 (+ 431us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 04:43:39.786218 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0810 04:43:39.787608 (+ 1390us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 04:43:39.789038 (+ 1430us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 04:43:39.789046 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 04:43:39.789049 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 04:43:39.789344 (+ 295us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 04:43:39.789522 (+ 178us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 04:43:39.789523 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 04:43:39.789523 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 04:43:39.789674 (+ 151us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 04:43:39.789879 (+ 205us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 04:43:39.789885 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 04:43:39.790042 (+ 157us) client_negotiation.cc:769] Sending connection context
0810 04:43:39.790196 (+ 154us) client_negotiation.cc:240] Negotiation successful
0810 04:43:39.790359 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":79,"threads_started":1}
I20260810 04:43:39.791118 13527 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.22.167.125" port: 39737 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.22.167.124@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.167.124@KRBTEST.COM" created.
Aug 10 04:43:39 dist-test-slave-bqcl krb5kdc[13266](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786337019, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.167.125@KRBTEST.COM for kudu/127.22.167.125@KRBTEST.COM
I20260810 04:43:39.799376 13530 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:43:39.791336 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.125:39737 (local address 127.0.0.1:52106)
0810 04:43:39.791383 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0810 04:43:39.791394 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0810 04:43:39.791436 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 04:43:39.791670 (+ 234us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 04:43:39.791670 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 04:43:39.791807 (+ 137us) client_negotiation.cc:189] Negotiated authn=SASL
0810 04:43:39.791862 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:43:39.791863 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:43:39.792664 (+ 801us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:43:39.792666 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:43:39.793410 (+ 744us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:43:39.793413 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:43:39.793867 (+ 454us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:43:39.793868 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:43:39.793929 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:43:39.794255 (+ 326us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 04:43:39.794259 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0810 04:43:39.796642 (+ 2383us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 04:43:39.798383 (+ 1741us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 04:43:39.798384 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 04:43:39.798384 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 04:43:39.798536 (+ 152us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 04:43:39.798756 (+ 220us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 04:43:39.798757 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 04:43:39.798757 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 04:43:39.798821 (+ 64us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 04:43:39.799055 (+ 234us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 04:43:39.799056 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 04:43:39.799100 (+ 44us) client_negotiation.cc:769] Sending connection context
0810 04:43:39.799168 (+ 68us) client_negotiation.cc:240] Negotiation successful
0810 04:43:39.799260 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":27}
I20260810 04:43:39.799405 13533 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:43:39.791395 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52106 (local address 127.22.167.125:39737)
0810 04:43:39.791540 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0810 04:43:39.791545 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 04:43:39.791564 (+ 19us) server_negotiation.cc:407] Connection header received
0810 04:43:39.791584 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:43:39.791586 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:43:39.791596 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:43:39.791655 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0810 04:43:39.791930 (+ 275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:43:39.792582 (+ 652us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:43:39.793481 (+ 899us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:43:39.793798 (+ 317us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:43:39.793848 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:43:39.796723 (+ 2875us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 04:43:39.796732 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 04:43:39.796733 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 04:43:39.796738 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0810 04:43:39.798309 (+ 1571us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 04:43:39.798619 (+ 310us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 04:43:39.798622 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 04:43:39.798622 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 04:43:39.798693 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 04:43:39.798879 (+ 186us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 04:43:39.798880 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 04:43:39.798880 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 04:43:39.799002 (+ 122us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 04:43:39.799057 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0810 04:43:39.799197 (+ 140us) server_negotiation.cc:299] Negotiation successful
0810 04:43:39.799318 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":49,"server-negotiator.queue_time_us":117,"thread_start_us":80,"threads_started":1}
I20260810 04:43:39.800104 13527 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.22.167.124" port: 34159 } has no permanent_uuid. Determining permanent_uuid...
I20260810 04:43:39.800419 13530 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:43:39.800299 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.124:34159 (local address 127.0.0.1:40570)
0810 04:43:39.800342 (+ 43us) negotiation.cc:107] Waiting for socket to connect
0810 04:43:39.800395 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.167.124:34159: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":26}
W20260810 04:43:39.800568 13449 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.22.167.124:34159: connect: Connection refused (error 111)
I20260810 04:43:39.800956 13530 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:43:39.800854 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.124:34159 (local address 127.0.0.1:40584)
0810 04:43:39.800900 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0810 04:43:39.800931 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.167.124:34159: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
W20260810 04:43:39.801061 13527 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.22.167.124:34159: Network error: Client connection negotiation failed: client connection to 127.22.167.124:34159: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.167.124 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786336863388938-23197-0/krb5kdc/kudu_127.22.167.124.keytab.
Entry for principal kudu/127.22.167.124 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786336863388938-23197-0/krb5kdc/kudu_127.22.167.124.keytab.
I20260810 04:43:39.810637 23197 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskemv_AM/build/release/bin/kudu
/tmp/dist-test-taskemv_AM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786336863388938-23197-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786336863388938-23197-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786336863388938-23197-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786336863388938-23197-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.167.124:34159
--webserver_interface=127.22.167.124
--webserver_port=0
--keytab_file=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786336863388938-23197-0/krb5kdc/kudu_127.22.167.124.keytab
--principal=kudu/127.22.167.124
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.167.84:33893
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.167.126:41211,127.22.167.125:39737,127.22.167.124:34159
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786336863388938-23197-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786336863388938-23197-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786336863388938-23197-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 04:43:39.850266 13527 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.22.167.124" port: 34159 } attempt: 1
I20260810 04:43:39.850718 13530 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:43:39.850573 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.124:34159 (local address 127.0.0.1:40586)
0810 04:43:39.850640 (+ 67us) negotiation.cc:107] Waiting for socket to connect
0810 04:43:39.850684 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.167.124:34159: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
I20260810 04:43:39.851069 13530 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:43:39.850967 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.124:34159 (local address 127.0.0.1:40598)
0810 04:43:39.851010 (+ 43us) negotiation.cc:107] Waiting for socket to connect
0810 04:43:39.851045 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.167.124:34159: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
W20260810 04:43:39.851171 13527 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.22.167.124:34159: Network error: Client connection negotiation failed: client connection to 127.22.167.124:34159: connect: Connection refused (error 111)
I20260810 04:43:39.857959 13407 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.22.167.125" port: 39737 } attempt: 3
Aug 10 04:43:39 dist-test-slave-bqcl krb5kdc[13266](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786337019, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.167.126@KRBTEST.COM for kudu/127.22.167.125@KRBTEST.COM
I20260810 04:43:39.868009 13533 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:43:39.858330 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52110 (local address 127.22.167.125:39737)
0810 04:43:39.858376 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0810 04:43:39.858378 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 04:43:39.858396 (+ 18us) server_negotiation.cc:407] Connection header received
0810 04:43:39.858456 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:43:39.858456 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:43:39.858469 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:43:39.858520 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0810 04:43:39.858840 (+ 320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:43:39.859635 (+ 795us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:43:39.860287 (+ 652us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:43:39.860620 (+ 333us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:43:39.860692 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:43:39.865775 (+ 5083us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 04:43:39.865777 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 04:43:39.865777 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 04:43:39.865783 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0810 04:43:39.867036 (+ 1253us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 04:43:39.867396 (+ 360us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 04:43:39.867396 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 04:43:39.867397 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 04:43:39.867444 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 04:43:39.867612 (+ 168us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 04:43:39.867612 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 04:43:39.867613 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 04:43:39.867705 (+ 92us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 04:43:39.867749 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0810 04:43:39.867889 (+ 140us) server_negotiation.cc:299] Negotiation successful
0810 04:43:39.867949 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260810 04:43:39.868059 13411 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:43:39.858282 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.125:39737 (local address 127.0.0.1:52110)
0810 04:43:39.858340 (+ 58us) negotiation.cc:107] Waiting for socket to connect
0810 04:43:39.858352 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0810 04:43:39.858407 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 04:43:39.858537 (+ 130us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 04:43:39.858538 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 04:43:39.858705 (+ 167us) client_negotiation.cc:189] Negotiated authn=SASL
0810 04:43:39.858773 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:43:39.858776 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:43:39.859708 (+ 932us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:43:39.859709 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:43:39.860220 (+ 511us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:43:39.860222 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:43:39.860716 (+ 494us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:43:39.860718 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:43:39.860783 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:43:39.864261 (+ 3478us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 04:43:39.864265 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0810 04:43:39.865696 (+ 1431us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 04:43:39.867113 (+ 1417us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 04:43:39.867114 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 04:43:39.867115 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 04:43:39.867331 (+ 216us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 04:43:39.867494 (+ 163us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 04:43:39.867495 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 04:43:39.867495 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 04:43:39.867559 (+ 64us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 04:43:39.867751 (+ 192us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 04:43:39.867752 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 04:43:39.867801 (+ 49us) client_negotiation.cc:769] Sending connection context
0810 04:43:39.867864 (+ 63us) client_negotiation.cc:240] Negotiation successful
0810 04:43:39.867962 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":36}
I20260810 04:43:39.868438 13407 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.22.167.124" port: 34159 } has no permanent_uuid. Determining permanent_uuid...
I20260810 04:43:39.868752 13411 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:43:39.868644 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.124:34159 (local address 127.0.0.1:40602)
0810 04:43:39.868692 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0810 04:43:39.868726 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.167.124:34159: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
I20260810 04:43:39.869059 13411 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:43:39.868975 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.124:34159 (local address 127.0.0.1:40618)
0810 04:43:39.869014 (+ 39us) negotiation.cc:107] Waiting for socket to connect
0810 04:43:39.869040 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.167.124:34159: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
W20260810 04:43:39.869145 13407 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.22.167.124:34159: Network error: Client connection negotiation failed: client connection to 127.22.167.124:34159: connect: Connection refused (error 111)
W20260810 04:43:39.892606 13536 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 04:43:39.892792 13536 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 04:43:39.892820 13536 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 04:43:39.893889 13536 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 04:43:39.893927 13536 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 04:43:39.893946 13536 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 04:43:39.893961 13536 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 04:43:39.893977 13536 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 04:43:39.895467 13536 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.167.84:33893
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786336863388938-23197-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786336863388938-23197-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.22.167.126:41211,127.22.167.125:39737,127.22.167.124:34159
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786336863388938-23197-0/krb5kdc/kudu_127.22.167.124.keytab
--principal=kudu/127.22.167.124
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.167.124:34159
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786336863388938-23197-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.22.167.124
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13536
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786336863388938-23197-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 03:54:01 UTC on ccycloud.quasar-icijmt.root.comops.site
build id 4577009
I20260810 04:43:39.895705 13536 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 04:43:39.895901 13536 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260810 04:43:39.900280 13407 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.22.167.124" port: 34159 } attempt: 1
W20260810 04:43:39.900408 13541 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 04:43:39.900390 13542 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 04:43:39.900404 13544 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 04:43:39.900609 13536 server_base.cc:1034] running on GCE node
I20260810 04:43:39.900671 13411 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:43:39.900561 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.124:34159 (local address 127.0.0.1:40622)
0810 04:43:39.900614 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0810 04:43:39.900639 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.167.124:34159: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
I20260810 04:43:39.900907 13536 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 04:43:39.900947 13411 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:43:39.900867 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.124:34159 (local address 127.0.0.1:40634)
0810 04:43:39.900902 (+ 35us) negotiation.cc:107] Waiting for socket to connect
0810 04:43:39.900925 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.167.124:34159: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":23}
W20260810 04:43:39.901033 13407 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.22.167.124:34159: Network error: Client connection negotiation failed: client connection to 127.22.167.124:34159: connect: Connection refused (error 111)
I20260810 04:43:39.901217 13536 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 04:43:39.902367 13536 hybrid_clock.cc:648] HybridClock initialized: now 1786337019902353 us; error 26 us; skew 500 ppm
Aug 10 04:43:39 dist-test-slave-bqcl krb5kdc[13266](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786337019, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.167.124@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 04:43:39.904685 13536 init.cc:377] Logged in from keytab as kudu/127.22.167.124@KRBTEST.COM (short username kudu)
I20260810 04:43:39.905087 13536 webserver.cc:460] Webserver started at http://127.22.167.124:35499/ using document root <none> and password file <none>
I20260810 04:43:39.905274 13536 fs_manager.cc:359] Metadata directory not provided
I20260810 04:43:39.905314 13536 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 04:43:39.905405 13536 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 04:43:39.906392 13536 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786336863388938-23197-0/minicluster-data/master-2/data/instance:
uuid: "33d9c156ea8449ec98a075ad65de8839"
format_stamp: "Formatted at 2026-08-10 04:43:39 on dist-test-slave-bqcl"
server_key: "06b89186be96b02cb40784701621090d"
server_key_iv: "3da02b6de5445c609e58907348dde5d8"
server_key_version: "encryptionkey@0"
I20260810 04:43:39.906680 13536 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786336863388938-23197-0/minicluster-data/master-2/wal/instance:
uuid: "33d9c156ea8449ec98a075ad65de8839"
format_stamp: "Formatted at 2026-08-10 04:43:39 on dist-test-slave-bqcl"
server_key: "06b89186be96b02cb40784701621090d"
server_key_iv: "3da02b6de5445c609e58907348dde5d8"
server_key_version: "encryptionkey@0"
I20260810 04:43:39.907817 13536 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260810 04:43:39.908417 13555 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 04:43:39.908566 13536 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260810 04:43:39.908643 13536 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786336863388938-23197-0/minicluster-data/master-2/data,/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786336863388938-23197-0/minicluster-data/master-2/wal
uuid: "33d9c156ea8449ec98a075ad65de8839"
format_stamp: "Formatted at 2026-08-10 04:43:39 on dist-test-slave-bqcl"
server_key: "06b89186be96b02cb40784701621090d"
server_key_iv: "3da02b6de5445c609e58907348dde5d8"
server_key_version: "encryptionkey@0"
I20260810 04:43:39.908703 13536 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786336863388938-23197-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786336863388938-23197-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786336863388938-23197-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 04:43:39.919313 13527 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.22.167.124" port: 34159 } attempt: 2
I20260810 04:43:39.919672 13530 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:43:39.919550 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.124:34159 (local address 127.0.0.1:40644)
0810 04:43:39.919601 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0810 04:43:39.919639 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.167.124:34159: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
I20260810 04:43:39.919971 13530 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:43:39.919897 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.124:34159 (local address 127.0.0.1:40648)
0810 04:43:39.919933 (+ 36us) negotiation.cc:107] Waiting for socket to connect
0810 04:43:39.919955 (+ 22us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.167.124:34159: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":24}
W20260810 04:43:39.920053 13527 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.22.167.124:34159: Network error: Client connection negotiation failed: client connection to 127.22.167.124:34159: connect: Connection refused (error 111)
I20260810 04:43:39.931669 13536 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 04:43:39.933416 13536 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 04:43:39.933544 13536 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 04:43:39.937341 13536 rpc_server.cc:307] RPC server started. Bound to: 127.22.167.124:34159
I20260810 04:43:39.937390 13636 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.167.124:34159 every 8 connection(s)
I20260810 04:43:39.937711 13536 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786336863388938-23197-0/minicluster-data/master-2/data/info.pb
I20260810 04:43:39.938592 13637 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 04:43:39.940032 13637 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.22.167.126" port: 41211 } has no permanent_uuid. Determining permanent_uuid...
I20260810 04:43:39.945111 23197 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskemv_AM/build/release/bin/kudu as pid 13536
I20260810 04:43:39.945192 23197 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786336863388938-23197-0/minicluster-data/master-2/wal/instance
Aug 10 04:43:39 dist-test-slave-bqcl krb5kdc[13266](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786337019, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.167.124@KRBTEST.COM for kudu/127.22.167.126@KRBTEST.COM
I20260810 04:43:39.945349 23197 external_mini_cluster.cc:1428] Setting key 2c92bbac94bc9a069e2dae5a3c0b2327
I20260810 04:43:39.948472 13413 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:43:39.940671 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48640 (local address 127.22.167.126:41211)
0810 04:43:39.940733 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0810 04:43:39.940735 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 04:43:39.940900 (+ 165us) server_negotiation.cc:407] Connection header received
0810 04:43:39.941361 (+ 461us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:43:39.941362 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:43:39.941374 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:43:39.941422 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0810 04:43:39.942020 (+ 598us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:43:39.942573 (+ 553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:43:39.943515 (+ 942us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:43:39.943715 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:43:39.943769 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:43:39.945814 (+ 2045us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 04:43:39.945816 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 04:43:39.945816 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 04:43:39.945824 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0810 04:43:39.947245 (+ 1421us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 04:43:39.947615 (+ 370us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 04:43:39.947616 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 04:43:39.947617 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 04:43:39.947678 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 04:43:39.947898 (+ 220us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 04:43:39.947899 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 04:43:39.947899 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 04:43:39.948005 (+ 106us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 04:43:39.948053 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0810 04:43:39.948348 (+ 295us) server_negotiation.cc:299] Negotiation successful
0810 04:43:39.948411 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260810 04:43:39.948681 13640 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:43:39.940613 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.126:41211 (local address 127.0.0.1:48640)
0810 04:43:39.940793 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0810 04:43:39.940828 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0810 04:43:39.941270 (+ 442us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 04:43:39.941502 (+ 232us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 04:43:39.941505 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 04:43:39.941753 (+ 248us) client_negotiation.cc:189] Negotiated authn=SASL
0810 04:43:39.941935 (+ 182us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:43:39.941940 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:43:39.942657 (+ 717us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:43:39.942659 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:43:39.943437 (+ 778us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:43:39.943440 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:43:39.943782 (+ 342us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:43:39.943783 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:43:39.943894 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:43:39.944306 (+ 412us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 04:43:39.944314 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0810 04:43:39.945730 (+ 1416us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 04:43:39.947332 (+ 1602us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 04:43:39.947339 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 04:43:39.947341 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 04:43:39.947550 (+ 209us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 04:43:39.947734 (+ 184us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 04:43:39.947735 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 04:43:39.947735 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 04:43:39.947838 (+ 103us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 04:43:39.948058 (+ 220us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 04:43:39.948063 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 04:43:39.948223 (+ 160us) client_negotiation.cc:769] Sending connection context
0810 04:43:39.948325 (+ 102us) client_negotiation.cc:240] Negotiation successful
0810 04:43:39.948471 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":92,"threads_started":1}
I20260810 04:43:39.949189 13637 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.22.167.125" port: 39737 } has no permanent_uuid. Determining permanent_uuid...
Aug 10 04:43:39 dist-test-slave-bqcl krb5kdc[13266](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786337019, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.167.124@KRBTEST.COM
Aug 10 04:43:39 dist-test-slave-bqcl krb5kdc[13266](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786337019, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.167.124@KRBTEST.COM for kudu/127.22.167.125@KRBTEST.COM
I20260810 04:43:39.954221 13642 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:43:39.945825 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40654 (local address 127.22.167.124:34159)
0810 04:43:39.945980 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0810 04:43:39.945984 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 04:43:39.946009 (+ 25us) server_negotiation.cc:407] Connection header received
0810 04:43:39.946031 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:43:39.946033 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:43:39.946047 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:43:39.946110 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0810 04:43:39.946817 (+ 707us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:43:39.947753 (+ 936us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:43:39.948719 (+ 966us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:43:39.949140 (+ 421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:43:39.949214 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:43:39.951352 (+ 2138us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 04:43:39.951360 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 04:43:39.951361 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 04:43:39.951367 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0810 04:43:39.953092 (+ 1725us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 04:43:39.953421 (+ 329us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 04:43:39.953424 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 04:43:39.953425 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 04:43:39.953489 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 04:43:39.953684 (+ 195us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 04:43:39.953684 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 04:43:39.953685 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 04:43:39.953826 (+ 141us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 04:43:39.953976 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0810 04:43:39.954022 (+ 46us) server_negotiation.cc:299] Negotiation successful
0810 04:43:39.954098 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":124,"thread_start_us":81,"threads_started":1}
I20260810 04:43:39.955725 13640 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:43:39.949410 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.125:39737 (local address 127.0.0.1:52122)
0810 04:43:39.949511 (+ 101us) negotiation.cc:107] Waiting for socket to connect
0810 04:43:39.949521 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0810 04:43:39.949566 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 04:43:39.949694 (+ 128us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 04:43:39.949695 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 04:43:39.949819 (+ 124us) client_negotiation.cc:189] Negotiated authn=SASL
0810 04:43:39.949863 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:43:39.949864 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:43:39.950508 (+ 644us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:43:39.950509 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:43:39.951005 (+ 496us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:43:39.951007 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:43:39.951336 (+ 329us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:43:39.951337 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:43:39.951407 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:43:39.951722 (+ 315us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 04:43:39.951726 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0810 04:43:39.953053 (+ 1327us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 04:43:39.954673 (+ 1620us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 04:43:39.954674 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 04:43:39.954674 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 04:43:39.954865 (+ 191us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 04:43:39.955184 (+ 319us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 04:43:39.955185 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 04:43:39.955185 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 04:43:39.955244 (+ 59us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 04:43:39.955449 (+ 205us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 04:43:39.955450 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 04:43:39.955488 (+ 38us) client_negotiation.cc:769] Sending connection context
0810 04:43:39.955548 (+ 60us) client_negotiation.cc:240] Negotiation successful
0810 04:43:39.955641 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":87}
I20260810 04:43:39.955744 13533 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:43:39.949469 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52122 (local address 127.22.167.125:39737)
0810 04:43:39.949509 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0810 04:43:39.949510 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0810 04:43:39.949554 (+ 44us) server_negotiation.cc:407] Connection header received
0810 04:43:39.949622 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:43:39.949623 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:43:39.949632 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:43:39.949690 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0810 04:43:39.949932 (+ 242us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:43:39.950436 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:43:39.951083 (+ 647us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:43:39.951267 (+ 184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:43:39.951323 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:43:39.953131 (+ 1808us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 04:43:39.953133 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 04:43:39.953133 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 04:43:39.953141 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0810 04:43:39.954604 (+ 1463us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 04:43:39.955056 (+ 452us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 04:43:39.955057 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 04:43:39.955057 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 04:43:39.955124 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 04:43:39.955293 (+ 169us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 04:43:39.955294 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 04:43:39.955294 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 04:43:39.955399 (+ 105us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 04:43:39.955452 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0810 04:43:39.955592 (+ 140us) server_negotiation.cc:299] Negotiation successful
0810 04:43:39.955659 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260810 04:43:39.956048 13637 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.22.167.124" port: 34159 } has no permanent_uuid. Determining permanent_uuid...
Aug 10 04:43:39 dist-test-slave-bqcl krb5kdc[13266](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786337019, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.167.126@KRBTEST.COM
Aug 10 04:43:39 dist-test-slave-bqcl krb5kdc[13266](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786337019, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.167.124@KRBTEST.COM for kudu/127.22.167.124@KRBTEST.COM
I20260810 04:43:39.961742 13413 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:43:39.955066 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48654 (local address 127.22.167.126:41211)
0810 04:43:39.955105 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0810 04:43:39.955106 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0810 04:43:39.955114 (+ 8us) server_negotiation.cc:407] Connection header received
0810 04:43:39.955174 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:43:39.955175 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:43:39.955183 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:43:39.955226 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0810 04:43:39.955619 (+ 393us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:43:39.956178 (+ 559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:43:39.956990 (+ 812us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:43:39.957168 (+ 178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:43:39.957209 (+ 41us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:43:39.959194 (+ 1985us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 04:43:39.959196 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 04:43:39.959196 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 04:43:39.959202 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0810 04:43:39.960660 (+ 1458us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 04:43:39.961027 (+ 367us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 04:43:39.961027 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 04:43:39.961028 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 04:43:39.961081 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 04:43:39.961266 (+ 185us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 04:43:39.961266 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 04:43:39.961267 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 04:43:39.961358 (+ 91us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 04:43:39.961534 (+ 176us) server_negotiation.cc:1035] Waiting for connection context
0810 04:43:39.961568 (+ 34us) server_negotiation.cc:299] Negotiation successful
0810 04:43:39.961654 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260810 04:43:39.962255 13640 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:43:39.956262 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.124:34159 (local address 127.0.0.1:40656)
0810 04:43:39.956311 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0810 04:43:39.956320 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0810 04:43:39.956354 (+ 34us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 04:43:39.956492 (+ 138us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 04:43:39.956493 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 04:43:39.956598 (+ 105us) client_negotiation.cc:189] Negotiated authn=SASL
0810 04:43:39.956623 (+ 25us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:43:39.956624 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:43:39.957345 (+ 721us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:43:39.957346 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:43:39.957917 (+ 571us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:43:39.957919 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:43:39.958410 (+ 491us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:43:39.958411 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:43:39.958445 (+ 34us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:43:39.958748 (+ 303us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 04:43:39.958753 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0810 04:43:39.959882 (+ 1129us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 04:43:39.961338 (+ 1456us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 04:43:39.961339 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 04:43:39.961339 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 04:43:39.961485 (+ 146us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 04:43:39.961641 (+ 156us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 04:43:39.961641 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 04:43:39.961642 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 04:43:39.961695 (+ 53us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 04:43:39.961941 (+ 246us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 04:43:39.961942 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 04:43:39.961979 (+ 37us) client_negotiation.cc:769] Sending connection context
0810 04:43:39.962039 (+ 60us) client_negotiation.cc:240] Negotiation successful
0810 04:43:39.962161 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":30}
I20260810 04:43:39.962265 13642 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:43:39.956295 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40656 (local address 127.22.167.124:34159)
0810 04:43:39.956407 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0810 04:43:39.956409 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 04:43:39.956413 (+ 4us) server_negotiation.cc:407] Connection header received
0810 04:43:39.956426 (+ 13us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:43:39.956427 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:43:39.956435 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:43:39.956477 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0810 04:43:39.956682 (+ 205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:43:39.957276 (+ 594us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:43:39.957992 (+ 716us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:43:39.958335 (+ 343us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:43:39.958413 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:43:39.959958 (+ 1545us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 04:43:39.959959 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 04:43:39.959960 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 04:43:39.959965 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0810 04:43:39.961288 (+ 1323us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 04:43:39.961541 (+ 253us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 04:43:39.961542 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 04:43:39.961542 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 04:43:39.961590 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 04:43:39.961751 (+ 161us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 04:43:39.961752 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 04:43:39.961752 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 04:43:39.961877 (+ 125us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 04:43:39.961936 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0810 04:43:39.962064 (+ 128us) server_negotiation.cc:299] Negotiation successful
0810 04:43:39.962191 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":65,"server-negotiator.queue_time_us":101}
I20260810 04:43:39.963814 13637 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 33d9c156ea8449ec98a075ad65de8839: Bootstrap starting.
I20260810 04:43:39.964691 13637 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 33d9c156ea8449ec98a075ad65de8839: Neither blocks nor log segments found. Creating new log.
I20260810 04:43:39.964967 13637 log.cc:824] T 00000000000000000000000000000000 P 33d9c156ea8449ec98a075ad65de8839: Log is configured to *not* fsync() on all Append() calls
I20260810 04:43:39.965648 13637 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 33d9c156ea8449ec98a075ad65de8839: No bootstrap required, opened a new log
Aug 10 04:43:39 dist-test-slave-bqcl krb5kdc[13266](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786337019, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.167.125@KRBTEST.COM
I20260810 04:43:39.967322 13637 raft_consensus.cc:348] T 00000000000000000000000000000000 P 33d9c156ea8449ec98a075ad65de8839 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ced6bb2b892b4c0a9db68c3798caf0a4" member_type: VOTER last_known_addr { host: "127.22.167.126" port: 41211 } } peers { permanent_uuid: "80b7340e2c7645bdb5f15237334a54e0" member_type: VOTER last_known_addr { host: "127.22.167.125" port: 39737 } } peers { permanent_uuid: "33d9c156ea8449ec98a075ad65de8839" member_type: VOTER last_known_addr { host: "127.22.167.124" port: 34159 } }
I20260810 04:43:39.967473 13637 raft_consensus.cc:374] T 00000000000000000000000000000000 P 33d9c156ea8449ec98a075ad65de8839 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 04:43:39.967520 13637 raft_consensus.cc:729] T 00000000000000000000000000000000 P 33d9c156ea8449ec98a075ad65de8839 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 33d9c156ea8449ec98a075ad65de8839, State: Initialized, Role: FOLLOWER
I20260810 04:43:39.967646 13637 consensus_queue.cc:260] T 00000000000000000000000000000000 P 33d9c156ea8449ec98a075ad65de8839 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ced6bb2b892b4c0a9db68c3798caf0a4" member_type: VOTER last_known_addr { host: "127.22.167.126" port: 41211 } } peers { permanent_uuid: "80b7340e2c7645bdb5f15237334a54e0" member_type: VOTER last_known_addr { host: "127.22.167.125" port: 39737 } } peers { permanent_uuid: "33d9c156ea8449ec98a075ad65de8839" member_type: VOTER last_known_addr { host: "127.22.167.124" port: 34159 } }
I20260810 04:43:39.967901 13645 sys_catalog.cc:455] T 00000000000000000000000000000000 P 33d9c156ea8449ec98a075ad65de8839 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ced6bb2b892b4c0a9db68c3798caf0a4" member_type: VOTER last_known_addr { host: "127.22.167.126" port: 41211 } } peers { permanent_uuid: "80b7340e2c7645bdb5f15237334a54e0" member_type: VOTER last_known_addr { host: "127.22.167.125" port: 39737 } } peers { permanent_uuid: "33d9c156ea8449ec98a075ad65de8839" member_type: VOTER last_known_addr { host: "127.22.167.124" port: 34159 } } }
I20260810 04:43:39.967954 13637 sys_catalog.cc:564] T 00000000000000000000000000000000 P 33d9c156ea8449ec98a075ad65de8839 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 04:43:39.967976 13645 sys_catalog.cc:458] T 00000000000000000000000000000000 P 33d9c156ea8449ec98a075ad65de8839 [sys.catalog]: This master's current role is: FOLLOWER
I20260810 04:43:39.969192 13533 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:43:39.962600 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52124 (local address 127.22.167.125:39737)
0810 04:43:39.962635 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0810 04:43:39.962637 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 04:43:39.962645 (+ 8us) server_negotiation.cc:407] Connection header received
0810 04:43:39.962682 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:43:39.962683 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:43:39.962691 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:43:39.962724 (+ 33us) server_negotiation.cc:226] Negotiated authn=SASL
0810 04:43:39.963154 (+ 430us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:43:39.963784 (+ 630us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:43:39.964411 (+ 627us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:43:39.964677 (+ 266us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:43:39.964745 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:43:39.966648 (+ 1903us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 04:43:39.966650 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 04:43:39.966650 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 04:43:39.966657 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 04:43:39.968053 (+ 1396us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 04:43:39.968341 (+ 288us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 04:43:39.968342 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 04:43:39.968342 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 04:43:39.968404 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 04:43:39.968664 (+ 260us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 04:43:39.968665 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 04:43:39.968666 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 04:43:39.968794 (+ 128us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 04:43:39.968986 (+ 192us) server_negotiation.cc:1035] Waiting for connection context
0810 04:43:39.969029 (+ 43us) server_negotiation.cc:299] Negotiation successful
0810 04:43:39.969105 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260810 04:43:39.969347 13669 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
I20260810 04:43:39.970805 23197 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 04:43:39.970178 13536 master_runner.cc:186] Error getting master registration for 127.22.167.126:41211: OK, instance_id { permanent_uuid: "ced6bb2b892b4c0a9db68c3798caf0a4" instance_seqno: 1786337019632699 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260810 04:43:39.972867 13536 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260810 04:43:39.976166 13407 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.22.167.124" port: 34159 } attempt: 2
I20260810 04:43:39.979810 13642 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:43:39.972326 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40664 (local address 127.22.167.124:34159)
0810 04:43:39.972407 (+ 81us) server_negotiation.cc:206] Beginning negotiation
0810 04:43:39.972409 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 04:43:39.972565 (+ 156us) server_negotiation.cc:407] Connection header received
0810 04:43:39.972674 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:43:39.972675 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:43:39.972684 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:43:39.972712 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0810 04:43:39.973369 (+ 657us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:43:39.974068 (+ 699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:43:39.975122 (+ 1054us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:43:39.975383 (+ 261us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:43:39.975435 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:43:39.976810 (+ 1375us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 04:43:39.976811 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 04:43:39.976811 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 04:43:39.976818 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 04:43:39.978365 (+ 1547us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 04:43:39.978740 (+ 375us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 04:43:39.978742 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 04:43:39.978742 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 04:43:39.978796 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 04:43:39.978995 (+ 199us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 04:43:39.978996 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 04:43:39.978996 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 04:43:39.979139 (+ 143us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 04:43:39.979203 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0810 04:43:39.979647 (+ 444us) server_negotiation.cc:299] Negotiation successful
0810 04:43:39.979735 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260810 04:43:39.979872 13533 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:43:39.972549 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52130 (local address 127.22.167.125:39737)
0810 04:43:39.972582 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0810 04:43:39.972584 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 04:43:39.972702 (+ 118us) server_negotiation.cc:407] Connection header received
0810 04:43:39.972757 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:43:39.972758 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:43:39.972767 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:43:39.972813 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0810 04:43:39.974193 (+ 1380us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:43:39.974613 (+ 420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:43:39.975310 (+ 697us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:43:39.975558 (+ 248us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:43:39.975595 (+ 37us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:43:39.977245 (+ 1650us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 04:43:39.977246 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 04:43:39.977246 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 04:43:39.977252 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0810 04:43:39.978685 (+ 1433us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 04:43:39.979060 (+ 375us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 04:43:39.979061 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 04:43:39.979061 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 04:43:39.979116 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 04:43:39.979321 (+ 205us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 04:43:39.979322 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 04:43:39.979322 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 04:43:39.979438 (+ 116us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 04:43:39.979515 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0810 04:43:39.979723 (+ 208us) server_negotiation.cc:299] Negotiation successful
0810 04:43:39.979791 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260810 04:43:39.980217 13413 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:43:39.972367 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48666 (local address 127.22.167.126:41211)
0810 04:43:39.972427 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0810 04:43:39.972429 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 04:43:39.972606 (+ 177us) server_negotiation.cc:407] Connection header received
0810 04:43:39.972817 (+ 211us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:43:39.972818 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:43:39.972828 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:43:39.972854 (+ 26us) server_negotiation.cc:226] Negotiated authn=SASL
0810 04:43:39.973417 (+ 563us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:43:39.974094 (+ 677us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:43:39.975148 (+ 1054us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:43:39.975342 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:43:39.975394 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:43:39.976920 (+ 1526us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 04:43:39.976921 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 04:43:39.976922 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 04:43:39.976928 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0810 04:43:39.979041 (+ 2113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 04:43:39.979392 (+ 351us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 04:43:39.979393 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 04:43:39.979393 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 04:43:39.979447 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 04:43:39.979629 (+ 182us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 04:43:39.979630 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 04:43:39.979630 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 04:43:39.979750 (+ 120us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 04:43:39.979837 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0810 04:43:39.980055 (+ 218us) server_negotiation.cc:299] Negotiation successful
0810 04:43:39.980125 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
Aug 10 04:43:39 dist-test-slave-bqcl krb5kdc[13266](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786337019, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.167.126@KRBTEST.COM for kudu/127.22.167.124@KRBTEST.COM
I20260810 04:43:39.982810 13411 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:43:39.976410 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.124:34159 (local address 127.0.0.1:40672)
0810 04:43:39.976447 (+ 37us) negotiation.cc:107] Waiting for socket to connect
0810 04:43:39.976456 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0810 04:43:39.976493 (+ 37us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 04:43:39.976847 (+ 354us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 04:43:39.976848 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 04:43:39.976987 (+ 139us) client_negotiation.cc:189] Negotiated authn=SASL
0810 04:43:39.977027 (+ 40us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:43:39.977028 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:43:39.977909 (+ 881us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:43:39.977910 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:43:39.978667 (+ 757us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:43:39.978670 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:43:39.979207 (+ 537us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:43:39.979208 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:43:39.979262 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:43:39.979668 (+ 406us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 04:43:39.979672 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0810 04:43:39.980905 (+ 1233us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 04:43:39.982103 (+ 1198us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 04:43:39.982104 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 04:43:39.982104 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 04:43:39.982293 (+ 189us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 04:43:39.982432 (+ 139us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 04:43:39.982433 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 04:43:39.982433 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 04:43:39.982474 (+ 41us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 04:43:39.982622 (+ 148us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 04:43:39.982623 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 04:43:39.982652 (+ 29us) client_negotiation.cc:769] Sending connection context
0810 04:43:39.982696 (+ 44us) client_negotiation.cc:240] Negotiation successful
0810 04:43:39.982753 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":21}
I20260810 04:43:39.982829 13682 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:43:39.976543 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40672 (local address 127.22.167.124:34159)
0810 04:43:39.976704 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0810 04:43:39.976706 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 04:43:39.976713 (+ 7us) server_negotiation.cc:407] Connection header received
0810 04:43:39.976730 (+ 17us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:43:39.976731 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:43:39.976739 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:43:39.976836 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0810 04:43:39.977097 (+ 261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:43:39.977848 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:43:39.978797 (+ 949us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:43:39.979122 (+ 325us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:43:39.979392 (+ 270us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:43:39.980979 (+ 1587us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 04:43:39.980980 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 04:43:39.980980 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 04:43:39.980987 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 04:43:39.982034 (+ 1047us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 04:43:39.982347 (+ 313us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 04:43:39.982348 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 04:43:39.982348 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 04:43:39.982393 (+ 45us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 04:43:39.982505 (+ 112us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 04:43:39.982505 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 04:43:39.982505 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 04:43:39.982583 (+ 78us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 04:43:39.982624 (+ 41us) server_negotiation.cc:1035] Waiting for connection context
0810 04:43:39.982711 (+ 87us) server_negotiation.cc:299] Negotiation successful
0810 04:43:39.982764 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":142,"thread_start_us":96,"threads_started":1}
I20260810 04:43:39.983974 13407 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ced6bb2b892b4c0a9db68c3798caf0a4: Bootstrap starting.
I20260810 04:43:39.984544 13407 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ced6bb2b892b4c0a9db68c3798caf0a4: Neither blocks nor log segments found. Creating new log.
I20260810 04:43:39.984797 13407 log.cc:824] T 00000000000000000000000000000000 P ced6bb2b892b4c0a9db68c3798caf0a4: Log is configured to *not* fsync() on all Append() calls
I20260810 04:43:39.985574 13407 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ced6bb2b892b4c0a9db68c3798caf0a4: No bootstrap required, opened a new log
I20260810 04:43:39.987053 13407 raft_consensus.cc:348] T 00000000000000000000000000000000 P ced6bb2b892b4c0a9db68c3798caf0a4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ced6bb2b892b4c0a9db68c3798caf0a4" member_type: VOTER last_known_addr { host: "127.22.167.126" port: 41211 } } peers { permanent_uuid: "80b7340e2c7645bdb5f15237334a54e0" member_type: VOTER last_known_addr { host: "127.22.167.125" port: 39737 } } peers { permanent_uuid: "33d9c156ea8449ec98a075ad65de8839" member_type: VOTER last_known_addr { host: "127.22.167.124" port: 34159 } }
I20260810 04:43:39.987162 13407 raft_consensus.cc:374] T 00000000000000000000000000000000 P ced6bb2b892b4c0a9db68c3798caf0a4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 04:43:39.987186 13407 raft_consensus.cc:729] T 00000000000000000000000000000000 P ced6bb2b892b4c0a9db68c3798caf0a4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ced6bb2b892b4c0a9db68c3798caf0a4, State: Initialized, Role: FOLLOWER
I20260810 04:43:39.987273 13407 consensus_queue.cc:260] T 00000000000000000000000000000000 P ced6bb2b892b4c0a9db68c3798caf0a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ced6bb2b892b4c0a9db68c3798caf0a4" member_type: VOTER last_known_addr { host: "127.22.167.126" port: 41211 } } peers { permanent_uuid: "80b7340e2c7645bdb5f15237334a54e0" member_type: VOTER last_known_addr { host: "127.22.167.125" port: 39737 } } peers { permanent_uuid: "33d9c156ea8449ec98a075ad65de8839" member_type: VOTER last_known_addr { host: "127.22.167.124" port: 34159 } }
I20260810 04:43:39.987499 13407 sys_catalog.cc:564] T 00000000000000000000000000000000 P ced6bb2b892b4c0a9db68c3798caf0a4 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 04:43:39.987478 13687 sys_catalog.cc:455] T 00000000000000000000000000000000 P ced6bb2b892b4c0a9db68c3798caf0a4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ced6bb2b892b4c0a9db68c3798caf0a4" member_type: VOTER last_known_addr { host: "127.22.167.126" port: 41211 } } peers { permanent_uuid: "80b7340e2c7645bdb5f15237334a54e0" member_type: VOTER last_known_addr { host: "127.22.167.125" port: 39737 } } peers { permanent_uuid: "33d9c156ea8449ec98a075ad65de8839" member_type: VOTER last_known_addr { host: "127.22.167.124" port: 34159 } } }
I20260810 04:43:39.987557 13687 sys_catalog.cc:458] T 00000000000000000000000000000000 P ced6bb2b892b4c0a9db68c3798caf0a4 [sys.catalog]: This master's current role is: FOLLOWER
W20260810 04:43:39.989216 13699 catalog_manager.cc:1558] T 00000000000000000000000000000000 P ced6bb2b892b4c0a9db68c3798caf0a4: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260810 04:43:39.989274 13699 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260810 04:43:39.989938 13290 master_runner.cc:186] Error getting master registration for 127.22.167.125:39737: OK, instance_id { permanent_uuid: "80b7340e2c7645bdb5f15237334a54e0" instance_seqno: 1786337019778285 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260810 04:43:39.992570 13290 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260810 04:43:40.011202 13527 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.22.167.124" port: 34159 } attempt: 3
Aug 10 04:43:40 dist-test-slave-bqcl krb5kdc[13266](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786337019, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.167.125@KRBTEST.COM for kudu/127.22.167.124@KRBTEST.COM
I20260810 04:43:40.017575 13645 raft_consensus.cc:482] T 00000000000000000000000000000000 P 33d9c156ea8449ec98a075ad65de8839 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 04:43:40.017707 13645 raft_consensus.cc:504] T 00000000000000000000000000000000 P 33d9c156ea8449ec98a075ad65de8839 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ced6bb2b892b4c0a9db68c3798caf0a4" member_type: VOTER last_known_addr { host: "127.22.167.126" port: 41211 } } peers { permanent_uuid: "80b7340e2c7645bdb5f15237334a54e0" member_type: VOTER last_known_addr { host: "127.22.167.125" port: 39737 } } peers { permanent_uuid: "33d9c156ea8449ec98a075ad65de8839" member_type: VOTER last_known_addr { host: "127.22.167.124" port: 34159 } }
I20260810 04:43:40.017917 13530 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:43:40.011544 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.124:34159 (local address 127.0.0.1:40676)
0810 04:43:40.011615 (+ 71us) negotiation.cc:107] Waiting for socket to connect
0810 04:43:40.011629 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0810 04:43:40.011706 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 04:43:40.011876 (+ 170us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 04:43:40.011878 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 04:43:40.012097 (+ 219us) client_negotiation.cc:189] Negotiated authn=SASL
0810 04:43:40.012168 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:43:40.012170 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:43:40.012867 (+ 697us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:43:40.012868 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:43:40.013363 (+ 495us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:43:40.013365 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:43:40.013696 (+ 331us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:43:40.013697 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:43:40.013750 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:43:40.014177 (+ 427us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 04:43:40.014184 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0810 04:43:40.015355 (+ 1171us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 04:43:40.016913 (+ 1558us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 04:43:40.016914 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 04:43:40.016915 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 04:43:40.017129 (+ 214us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 04:43:40.017307 (+ 178us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 04:43:40.017308 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 04:43:40.017308 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 04:43:40.017361 (+ 53us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 04:43:40.017610 (+ 249us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 04:43:40.017611 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 04:43:40.017658 (+ 47us) client_negotiation.cc:769] Sending connection context
0810 04:43:40.017720 (+ 62us) client_negotiation.cc:240] Negotiation successful
0810 04:43:40.017817 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":43}
I20260810 04:43:40.017975 13645 leader_election.cc:288] T 00000000000000000000000000000000 P 33d9c156ea8449ec98a075ad65de8839 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ced6bb2b892b4c0a9db68c3798caf0a4 (127.22.167.126:41211), 80b7340e2c7645bdb5f15237334a54e0 (127.22.167.125:39737)
I20260810 04:43:40.018330 13365 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "33d9c156ea8449ec98a075ad65de8839" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ced6bb2b892b4c0a9db68c3798caf0a4" is_pre_election: true
I20260810 04:43:40.018476 13365 raft_consensus.cc:2454] T 00000000000000000000000000000000 P ced6bb2b892b4c0a9db68c3798caf0a4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33d9c156ea8449ec98a075ad65de8839 in term 0.
I20260810 04:43:40.018589 13682 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:43:40.011594 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40676 (local address 127.22.167.124:34159)
0810 04:43:40.011648 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0810 04:43:40.011650 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 04:43:40.011708 (+ 58us) server_negotiation.cc:407] Connection header received
0810 04:43:40.011778 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:43:40.011779 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:43:40.011789 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:43:40.012252 (+ 463us) server_negotiation.cc:226] Negotiated authn=SASL
0810 04:43:40.012328 (+ 76us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:43:40.012794 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:43:40.013432 (+ 638us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:43:40.013638 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:43:40.013692 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:43:40.015432 (+ 1740us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 04:43:40.015433 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 04:43:40.015434 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 04:43:40.015442 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0810 04:43:40.016827 (+ 1385us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 04:43:40.017194 (+ 367us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 04:43:40.017195 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 04:43:40.017196 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 04:43:40.017255 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 04:43:40.017410 (+ 155us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 04:43:40.017411 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 04:43:40.017411 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 04:43:40.017546 (+ 135us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 04:43:40.017615 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0810 04:43:40.018406 (+ 791us) server_negotiation.cc:299] Negotiation successful
0810 04:43:40.018498 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260810 04:43:40.018707 13558 leader_election.cc:302] T 00000000000000000000000000000000 P 33d9c156ea8449ec98a075ad65de8839 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d9c156ea8449ec98a075ad65de8839, ced6bb2b892b4c0a9db68c3798caf0a4; no voters:
I20260810 04:43:40.018925 13645 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 33d9c156ea8449ec98a075ad65de8839 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 04:43:40.018980 13645 raft_consensus.cc:482] T 00000000000000000000000000000000 P 33d9c156ea8449ec98a075ad65de8839 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 04:43:40.019002 13645 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 33d9c156ea8449ec98a075ad65de8839 [term 0 FOLLOWER]: Advancing to term 1
I20260810 04:43:40.019872 13645 raft_consensus.cc:504] T 00000000000000000000000000000000 P 33d9c156ea8449ec98a075ad65de8839 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ced6bb2b892b4c0a9db68c3798caf0a4" member_type: VOTER last_known_addr { host: "127.22.167.126" port: 41211 } } peers { permanent_uuid: "80b7340e2c7645bdb5f15237334a54e0" member_type: VOTER last_known_addr { host: "127.22.167.125" port: 39737 } } peers { permanent_uuid: "33d9c156ea8449ec98a075ad65de8839" member_type: VOTER last_known_addr { host: "127.22.167.124" port: 34159 } }
I20260810 04:43:40.020001 13645 leader_election.cc:288] T 00000000000000000000000000000000 P 33d9c156ea8449ec98a075ad65de8839 [CANDIDATE]: Term 1 election: Requested vote from peers ced6bb2b892b4c0a9db68c3798caf0a4 (127.22.167.126:41211), 80b7340e2c7645bdb5f15237334a54e0 (127.22.167.125:39737)
I20260810 04:43:40.020219 13365 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "33d9c156ea8449ec98a075ad65de8839" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ced6bb2b892b4c0a9db68c3798caf0a4"
I20260810 04:43:40.020292 13365 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ced6bb2b892b4c0a9db68c3798caf0a4 [term 0 FOLLOWER]: Advancing to term 1
I20260810 04:43:40.020310 13527 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 80b7340e2c7645bdb5f15237334a54e0: Bootstrap starting.
I20260810 04:43:40.021126 13527 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 80b7340e2c7645bdb5f15237334a54e0: Neither blocks nor log segments found. Creating new log.
I20260810 04:43:40.021145 13365 raft_consensus.cc:2454] T 00000000000000000000000000000000 P ced6bb2b892b4c0a9db68c3798caf0a4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 33d9c156ea8449ec98a075ad65de8839 in term 1.
I20260810 04:43:40.021318 13558 leader_election.cc:302] T 00000000000000000000000000000000 P 33d9c156ea8449ec98a075ad65de8839 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33d9c156ea8449ec98a075ad65de8839, ced6bb2b892b4c0a9db68c3798caf0a4; no voters:
I20260810 04:43:40.021417 13645 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 33d9c156ea8449ec98a075ad65de8839 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 04:43:40.021485 13527 log.cc:824] T 00000000000000000000000000000000 P 80b7340e2c7645bdb5f15237334a54e0: Log is configured to *not* fsync() on all Append() calls
I20260810 04:43:40.021569 13645 raft_consensus.cc:686] T 00000000000000000000000000000000 P 33d9c156ea8449ec98a075ad65de8839 [term 1 LEADER]: Becoming Leader. State: Replica: 33d9c156ea8449ec98a075ad65de8839, State: Running, Role: LEADER
I20260810 04:43:40.021651 13645 consensus_queue.cc:237] T 00000000000000000000000000000000 P 33d9c156ea8449ec98a075ad65de8839 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ced6bb2b892b4c0a9db68c3798caf0a4" member_type: VOTER last_known_addr { host: "127.22.167.126" port: 41211 } } peers { permanent_uuid: "80b7340e2c7645bdb5f15237334a54e0" member_type: VOTER last_known_addr { host: "127.22.167.125" port: 39737 } } peers { permanent_uuid: "33d9c156ea8449ec98a075ad65de8839" member_type: VOTER last_known_addr { host: "127.22.167.124" port: 34159 } }
I20260810 04:43:40.022037 13702 sys_catalog.cc:455] T 00000000000000000000000000000000 P 33d9c156ea8449ec98a075ad65de8839 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 33d9c156ea8449ec98a075ad65de8839. Latest consensus state: current_term: 1 leader_uuid: "33d9c156ea8449ec98a075ad65de8839" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ced6bb2b892b4c0a9db68c3798caf0a4" member_type: VOTER last_known_addr { host: "127.22.167.126" port: 41211 } } peers { permanent_uuid: "80b7340e2c7645bdb5f15237334a54e0" member_type: VOTER last_known_addr { host: "127.22.167.125" port: 39737 } } peers { permanent_uuid: "33d9c156ea8449ec98a075ad65de8839" member_type: VOTER last_known_addr { host: "127.22.167.124" port: 34159 } } }
I20260810 04:43:40.022271 13702 sys_catalog.cc:458] T 00000000000000000000000000000000 P 33d9c156ea8449ec98a075ad65de8839 [sys.catalog]: This master's current role is: LEADER
I20260810 04:43:40.022349 13527 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 80b7340e2c7645bdb5f15237334a54e0: No bootstrap required, opened a new log
I20260810 04:43:40.022755 13707 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 04:43:40.023258 13707 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 04:43:40.024338 13527 raft_consensus.cc:348] T 00000000000000000000000000000000 P 80b7340e2c7645bdb5f15237334a54e0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ced6bb2b892b4c0a9db68c3798caf0a4" member_type: VOTER last_known_addr { host: "127.22.167.126" port: 41211 } } peers { permanent_uuid: "80b7340e2c7645bdb5f15237334a54e0" member_type: VOTER last_known_addr { host: "127.22.167.125" port: 39737 } } peers { permanent_uuid: "33d9c156ea8449ec98a075ad65de8839" member_type: VOTER last_known_addr { host: "127.22.167.124" port: 34159 } }
I20260810 04:43:40.024480 13527 raft_consensus.cc:374] T 00000000000000000000000000000000 P 80b7340e2c7645bdb5f15237334a54e0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 04:43:40.024533 13527 raft_consensus.cc:729] T 00000000000000000000000000000000 P 80b7340e2c7645bdb5f15237334a54e0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 80b7340e2c7645bdb5f15237334a54e0, State: Initialized, Role: FOLLOWER
I20260810 04:43:40.024657 13527 consensus_queue.cc:260] T 00000000000000000000000000000000 P 80b7340e2c7645bdb5f15237334a54e0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ced6bb2b892b4c0a9db68c3798caf0a4" member_type: VOTER last_known_addr { host: "127.22.167.126" port: 41211 } } peers { permanent_uuid: "80b7340e2c7645bdb5f15237334a54e0" member_type: VOTER last_known_addr { host: "127.22.167.125" port: 39737 } } peers { permanent_uuid: "33d9c156ea8449ec98a075ad65de8839" member_type: VOTER last_known_addr { host: "127.22.167.124" port: 34159 } }
I20260810 04:43:40.024860 13365 raft_consensus.cc:1261] T 00000000000000000000000000000000 P ced6bb2b892b4c0a9db68c3798caf0a4 [term 1 FOLLOWER]: Refusing update from remote peer 33d9c156ea8449ec98a075ad65de8839: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 04:43:40.024964 13527 sys_catalog.cc:564] T 00000000000000000000000000000000 P 80b7340e2c7645bdb5f15237334a54e0 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 04:43:40.025079 13490 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "33d9c156ea8449ec98a075ad65de8839" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80b7340e2c7645bdb5f15237334a54e0"
I20260810 04:43:40.025153 13702 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 33d9c156ea8449ec98a075ad65de8839 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ced6bb2b892b4c0a9db68c3798caf0a4" member_type: VOTER last_known_addr { host: "127.22.167.126" port: 41211 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 04:43:40.025189 13490 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 80b7340e2c7645bdb5f15237334a54e0 [term 0 FOLLOWER]: Advancing to term 1
I20260810 04:43:40.026021 13490 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 80b7340e2c7645bdb5f15237334a54e0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 33d9c156ea8449ec98a075ad65de8839 in term 1.
I20260810 04:43:40.026240 13488 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 80b7340e2c7645bdb5f15237334a54e0 [term 1 FOLLOWER]: Refusing update from remote peer 33d9c156ea8449ec98a075ad65de8839: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260810 04:43:40.026451 13455 tablet.cc:2369] T 00000000000000000000000000000000 P 80b7340e2c7645bdb5f15237334a54e0: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 04:43:40.026432 13687 sys_catalog.cc:455] T 00000000000000000000000000000000 P ced6bb2b892b4c0a9db68c3798caf0a4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 33d9c156ea8449ec98a075ad65de8839. Latest consensus state: current_term: 1 leader_uuid: "33d9c156ea8449ec98a075ad65de8839" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ced6bb2b892b4c0a9db68c3798caf0a4" member_type: VOTER last_known_addr { host: "127.22.167.126" port: 41211 } } peers { permanent_uuid: "80b7340e2c7645bdb5f15237334a54e0" member_type: VOTER last_known_addr { host: "127.22.167.125" port: 39737 } } peers { permanent_uuid: "33d9c156ea8449ec98a075ad65de8839" member_type: VOTER last_known_addr { host: "127.22.167.124" port: 34159 } } }
I20260810 04:43:40.026511 13687 sys_catalog.cc:458] T 00000000000000000000000000000000 P ced6bb2b892b4c0a9db68c3798caf0a4 [sys.catalog]: This master's current role is: FOLLOWER
I20260810 04:43:40.026679 13491 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "33d9c156ea8449ec98a075ad65de8839" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80b7340e2c7645bdb5f15237334a54e0" is_pre_election: true
I20260810 04:43:40.026764 13491 raft_consensus.cc:2362] T 00000000000000000000000000000000 P 80b7340e2c7645bdb5f15237334a54e0 [term 1 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 33d9c156ea8449ec98a075ad65de8839 in term 1. Re-sending same reply.
I20260810 04:43:40.027020 13702 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 33d9c156ea8449ec98a075ad65de8839 [LEADER]: Connected to new peer: Peer: permanent_uuid: "80b7340e2c7645bdb5f15237334a54e0" member_type: VOTER last_known_addr { host: "127.22.167.125" port: 39737 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 04:43:40.026975 13710 sys_catalog.cc:455] T 00000000000000000000000000000000 P 80b7340e2c7645bdb5f15237334a54e0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ced6bb2b892b4c0a9db68c3798caf0a4" member_type: VOTER last_known_addr { host: "127.22.167.126" port: 41211 } } peers { permanent_uuid: "80b7340e2c7645bdb5f15237334a54e0" member_type: VOTER last_known_addr { host: "127.22.167.125" port: 39737 } } peers { permanent_uuid: "33d9c156ea8449ec98a075ad65de8839" member_type: VOTER last_known_addr { host: "127.22.167.124" port: 34159 } } }
I20260810 04:43:40.027343 13710 sys_catalog.cc:458] T 00000000000000000000000000000000 P 80b7340e2c7645bdb5f15237334a54e0 [sys.catalog]: This master's current role is: FOLLOWER
I20260810 04:43:40.027909 13645 sys_catalog.cc:455] T 00000000000000000000000000000000 P 33d9c156ea8449ec98a075ad65de8839 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "33d9c156ea8449ec98a075ad65de8839" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ced6bb2b892b4c0a9db68c3798caf0a4" member_type: VOTER last_known_addr { host: "127.22.167.126" port: 41211 } } peers { permanent_uuid: "80b7340e2c7645bdb5f15237334a54e0" member_type: VOTER last_known_addr { host: "127.22.167.125" port: 39737 } } peers { permanent_uuid: "33d9c156ea8449ec98a075ad65de8839" member_type: VOTER last_known_addr { host: "127.22.167.124" port: 34159 } } }
I20260810 04:43:40.027984 13645 sys_catalog.cc:458] T 00000000000000000000000000000000 P 33d9c156ea8449ec98a075ad65de8839 [sys.catalog]: This master's current role is: LEADER
I20260810 04:43:40.028262 13707 catalog_manager.cc:1347] Generated new cluster ID: 866b3d4aaf9b445b8d7c798ce99f377c
I20260810 04:43:40.028306 13707 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 04:43:40.028308 13710 sys_catalog.cc:455] T 00000000000000000000000000000000 P 80b7340e2c7645bdb5f15237334a54e0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 33d9c156ea8449ec98a075ad65de8839. Latest consensus state: current_term: 1 leader_uuid: "33d9c156ea8449ec98a075ad65de8839" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ced6bb2b892b4c0a9db68c3798caf0a4" member_type: VOTER last_known_addr { host: "127.22.167.126" port: 41211 } } peers { permanent_uuid: "80b7340e2c7645bdb5f15237334a54e0" member_type: VOTER last_known_addr { host: "127.22.167.125" port: 39737 } } peers { permanent_uuid: "33d9c156ea8449ec98a075ad65de8839" member_type: VOTER last_known_addr { host: "127.22.167.124" port: 34159 } } }
I20260810 04:43:40.028395 13710 sys_catalog.cc:458] T 00000000000000000000000000000000 P 80b7340e2c7645bdb5f15237334a54e0 [sys.catalog]: This master's current role is: FOLLOWER
I20260810 04:43:40.028713 13687 sys_catalog.cc:455] T 00000000000000000000000000000000 P ced6bb2b892b4c0a9db68c3798caf0a4 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "33d9c156ea8449ec98a075ad65de8839" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ced6bb2b892b4c0a9db68c3798caf0a4" member_type: VOTER last_known_addr { host: "127.22.167.126" port: 41211 } } peers { permanent_uuid: "80b7340e2c7645bdb5f15237334a54e0" member_type: VOTER last_known_addr { host: "127.22.167.125" port: 39737 } } peers { permanent_uuid: "33d9c156ea8449ec98a075ad65de8839" member_type: VOTER last_known_addr { host: "127.22.167.124" port: 34159 } } }
I20260810 04:43:40.028771 13687 sys_catalog.cc:458] T 00000000000000000000000000000000 P ced6bb2b892b4c0a9db68c3798caf0a4 [sys.catalog]: This master's current role is: FOLLOWER
I20260810 04:43:40.030578 13728 mvcc.cc:205] Tried to move back new op lower bound from 7316836434018938880 to 7316836434009288704. Current Snapshot: MvccSnapshot[applied={T|T < 7316836434018938880}]
I20260810 04:43:40.030767 13709 mvcc.cc:205] Tried to move back new op lower bound from 7316836434018938880 to 7316836434009288704. Current Snapshot: MvccSnapshot[applied={T|T < 7316836434018938880 or (T in {7316836434018938880})}]
I20260810 04:43:40.035491 13707 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 04:43:40.035970 13707 catalog_manager.cc:1504] Loading token signing keys...
W20260810 04:43:40.038669 13735 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 80b7340e2c7645bdb5f15237334a54e0: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260810 04:43:40.038726 13735 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260810 04:43:40.038925 13702 sys_catalog.cc:455] T 00000000000000000000000000000000 P 33d9c156ea8449ec98a075ad65de8839 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "33d9c156ea8449ec98a075ad65de8839" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ced6bb2b892b4c0a9db68c3798caf0a4" member_type: VOTER last_known_addr { host: "127.22.167.126" port: 41211 } } peers { permanent_uuid: "80b7340e2c7645bdb5f15237334a54e0" member_type: VOTER last_known_addr { host: "127.22.167.125" port: 39737 } } peers { permanent_uuid: "33d9c156ea8449ec98a075ad65de8839" member_type: VOTER last_known_addr { host: "127.22.167.124" port: 34159 } } }
I20260810 04:43:40.039008 13702 sys_catalog.cc:458] T 00000000000000000000000000000000 P 33d9c156ea8449ec98a075ad65de8839 [sys.catalog]: This master's current role is: LEADER
I20260810 04:43:40.039480 13710 sys_catalog.cc:455] T 00000000000000000000000000000000 P 80b7340e2c7645bdb5f15237334a54e0 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "33d9c156ea8449ec98a075ad65de8839" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ced6bb2b892b4c0a9db68c3798caf0a4" member_type: VOTER last_known_addr { host: "127.22.167.126" port: 41211 } } peers { permanent_uuid: "80b7340e2c7645bdb5f15237334a54e0" member_type: VOTER last_known_addr { host: "127.22.167.125" port: 39737 } } peers { permanent_uuid: "33d9c156ea8449ec98a075ad65de8839" member_type: VOTER last_known_addr { host: "127.22.167.124" port: 34159 } } }
I20260810 04:43:40.039554 13710 sys_catalog.cc:458] T 00000000000000000000000000000000 P 80b7340e2c7645bdb5f15237334a54e0 [sys.catalog]: This master's current role is: FOLLOWER
I20260810 04:43:40.039902 13707 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 33d9c156ea8449ec98a075ad65de8839: Generated new TSK 0
I20260810 04:43:40.040120 13707 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 04:43:40.040135 13737 mvcc.cc:205] Tried to move back new op lower bound from 7316836434018938880 to 7316836434009288704. Current Snapshot: MvccSnapshot[applied={T|T < 7316836434018938880}]
I20260810 04:43:40.989713 13699 catalog_manager.cc:1259] Loaded cluster ID: 866b3d4aaf9b445b8d7c798ce99f377c
I20260810 04:43:40.989794 13699 catalog_manager.cc:1552] T 00000000000000000000000000000000 P ced6bb2b892b4c0a9db68c3798caf0a4: loading cluster ID for follower catalog manager: success
I20260810 04:43:40.990643 13699 catalog_manager.cc:1574] T 00000000000000000000000000000000 P ced6bb2b892b4c0a9db68c3798caf0a4: acquiring CA information for follower catalog manager: success
I20260810 04:43:40.990952 13699 catalog_manager.cc:1602] T 00000000000000000000000000000000 P ced6bb2b892b4c0a9db68c3798caf0a4: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260810 04:43:41.039177 13735 catalog_manager.cc:1259] Loaded cluster ID: 866b3d4aaf9b445b8d7c798ce99f377c
I20260810 04:43:41.039255 13735 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 80b7340e2c7645bdb5f15237334a54e0: loading cluster ID for follower catalog manager: success
I20260810 04:43:41.040153 13735 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 80b7340e2c7645bdb5f15237334a54e0: acquiring CA information for follower catalog manager: success
I20260810 04:43:41.040467 13735 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 80b7340e2c7645bdb5f15237334a54e0: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260810 04:43:41.094583 23197 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskemv_AM/build/release/bin/kudu with pid 13290
I20260810 04:43:41.114041 23197 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskemv_AM/build/release/bin/kudu with pid 13419
I20260810 04:43:41.127940 23197 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskemv_AM/build/release/bin/kudu with pid 13536
2026-08-10T04:43:41Z chronyd exiting
[ OK ] SecurityITest.IPKICACert (2893 ms)
[ RUN ] SecurityITest.IPKICACertWebServerEndPoints
Loading random data
Initializing database '/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786336863388938-23197-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 04:43:41 dist-test-slave-bqcl krb5kdc[13756](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 04:43:41 dist-test-slave-bqcl krb5kdc[13756](info): set up 2 sockets
Aug 10 04:43:41 dist-test-slave-bqcl krb5kdc[13756](info): commencing operation
krb5kdc: starting...
W20260810 04:43:42.291285 23197 mini_kdc.cc:121] Time spent starting KDC: real 1.137s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786336863388938-23197-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786336863388938-23197-0/krb5kdc/test-user.keytab.
Aug 10 04:43:42 dist-test-slave-bqcl krb5kdc[13756](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786337022, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T04:43:42Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T04:43:42Z Disabled control of system clock
WARNING: no policy specified for kudu/127.22.167.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.167.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.167.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786336863388938-23197-0/krb5kdc/kudu_127.22.167.126.keytab.
Entry for principal kudu/127.22.167.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786336863388938-23197-0/krb5kdc/kudu_127.22.167.126.keytab.
I20260810 04:43:42.402247 23197 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskemv_AM/build/release/bin/kudu
/tmp/dist-test-taskemv_AM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786336863388938-23197-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786336863388938-23197-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786336863388938-23197-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786336863388938-23197-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.167.126:39893
--webserver_interface=127.22.167.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786336863388938-23197-0/krb5kdc/kudu_127.22.167.126.keytab
--principal=kudu/127.22.167.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.167.84:44743
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.167.126:39893
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786336863388938-23197-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786336863388938-23197-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786336863388938-23197-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 04:43:42.480942 13775 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 04:43:42.481148 13775 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 04:43:42.481187 13775 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 04:43:42.482327 13775 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 04:43:42.482367 13775 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 04:43:42.482385 13775 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 04:43:42.482401 13775 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 04:43:42.482416 13775 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 04:43:42.483836 13775 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.167.84:44743
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786336863388938-23197-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786336863388938-23197-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.22.167.126:39893
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786336863388938-23197-0/krb5kdc/kudu_127.22.167.126.keytab
--principal=kudu/127.22.167.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.167.126:39893
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786336863388938-23197-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.167.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13775
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786336863388938-23197-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 03:54:01 UTC on ccycloud.quasar-icijmt.root.comops.site
build id 4577009
I20260810 04:43:42.484077 13775 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 04:43:42.484292 13775 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 04:43:42.488884 13786 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 04:43:42.488884 13789 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 04:43:42.488967 13787 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 04:43:42.489178 13775 server_base.cc:1034] running on GCE node
I20260810 04:43:42.489428 13775 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 04:43:42.489697 13775 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 04:43:42.490832 13775 hybrid_clock.cc:648] HybridClock initialized: now 1786337022490816 us; error 28 us; skew 500 ppm
Aug 10 04:43:42 dist-test-slave-bqcl krb5kdc[13756](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786337022, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.167.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 04:43:42.493152 13775 init.cc:377] Logged in from keytab as kudu/127.22.167.126@KRBTEST.COM (short username kudu)
I20260810 04:43:42.493568 13775 webserver.cc:460] Webserver started at http://127.22.167.126:41071/ using document root <none> and password file <none>
I20260810 04:43:42.493793 13775 fs_manager.cc:359] Metadata directory not provided
I20260810 04:43:42.493842 13775 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 04:43:42.493968 13775 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 04:43:42.495004 13775 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786336863388938-23197-0/minicluster-data/master-0/data/instance:
uuid: "3b85045a26c3449f94499a7fd31e7b9a"
format_stamp: "Formatted at 2026-08-10 04:43:42 on dist-test-slave-bqcl"
server_key: "c96990cd66bf8dda2c084b778fc1b01b"
server_key_iv: "efa46bb933339e4e196864b4ae67c438"
server_key_version: "encryptionkey@0"
I20260810 04:43:42.495339 13775 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786336863388938-23197-0/minicluster-data/master-0/wal/instance:
uuid: "3b85045a26c3449f94499a7fd31e7b9a"
format_stamp: "Formatted at 2026-08-10 04:43:42 on dist-test-slave-bqcl"
server_key: "c96990cd66bf8dda2c084b778fc1b01b"
server_key_iv: "efa46bb933339e4e196864b4ae67c438"
server_key_version: "encryptionkey@0"
I20260810 04:43:42.496538 13775 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260810 04:43:42.497179 13802 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 04:43:42.497349 13775 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260810 04:43:42.497430 13775 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786336863388938-23197-0/minicluster-data/master-0/data,/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786336863388938-23197-0/minicluster-data/master-0/wal
uuid: "3b85045a26c3449f94499a7fd31e7b9a"
format_stamp: "Formatted at 2026-08-10 04:43:42 on dist-test-slave-bqcl"
server_key: "c96990cd66bf8dda2c084b778fc1b01b"
server_key_iv: "efa46bb933339e4e196864b4ae67c438"
server_key_version: "encryptionkey@0"
I20260810 04:43:42.497495 13775 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786336863388938-23197-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786336863388938-23197-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786336863388938-23197-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 04:43:42.506698 13775 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 04:43:42.508286 13775 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 04:43:42.508410 13775 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 04:43:42.512162 13775 rpc_server.cc:307] RPC server started. Bound to: 127.22.167.126:39893
I20260810 04:43:42.512197 13890 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.167.126:39893 every 8 connection(s)
I20260810 04:43:42.512501 13775 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786336863388938-23197-0/minicluster-data/master-0/data/info.pb
I20260810 04:43:42.513345 13891 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 04:43:42.515589 13891 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3b85045a26c3449f94499a7fd31e7b9a: Bootstrap starting.
I20260810 04:43:42.516147 13891 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3b85045a26c3449f94499a7fd31e7b9a: Neither blocks nor log segments found. Creating new log.
I20260810 04:43:42.516180 23197 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskemv_AM/build/release/bin/kudu as pid 13775
I20260810 04:43:42.516268 23197 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786336863388938-23197-0/minicluster-data/master-0/wal/instance
I20260810 04:43:42.516429 23197 external_mini_cluster.cc:1428] Setting key e343bae74c95a7f00622615da5eb9a31
I20260810 04:43:42.516441 13891 log.cc:824] T 00000000000000000000000000000000 P 3b85045a26c3449f94499a7fd31e7b9a: Log is configured to *not* fsync() on all Append() calls
I20260810 04:43:42.517174 13891 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3b85045a26c3449f94499a7fd31e7b9a: No bootstrap required, opened a new log
I20260810 04:43:42.519711 13891 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3b85045a26c3449f94499a7fd31e7b9a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3b85045a26c3449f94499a7fd31e7b9a" member_type: VOTER last_known_addr { host: "127.22.167.126" port: 39893 } }
I20260810 04:43:42.519877 13891 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3b85045a26c3449f94499a7fd31e7b9a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 04:43:42.519917 13891 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3b85045a26c3449f94499a7fd31e7b9a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3b85045a26c3449f94499a7fd31e7b9a, State: Initialized, Role: FOLLOWER
I20260810 04:43:42.520036 13891 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3b85045a26c3449f94499a7fd31e7b9a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3b85045a26c3449f94499a7fd31e7b9a" member_type: VOTER last_known_addr { host: "127.22.167.126" port: 39893 } }
I20260810 04:43:42.520099 13891 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3b85045a26c3449f94499a7fd31e7b9a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 04:43:42.520120 13891 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3b85045a26c3449f94499a7fd31e7b9a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 04:43:42.520164 13891 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3b85045a26c3449f94499a7fd31e7b9a [term 0 FOLLOWER]: Advancing to term 1
I20260810 04:43:42.520699 13891 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3b85045a26c3449f94499a7fd31e7b9a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3b85045a26c3449f94499a7fd31e7b9a" member_type: VOTER last_known_addr { host: "127.22.167.126" port: 39893 } }
I20260810 04:43:42.520802 13891 leader_election.cc:302] T 00000000000000000000000000000000 P 3b85045a26c3449f94499a7fd31e7b9a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3b85045a26c3449f94499a7fd31e7b9a; no voters:
I20260810 04:43:42.520953 13891 leader_election.cc:288] T 00000000000000000000000000000000 P 3b85045a26c3449f94499a7fd31e7b9a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 04:43:42.520990 13903 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3b85045a26c3449f94499a7fd31e7b9a [term 1 FOLLOWER]: Leader election won for term 1
I20260810 04:43:42.521185 13891 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3b85045a26c3449f94499a7fd31e7b9a [sys.catalog]: configured and running, proceeding with master startup.
I20260810 04:43:42.521189 13903 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3b85045a26c3449f94499a7fd31e7b9a [term 1 LEADER]: Becoming Leader. State: Replica: 3b85045a26c3449f94499a7fd31e7b9a, State: Running, Role: LEADER
I20260810 04:43:42.522480 13903 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3b85045a26c3449f94499a7fd31e7b9a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3b85045a26c3449f94499a7fd31e7b9a" member_type: VOTER last_known_addr { host: "127.22.167.126" port: 39893 } }
I20260810 04:43:42.522982 13904 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3b85045a26c3449f94499a7fd31e7b9a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3b85045a26c3449f94499a7fd31e7b9a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3b85045a26c3449f94499a7fd31e7b9a" member_type: VOTER last_known_addr { host: "127.22.167.126" port: 39893 } } }
I20260810 04:43:42.523066 13904 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3b85045a26c3449f94499a7fd31e7b9a [sys.catalog]: This master's current role is: LEADER
Aug 10 04:43:42 dist-test-slave-bqcl krb5kdc[13756](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786337022, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.167.126@KRBTEST.COM
W20260810 04:43:42.523475 13936 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 3b85045a26c3449f94499a7fd31e7b9a: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
I20260810 04:43:42.523394 13905 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3b85045a26c3449f94499a7fd31e7b9a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3b85045a26c3449f94499a7fd31e7b9a. Latest consensus state: current_term: 1 leader_uuid: "3b85045a26c3449f94499a7fd31e7b9a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3b85045a26c3449f94499a7fd31e7b9a" member_type: VOTER last_known_addr { host: "127.22.167.126" port: 39893 } } }
W20260810 04:43:42.523540 13936 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260810 04:43:42.523552 13905 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3b85045a26c3449f94499a7fd31e7b9a [sys.catalog]: This master's current role is: LEADER
I20260810 04:43:42.523645 13940 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 04:43:42.523857 13940 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 04:43:42.525314 13940 catalog_manager.cc:1347] Generated new cluster ID: 6fa3457f183f4a8bab664b6719f2396e
I20260810 04:43:42.525364 13940 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 04:43:42.527552 13899 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:43:42.516877 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40472 (local address 127.22.167.126:39893)
0810 04:43:42.517043 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0810 04:43:42.517052 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 04:43:42.517084 (+ 32us) server_negotiation.cc:407] Connection header received
0810 04:43:42.517581 (+ 497us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:43:42.517590 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:43:42.517654 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:43:42.517812 (+ 158us) server_negotiation.cc:226] Negotiated authn=SASL
0810 04:43:42.518408 (+ 596us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:43:42.519797 (+ 1389us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:43:42.520605 (+ 808us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:43:42.521215 (+ 610us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:43:42.521327 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:43:42.523856 (+ 2529us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 04:43:42.523860 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 04:43:42.523860 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 04:43:42.523872 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0810 04:43:42.525923 (+ 2051us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 04:43:42.526327 (+ 404us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 04:43:42.526330 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 04:43:42.526331 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 04:43:42.526443 (+ 112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 04:43:42.526634 (+ 191us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 04:43:42.526635 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 04:43:42.526636 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 04:43:42.526948 (+ 312us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 04:43:42.527124 (+ 176us) server_negotiation.cc:1035] Waiting for connection context
0810 04:43:42.527206 (+ 82us) server_negotiation.cc:299] Negotiation successful
0810 04:43:42.527344 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":116,"thread_start_us":83,"threads_started":1}
I20260810 04:43:42.528469 23197 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 04:43:42.537401 13940 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 04:43:42.538093 13940 catalog_manager.cc:1504] Loading token signing keys...
I20260810 04:43:42.545466 13940 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3b85045a26c3449f94499a7fd31e7b9a: Generated new TSK 0
I20260810 04:43:42.545642 13940 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 04:43:42.582151 23197 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskemv_AM/build/release/bin/kudu with pid 13775
2026-08-10T04:43:42Z chronyd exiting
[ OK ] SecurityITest.IPKICACertWebServerEndPoints (1453 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedIPKICert
Loading random data
Initializing database '/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786336863388938-23197-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 04:43:42 dist-test-slave-bqcl krb5kdc[13951](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 04:43:42 dist-test-slave-bqcl krb5kdc[13951](info): set up 2 sockets
Aug 10 04:43:42 dist-test-slave-bqcl krb5kdc[13951](info): commencing operation
krb5kdc: starting...
W20260810 04:43:43.730319 23197 mini_kdc.cc:121] Time spent starting KDC: real 1.123s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786336863388938-23197-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786336863388938-23197-0/krb5kdc/test-user.keytab.
Aug 10 04:43:43 dist-test-slave-bqcl krb5kdc[13951](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786337023, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T04:43:43Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T04:43:43Z Disabled control of system clock
WARNING: no policy specified for kudu/127.22.167.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.167.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.167.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786336863388938-23197-0/krb5kdc/kudu_127.22.167.126.keytab.
Entry for principal kudu/127.22.167.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786336863388938-23197-0/krb5kdc/kudu_127.22.167.126.keytab.
I20260810 04:43:43.841146 23197 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskemv_AM/build/release/bin/kudu
/tmp/dist-test-taskemv_AM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786336863388938-23197-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786336863388938-23197-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786336863388938-23197-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786336863388938-23197-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.167.126:46073
--webserver_interface=127.22.167.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786336863388938-23197-0/krb5kdc/kudu_127.22.167.126.keytab
--principal=kudu/127.22.167.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.167.84:34205
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.167.126:46073
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786336863388938-23197-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786336863388938-23197-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786336863388938-23197-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 04:43:43.920732 13977 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 04:43:43.920926 13977 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 04:43:43.920956 13977 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 04:43:43.922070 13977 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 04:43:43.922111 13977 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 04:43:43.922158 13977 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 04:43:43.922178 13977 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 04:43:43.922200 13977 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 04:43:43.923628 13977 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.167.84:34205
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786336863388938-23197-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786336863388938-23197-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.22.167.126:46073
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786336863388938-23197-0/krb5kdc/kudu_127.22.167.126.keytab
--principal=kudu/127.22.167.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.167.126:46073
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786336863388938-23197-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.167.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13977
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786336863388938-23197-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 03:54:01 UTC on ccycloud.quasar-icijmt.root.comops.site
build id 4577009
I20260810 04:43:43.923859 13977 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 04:43:43.924062 13977 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 04:43:43.928556 13986 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 04:43:43.928655 13991 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 04:43:43.928540 13994 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 04:43:43.928901 13977 server_base.cc:1034] running on GCE node
I20260810 04:43:43.929147 13977 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 04:43:43.929419 13977 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 04:43:43.930557 13977 hybrid_clock.cc:648] HybridClock initialized: now 1786337023930540 us; error 30 us; skew 500 ppm
Aug 10 04:43:43 dist-test-slave-bqcl krb5kdc[13951](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786337023, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.167.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 04:43:43.932937 13977 init.cc:377] Logged in from keytab as kudu/127.22.167.126@KRBTEST.COM (short username kudu)
I20260810 04:43:43.933347 13977 webserver.cc:460] Webserver started at http://127.22.167.126:35997/ using document root <none> and password file <none>
I20260810 04:43:43.933542 13977 fs_manager.cc:359] Metadata directory not provided
I20260810 04:43:43.933580 13977 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 04:43:43.933665 13977 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 04:43:43.934679 13977 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786336863388938-23197-0/minicluster-data/master-0/data/instance:
uuid: "1623678ec1f5413687149d71bb0bce92"
format_stamp: "Formatted at 2026-08-10 04:43:43 on dist-test-slave-bqcl"
server_key: "3bba34883e284cb5c91d59bf7a18c494"
server_key_iv: "1ae4bae684dc57bb70292d045cdf211c"
server_key_version: "encryptionkey@0"
I20260810 04:43:43.934978 13977 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786336863388938-23197-0/minicluster-data/master-0/wal/instance:
uuid: "1623678ec1f5413687149d71bb0bce92"
format_stamp: "Formatted at 2026-08-10 04:43:43 on dist-test-slave-bqcl"
server_key: "3bba34883e284cb5c91d59bf7a18c494"
server_key_iv: "1ae4bae684dc57bb70292d045cdf211c"
server_key_version: "encryptionkey@0"
I20260810 04:43:43.936133 13977 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260810 04:43:43.936764 14010 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 04:43:43.936914 13977 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260810 04:43:43.936990 13977 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786336863388938-23197-0/minicluster-data/master-0/data,/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786336863388938-23197-0/minicluster-data/master-0/wal
uuid: "1623678ec1f5413687149d71bb0bce92"
format_stamp: "Formatted at 2026-08-10 04:43:43 on dist-test-slave-bqcl"
server_key: "3bba34883e284cb5c91d59bf7a18c494"
server_key_iv: "1ae4bae684dc57bb70292d045cdf211c"
server_key_version: "encryptionkey@0"
I20260810 04:43:43.937053 13977 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786336863388938-23197-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786336863388938-23197-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786336863388938-23197-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 04:43:43.946580 13977 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 04:43:43.948134 13977 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 04:43:43.948256 13977 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 04:43:43.951926 13977 rpc_server.cc:307] RPC server started. Bound to: 127.22.167.126:46073
I20260810 04:43:43.951978 14088 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.167.126:46073 every 8 connection(s)
I20260810 04:43:43.952263 13977 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786336863388938-23197-0/minicluster-data/master-0/data/info.pb
I20260810 04:43:43.953177 14089 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 04:43:43.955296 23197 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskemv_AM/build/release/bin/kudu as pid 13977
I20260810 04:43:43.955377 23197 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786336863388938-23197-0/minicluster-data/master-0/wal/instance
I20260810 04:43:43.955370 14089 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1623678ec1f5413687149d71bb0bce92: Bootstrap starting.
I20260810 04:43:43.955526 23197 external_mini_cluster.cc:1428] Setting key 11901ea21402669fe33773955032eebe
I20260810 04:43:43.955974 14089 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1623678ec1f5413687149d71bb0bce92: Neither blocks nor log segments found. Creating new log.
I20260810 04:43:43.956339 14089 log.cc:824] T 00000000000000000000000000000000 P 1623678ec1f5413687149d71bb0bce92: Log is configured to *not* fsync() on all Append() calls
I20260810 04:43:43.957037 14089 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1623678ec1f5413687149d71bb0bce92: No bootstrap required, opened a new log
I20260810 04:43:43.959208 14089 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1623678ec1f5413687149d71bb0bce92 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1623678ec1f5413687149d71bb0bce92" member_type: VOTER last_known_addr { host: "127.22.167.126" port: 46073 } }
I20260810 04:43:43.959375 14089 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1623678ec1f5413687149d71bb0bce92 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 04:43:43.959411 14089 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1623678ec1f5413687149d71bb0bce92 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1623678ec1f5413687149d71bb0bce92, State: Initialized, Role: FOLLOWER
I20260810 04:43:43.959549 14089 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1623678ec1f5413687149d71bb0bce92 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1623678ec1f5413687149d71bb0bce92" member_type: VOTER last_known_addr { host: "127.22.167.126" port: 46073 } }
I20260810 04:43:43.959645 14089 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1623678ec1f5413687149d71bb0bce92 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 04:43:43.959679 14089 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1623678ec1f5413687149d71bb0bce92 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 04:43:43.959725 14089 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1623678ec1f5413687149d71bb0bce92 [term 0 FOLLOWER]: Advancing to term 1
I20260810 04:43:43.960443 14089 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1623678ec1f5413687149d71bb0bce92 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1623678ec1f5413687149d71bb0bce92" member_type: VOTER last_known_addr { host: "127.22.167.126" port: 46073 } }
I20260810 04:43:43.960570 14089 leader_election.cc:302] T 00000000000000000000000000000000 P 1623678ec1f5413687149d71bb0bce92 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1623678ec1f5413687149d71bb0bce92; no voters:
I20260810 04:43:43.960744 14089 leader_election.cc:288] T 00000000000000000000000000000000 P 1623678ec1f5413687149d71bb0bce92 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 04:43:43.960769 14097 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1623678ec1f5413687149d71bb0bce92 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 04:43:43.960929 14097 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1623678ec1f5413687149d71bb0bce92 [term 1 LEADER]: Becoming Leader. State: Replica: 1623678ec1f5413687149d71bb0bce92, State: Running, Role: LEADER
I20260810 04:43:43.961061 14089 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1623678ec1f5413687149d71bb0bce92 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 04:43:43.961074 14097 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1623678ec1f5413687149d71bb0bce92 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1623678ec1f5413687149d71bb0bce92" member_type: VOTER last_known_addr { host: "127.22.167.126" port: 46073 } }
I20260810 04:43:43.961472 14098 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1623678ec1f5413687149d71bb0bce92 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1623678ec1f5413687149d71bb0bce92" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1623678ec1f5413687149d71bb0bce92" member_type: VOTER last_known_addr { host: "127.22.167.126" port: 46073 } } }
I20260810 04:43:43.961547 14098 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1623678ec1f5413687149d71bb0bce92 [sys.catalog]: This master's current role is: LEADER
I20260810 04:43:43.961767 14099 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1623678ec1f5413687149d71bb0bce92 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1623678ec1f5413687149d71bb0bce92. Latest consensus state: current_term: 1 leader_uuid: "1623678ec1f5413687149d71bb0bce92" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1623678ec1f5413687149d71bb0bce92" member_type: VOTER last_known_addr { host: "127.22.167.126" port: 46073 } } }
I20260810 04:43:43.962158 14099 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1623678ec1f5413687149d71bb0bce92 [sys.catalog]: This master's current role is: LEADER
W20260810 04:43:43.962924 14122 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 1623678ec1f5413687149d71bb0bce92: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260810 04:43:43.962970 14122 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260810 04:43:43.963012 14103 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
Aug 10 04:43:43 dist-test-slave-bqcl krb5kdc[13951](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786337023, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.167.126@KRBTEST.COM
I20260810 04:43:43.963150 14103 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 04:43:43.964738 14103 catalog_manager.cc:1347] Generated new cluster ID: 83957b148ce04e079ff7496e9b48b4fa
I20260810 04:43:43.964795 14103 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 04:43:43.967188 14094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:43:43.956015 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38522 (local address 127.22.167.126:46073)
0810 04:43:43.956169 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0810 04:43:43.956177 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 04:43:43.956200 (+ 23us) server_negotiation.cc:407] Connection header received
0810 04:43:43.956635 (+ 435us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:43:43.956642 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:43:43.956687 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:43:43.956795 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0810 04:43:43.957541 (+ 746us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:43:43.958417 (+ 876us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:43:43.959486 (+ 1069us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:43:43.960087 (+ 601us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:43:43.960197 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:43:43.963606 (+ 3409us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 04:43:43.963610 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 04:43:43.963610 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 04:43:43.963621 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0810 04:43:43.965569 (+ 1948us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 04:43:43.965977 (+ 408us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 04:43:43.965980 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 04:43:43.965980 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 04:43:43.966088 (+ 108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 04:43:43.966332 (+ 244us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 04:43:43.966333 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 04:43:43.966333 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 04:43:43.966605 (+ 272us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 04:43:43.966747 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0810 04:43:43.966860 (+ 113us) server_negotiation.cc:299] Negotiation successful
0810 04:43:43.966985 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":111,"thread_start_us":71,"threads_started":1}
I20260810 04:43:43.969638 14103 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 04:43:43.970112 14103 catalog_manager.cc:1504] Loading token signing keys...
I20260810 04:43:43.979909 14103 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1623678ec1f5413687149d71bb0bce92: Generated new TSK 0
I20260810 04:43:43.980072 14103 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.22.167.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.167.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.167.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786336863388938-23197-0/krb5kdc/kudu_127.22.167.65.keytab.
Entry for principal kudu/127.22.167.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786336863388938-23197-0/krb5kdc/kudu_127.22.167.65.keytab.
I20260810 04:43:44.008632 23197 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskemv_AM/build/release/bin/kudu
/tmp/dist-test-taskemv_AM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786336863388938-23197-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786336863388938-23197-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786336863388938-23197-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786336863388938-23197-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.167.65:0
--local_ip_for_outbound_sockets=127.22.167.65
--webserver_interface=127.22.167.65
--webserver_port=0
--tserver_master_addrs=127.22.167.126:46073
--keytab_file=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786336863388938-23197-0/krb5kdc/kudu_127.22.167.65.keytab
--principal=kudu/127.22.167.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.167.84:34205
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786336863388938-23197-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786336863388938-23197-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786336863388938-23197-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 04:43:44.089133 14129 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 04:43:44.089334 14129 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 04:43:44.089356 14129 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 04:43:44.090480 14129 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 04:43:44.090518 14129 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 04:43:44.090543 14129 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.167.65
I20260810 04:43:44.091931 14129 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.167.84:34205
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786336863388938-23197-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786336863388938-23197-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786336863388938-23197-0/krb5kdc/kudu_127.22.167.65.keytab
--principal=kudu/127.22.167.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.167.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786336863388938-23197-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.22.167.65
--webserver_port=0
--tserver_master_addrs=127.22.167.126:46073
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14129
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.167.65
--log_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786336863388938-23197-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 03:54:01 UTC on ccycloud.quasar-icijmt.root.comops.site
build id 4577009
I20260810 04:43:44.092142 14129 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 04:43:44.092358 14129 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 04:43:44.097033 14134 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 04:43:44.097146 14135 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 04:43:44.097054 14137 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 04:43:44.097158 14129 server_base.cc:1034] running on GCE node
I20260810 04:43:44.097424 14129 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 04:43:44.097647 14129 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 04:43:44.098785 14129 hybrid_clock.cc:648] HybridClock initialized: now 1786337024098773 us; error 29 us; skew 500 ppm
Aug 10 04:43:44 dist-test-slave-bqcl krb5kdc[13951](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786337024, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.167.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 04:43:44.101136 14129 init.cc:377] Logged in from keytab as kudu/127.22.167.65@KRBTEST.COM (short username kudu)
I20260810 04:43:44.101511 14129 webserver.cc:460] Webserver started at http://127.22.167.65:42653/ using document root <none> and password file <none>
I20260810 04:43:44.101711 14129 fs_manager.cc:359] Metadata directory not provided
I20260810 04:43:44.101751 14129 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 04:43:44.101841 14129 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 04:43:44.102847 14129 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786336863388938-23197-0/minicluster-data/ts-0/data/instance:
uuid: "ed99582b3b574c00991af0deb19f27c1"
format_stamp: "Formatted at 2026-08-10 04:43:44 on dist-test-slave-bqcl"
server_key: "a79c210c6c8adf1172244207a1f9d808"
server_key_iv: "9e7a8eee06fb0a5364374b62d61ff33b"
server_key_version: "encryptionkey@0"
I20260810 04:43:44.103160 14129 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786336863388938-23197-0/minicluster-data/ts-0/wal/instance:
uuid: "ed99582b3b574c00991af0deb19f27c1"
format_stamp: "Formatted at 2026-08-10 04:43:44 on dist-test-slave-bqcl"
server_key: "a79c210c6c8adf1172244207a1f9d808"
server_key_iv: "9e7a8eee06fb0a5364374b62d61ff33b"
server_key_version: "encryptionkey@0"
I20260810 04:43:44.104357 14129 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260810 04:43:44.105077 14148 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 04:43:44.105273 14129 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260810 04:43:44.105358 14129 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786336863388938-23197-0/minicluster-data/ts-0/data,/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786336863388938-23197-0/minicluster-data/ts-0/wal
uuid: "ed99582b3b574c00991af0deb19f27c1"
format_stamp: "Formatted at 2026-08-10 04:43:44 on dist-test-slave-bqcl"
server_key: "a79c210c6c8adf1172244207a1f9d808"
server_key_iv: "9e7a8eee06fb0a5364374b62d61ff33b"
server_key_version: "encryptionkey@0"
I20260810 04:43:44.105412 14129 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786336863388938-23197-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786336863388938-23197-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786336863388938-23197-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 04:43:44.114929 14129 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 04:43:44.116899 14129 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 04:43:44.117007 14129 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 04:43:44.117224 14129 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 04:43:44.117519 14129 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 04:43:44.117555 14129 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 04:43:44.117576 14129 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 04:43:44.117587 14129 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 04:43:44.122918 14129 rpc_server.cc:307] RPC server started. Bound to: 127.22.167.65:44335
I20260810 04:43:44.122987 14339 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.167.65:44335 every 8 connection(s)
I20260810 04:43:44.123268 14129 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786336863388938-23197-0/minicluster-data/ts-0/data/info.pb
Aug 10 04:43:44 dist-test-slave-bqcl krb5kdc[13951](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786337024, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.167.65@KRBTEST.COM for kudu/127.22.167.126@KRBTEST.COM
I20260810 04:43:44.131801 14094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:43:44.123940 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.167.65:33521 (local address 127.22.167.126:46073)
0810 04:43:44.123984 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0810 04:43:44.123986 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 04:43:44.124187 (+ 201us) server_negotiation.cc:407] Connection header received
0810 04:43:44.124682 (+ 495us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:43:44.124683 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:43:44.124695 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:43:44.124757 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0810 04:43:44.125690 (+ 933us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:43:44.126229 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:43:44.127200 (+ 971us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:43:44.127405 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:43:44.127460 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:43:44.129414 (+ 1954us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 04:43:44.129415 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 04:43:44.129415 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 04:43:44.129423 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0810 04:43:44.130534 (+ 1111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 04:43:44.130919 (+ 385us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 04:43:44.130920 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 04:43:44.130921 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 04:43:44.130974 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 04:43:44.131189 (+ 215us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 04:43:44.131189 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 04:43:44.131190 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 04:43:44.131304 (+ 114us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 04:43:44.131359 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0810 04:43:44.131611 (+ 252us) server_negotiation.cc:299] Negotiation successful
0810 04:43:44.131709 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260810 04:43:44.131925 14348 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:43:44.123908 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.126:46073 (local address 127.22.167.65:33521)
0810 04:43:44.124067 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0810 04:43:44.124108 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0810 04:43:44.124582 (+ 474us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 04:43:44.124837 (+ 255us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 04:43:44.124842 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 04:43:44.125412 (+ 570us) client_negotiation.cc:189] Negotiated authn=SASL
0810 04:43:44.125594 (+ 182us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:43:44.125599 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:43:44.126317 (+ 718us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:43:44.126319 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:43:44.127117 (+ 798us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:43:44.127119 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:43:44.127476 (+ 357us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:43:44.127477 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:43:44.127604 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:43:44.127991 (+ 387us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 04:43:44.127999 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0810 04:43:44.129324 (+ 1325us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 04:43:44.130643 (+ 1319us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 04:43:44.130650 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 04:43:44.130652 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 04:43:44.130855 (+ 203us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 04:43:44.131028 (+ 173us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 04:43:44.131029 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 04:43:44.131029 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 04:43:44.131133 (+ 104us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 04:43:44.131360 (+ 227us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 04:43:44.131367 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 04:43:44.131492 (+ 125us) client_negotiation.cc:769] Sending connection context
0810 04:43:44.131577 (+ 85us) client_negotiation.cc:240] Negotiation successful
0810 04:43:44.131723 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":79,"threads_started":1}
I20260810 04:43:44.132436 14340 heartbeater.cc:347] Connected to a master server at 127.22.167.126:46073
I20260810 04:43:44.132516 14340 heartbeater.cc:464] Registering TS with master...
I20260810 04:43:44.132663 14340 heartbeater.cc:511] Master 127.22.167.126:46073 requested a full tablet report, sending...
I20260810 04:43:44.133108 14033 ts_manager.cc:194] Registered new tserver with Master: ed99582b3b574c00991af0deb19f27c1 (127.22.167.65:44335)
I20260810 04:43:44.133363 23197 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskemv_AM/build/release/bin/kudu as pid 14129
I20260810 04:43:44.133445 23197 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786336863388938-23197-0/minicluster-data/ts-0/wal/instance
I20260810 04:43:44.133652 23197 external_mini_cluster.cc:1428] Setting key 8db60b2646a0f53b580e682d8bd3f222
I20260810 04:43:44.133755 14033 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.167.65@KRBTEST.COM'} at 127.22.167.65:33521
WARNING: no policy specified for kudu/127.22.167.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.167.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.167.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786336863388938-23197-0/krb5kdc/kudu_127.22.167.66.keytab.
Entry for principal kudu/127.22.167.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786336863388938-23197-0/krb5kdc/kudu_127.22.167.66.keytab.
I20260810 04:43:44.156525 23197 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskemv_AM/build/release/bin/kudu
/tmp/dist-test-taskemv_AM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786336863388938-23197-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786336863388938-23197-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786336863388938-23197-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786336863388938-23197-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.167.66:0
--local_ip_for_outbound_sockets=127.22.167.66
--webserver_interface=127.22.167.66
--webserver_port=0
--tserver_master_addrs=127.22.167.126:46073
--keytab_file=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786336863388938-23197-0/krb5kdc/kudu_127.22.167.66.keytab
--principal=kudu/127.22.167.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.167.84:34205
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786336863388938-23197-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786336863388938-23197-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786336863388938-23197-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 04:43:44.236074 14355 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 04:43:44.236271 14355 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 04:43:44.236294 14355 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 04:43:44.237336 14355 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 04:43:44.237367 14355 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 04:43:44.237391 14355 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.167.66
I20260810 04:43:44.238816 14355 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.167.84:34205
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786336863388938-23197-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786336863388938-23197-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786336863388938-23197-0/krb5kdc/kudu_127.22.167.66.keytab
--principal=kudu/127.22.167.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.167.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786336863388938-23197-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.22.167.66
--webserver_port=0
--tserver_master_addrs=127.22.167.126:46073
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14355
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.167.66
--log_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786336863388938-23197-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 03:54:01 UTC on ccycloud.quasar-icijmt.root.comops.site
build id 4577009
I20260810 04:43:44.238998 14355 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 04:43:44.239176 14355 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 04:43:44.243880 14368 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 04:43:44.243921 14367 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 04:43:44.243881 14370 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 04:43:44.244091 14355 server_base.cc:1034] running on GCE node
I20260810 04:43:44.244275 14355 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 04:43:44.244480 14355 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 04:43:44.245641 14355 hybrid_clock.cc:648] HybridClock initialized: now 1786337024245624 us; error 32 us; skew 500 ppm
Aug 10 04:43:44 dist-test-slave-bqcl krb5kdc[13951](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786337024, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.167.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 04:43:44.248047 14355 init.cc:377] Logged in from keytab as kudu/127.22.167.66@KRBTEST.COM (short username kudu)
I20260810 04:43:44.248447 14355 webserver.cc:460] Webserver started at http://127.22.167.66:32775/ using document root <none> and password file <none>
I20260810 04:43:44.248625 14355 fs_manager.cc:359] Metadata directory not provided
I20260810 04:43:44.248663 14355 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 04:43:44.248752 14355 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 04:43:44.249667 14355 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786336863388938-23197-0/minicluster-data/ts-1/data/instance:
uuid: "61a9f79165b8438299946a8f4b034178"
format_stamp: "Formatted at 2026-08-10 04:43:44 on dist-test-slave-bqcl"
server_key: "91d51958193417a18cd51e9307e9fafc"
server_key_iv: "aa572f67f2d7ed0a2d36cdad795e0015"
server_key_version: "encryptionkey@0"
I20260810 04:43:44.249948 14355 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786336863388938-23197-0/minicluster-data/ts-1/wal/instance:
uuid: "61a9f79165b8438299946a8f4b034178"
format_stamp: "Formatted at 2026-08-10 04:43:44 on dist-test-slave-bqcl"
server_key: "91d51958193417a18cd51e9307e9fafc"
server_key_iv: "aa572f67f2d7ed0a2d36cdad795e0015"
server_key_version: "encryptionkey@0"
I20260810 04:43:44.251139 14355 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260810 04:43:44.251776 14379 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 04:43:44.251914 14355 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260810 04:43:44.251991 14355 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786336863388938-23197-0/minicluster-data/ts-1/data,/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786336863388938-23197-0/minicluster-data/ts-1/wal
uuid: "61a9f79165b8438299946a8f4b034178"
format_stamp: "Formatted at 2026-08-10 04:43:44 on dist-test-slave-bqcl"
server_key: "91d51958193417a18cd51e9307e9fafc"
server_key_iv: "aa572f67f2d7ed0a2d36cdad795e0015"
server_key_version: "encryptionkey@0"
I20260810 04:43:44.252053 14355 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786336863388938-23197-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786336863388938-23197-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786336863388938-23197-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 04:43:44.262946 14355 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 04:43:44.264868 14355 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 04:43:44.264991 14355 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 04:43:44.265183 14355 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 04:43:44.265458 14355 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 04:43:44.265491 14355 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 04:43:44.265518 14355 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 04:43:44.265542 14355 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 04:43:44.271190 14355 rpc_server.cc:307] RPC server started. Bound to: 127.22.167.66:38979
I20260810 04:43:44.271246 14572 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.167.66:38979 every 8 connection(s)
I20260810 04:43:44.271513 14355 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786336863388938-23197-0/minicluster-data/ts-1/data/info.pb
Aug 10 04:43:44 dist-test-slave-bqcl krb5kdc[13951](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786337024, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.167.66@KRBTEST.COM for kudu/127.22.167.126@KRBTEST.COM
I20260810 04:43:44.279413 14094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:43:44.272133 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.167.66:49395 (local address 127.22.167.126:46073)
0810 04:43:44.272178 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0810 04:43:44.272180 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 04:43:44.272377 (+ 197us) server_negotiation.cc:407] Connection header received
0810 04:43:44.272876 (+ 499us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:43:44.272877 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:43:44.272887 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:43:44.272939 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0810 04:43:44.273796 (+ 857us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:43:44.274298 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:43:44.275231 (+ 933us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:43:44.275404 (+ 173us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:43:44.275446 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:43:44.277328 (+ 1882us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 04:43:44.277330 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 04:43:44.277330 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 04:43:44.277337 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 04:43:44.278418 (+ 1081us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 04:43:44.278758 (+ 340us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 04:43:44.278758 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 04:43:44.278759 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 04:43:44.278803 (+ 44us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 04:43:44.278969 (+ 166us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 04:43:44.278970 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 04:43:44.278970 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 04:43:44.279059 (+ 89us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 04:43:44.279102 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0810 04:43:44.279288 (+ 186us) server_negotiation.cc:299] Negotiation successful
0810 04:43:44.279350 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260810 04:43:44.279585 14576 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:43:44.272098 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.126:46073 (local address 127.22.167.66:49395)
0810 04:43:44.272256 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0810 04:43:44.272296 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0810 04:43:44.272801 (+ 505us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 04:43:44.272998 (+ 197us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 04:43:44.273001 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 04:43:44.273525 (+ 524us) client_negotiation.cc:189] Negotiated authn=SASL
0810 04:43:44.273730 (+ 205us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:43:44.273735 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:43:44.274366 (+ 631us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:43:44.274366 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:43:44.275171 (+ 805us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:43:44.275173 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:43:44.275453 (+ 280us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:43:44.275453 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:43:44.275546 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:43:44.275979 (+ 433us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 04:43:44.275986 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0810 04:43:44.277242 (+ 1256us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 04:43:44.278476 (+ 1234us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 04:43:44.278483 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 04:43:44.278484 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 04:43:44.278704 (+ 220us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 04:43:44.278843 (+ 139us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 04:43:44.278844 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 04:43:44.278844 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 04:43:44.278930 (+ 86us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 04:43:44.279098 (+ 168us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 04:43:44.279101 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 04:43:44.279200 (+ 99us) client_negotiation.cc:769] Sending connection context
0810 04:43:44.279272 (+ 72us) client_negotiation.cc:240] Negotiation successful
0810 04:43:44.279378 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":83,"threads_started":1}
I20260810 04:43:44.280023 14573 heartbeater.cc:347] Connected to a master server at 127.22.167.126:46073
I20260810 04:43:44.280115 14573 heartbeater.cc:464] Registering TS with master...
I20260810 04:43:44.280290 14573 heartbeater.cc:511] Master 127.22.167.126:46073 requested a full tablet report, sending...
I20260810 04:43:44.280648 14033 ts_manager.cc:194] Registered new tserver with Master: 61a9f79165b8438299946a8f4b034178 (127.22.167.66:38979)
I20260810 04:43:44.280876 23197 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskemv_AM/build/release/bin/kudu as pid 14355
I20260810 04:43:44.280959 23197 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786336863388938-23197-0/minicluster-data/ts-1/wal/instance
I20260810 04:43:44.281059 14033 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.167.66@KRBTEST.COM'} at 127.22.167.66:49395
I20260810 04:43:44.281141 23197 external_mini_cluster.cc:1428] Setting key bbff3372331e3d8ba6ff34b92dc3d0d6
WARNING: no policy specified for kudu/127.22.167.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.167.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.167.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786336863388938-23197-0/krb5kdc/kudu_127.22.167.67.keytab.
Entry for principal kudu/127.22.167.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786336863388938-23197-0/krb5kdc/kudu_127.22.167.67.keytab.
I20260810 04:43:44.303920 23197 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskemv_AM/build/release/bin/kudu
/tmp/dist-test-taskemv_AM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786336863388938-23197-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786336863388938-23197-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786336863388938-23197-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786336863388938-23197-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.167.67:0
--local_ip_for_outbound_sockets=127.22.167.67
--webserver_interface=127.22.167.67
--webserver_port=0
--tserver_master_addrs=127.22.167.126:46073
--keytab_file=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786336863388938-23197-0/krb5kdc/kudu_127.22.167.67.keytab
--principal=kudu/127.22.167.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.167.84:34205
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786336863388938-23197-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786336863388938-23197-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786336863388938-23197-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 04:43:44.383266 14581 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 04:43:44.383464 14581 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 04:43:44.383494 14581 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 04:43:44.384600 14581 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 04:43:44.384639 14581 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 04:43:44.384675 14581 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.167.67
I20260810 04:43:44.386096 14581 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.167.84:34205
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786336863388938-23197-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786336863388938-23197-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786336863388938-23197-0/krb5kdc/kudu_127.22.167.67.keytab
--principal=kudu/127.22.167.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.167.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786336863388938-23197-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.22.167.67
--webserver_port=0
--tserver_master_addrs=127.22.167.126:46073
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14581
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.167.67
--log_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786336863388938-23197-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 03:54:01 UTC on ccycloud.quasar-icijmt.root.comops.site
build id 4577009
I20260810 04:43:44.386354 14581 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 04:43:44.386560 14581 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260810 04:43:44.391332 14581 server_base.cc:1034] running on GCE node
W20260810 04:43:44.391407 14593 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 04:43:44.391341 14589 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 04:43:44.391448 14588 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 04:43:44.391724 14581 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 04:43:44.391920 14581 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 04:43:44.393051 14581 hybrid_clock.cc:648] HybridClock initialized: now 1786337024393041 us; error 31 us; skew 500 ppm
Aug 10 04:43:44 dist-test-slave-bqcl krb5kdc[13951](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786337024, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.167.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 04:43:44.395437 14581 init.cc:377] Logged in from keytab as kudu/127.22.167.67@KRBTEST.COM (short username kudu)
I20260810 04:43:44.395798 14581 webserver.cc:460] Webserver started at http://127.22.167.67:35631/ using document root <none> and password file <none>
I20260810 04:43:44.395963 14581 fs_manager.cc:359] Metadata directory not provided
I20260810 04:43:44.396001 14581 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 04:43:44.396097 14581 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 04:43:44.396973 14581 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786336863388938-23197-0/minicluster-data/ts-2/data/instance:
uuid: "3a46d62cc08a49938add205cef6d55aa"
format_stamp: "Formatted at 2026-08-10 04:43:44 on dist-test-slave-bqcl"
server_key: "f5fb01f1cfad0883040e79fcf3eb13d1"
server_key_iv: "8a368d70585170c801565df9a6544025"
server_key_version: "encryptionkey@0"
I20260810 04:43:44.397274 14581 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786336863388938-23197-0/minicluster-data/ts-2/wal/instance:
uuid: "3a46d62cc08a49938add205cef6d55aa"
format_stamp: "Formatted at 2026-08-10 04:43:44 on dist-test-slave-bqcl"
server_key: "f5fb01f1cfad0883040e79fcf3eb13d1"
server_key_iv: "8a368d70585170c801565df9a6544025"
server_key_version: "encryptionkey@0"
I20260810 04:43:44.398437 14581 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260810 04:43:44.399053 14601 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 04:43:44.399214 14581 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260810 04:43:44.399291 14581 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786336863388938-23197-0/minicluster-data/ts-2/data,/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786336863388938-23197-0/minicluster-data/ts-2/wal
uuid: "3a46d62cc08a49938add205cef6d55aa"
format_stamp: "Formatted at 2026-08-10 04:43:44 on dist-test-slave-bqcl"
server_key: "f5fb01f1cfad0883040e79fcf3eb13d1"
server_key_iv: "8a368d70585170c801565df9a6544025"
server_key_version: "encryptionkey@0"
I20260810 04:43:44.399344 14581 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786336863388938-23197-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786336863388938-23197-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786336863388938-23197-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 04:43:44.428323 14581 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 04:43:44.430047 14581 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 04:43:44.430203 14581 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 04:43:44.430392 14581 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 04:43:44.430668 14581 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 04:43:44.430702 14581 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 04:43:44.430722 14581 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 04:43:44.430747 14581 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 04:43:44.436225 14581 rpc_server.cc:307] RPC server started. Bound to: 127.22.167.67:39865
I20260810 04:43:44.436266 14799 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.167.67:39865 every 8 connection(s)
I20260810 04:43:44.436547 14581 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786336863388938-23197-0/minicluster-data/ts-2/data/info.pb
I20260810 04:43:44.438333 23197 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskemv_AM/build/release/bin/kudu as pid 14581
I20260810 04:43:44.438422 23197 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786336863388938-23197-0/minicluster-data/ts-2/wal/instance
I20260810 04:43:44.438585 23197 external_mini_cluster.cc:1428] Setting key dfd12bdbe58722a92e2453d6d9c139fb
Aug 10 04:43:44 dist-test-slave-bqcl krb5kdc[13951](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786337024, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.167.67@KRBTEST.COM for kudu/127.22.167.126@KRBTEST.COM
I20260810 04:43:44.445204 14094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:43:44.437232 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.167.67:35749 (local address 127.22.167.126:46073)
0810 04:43:44.437292 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0810 04:43:44.437294 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 04:43:44.437486 (+ 192us) server_negotiation.cc:407] Connection header received
0810 04:43:44.437945 (+ 459us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:43:44.437946 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:43:44.437956 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:43:44.437993 (+ 37us) server_negotiation.cc:226] Negotiated authn=SASL
0810 04:43:44.439059 (+ 1066us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:43:44.439715 (+ 656us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:43:44.440803 (+ 1088us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:43:44.440996 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:43:44.441043 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:43:44.443098 (+ 2055us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 04:43:44.443099 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 04:43:44.443100 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 04:43:44.443106 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0810 04:43:44.444140 (+ 1034us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 04:43:44.444474 (+ 334us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 04:43:44.444475 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 04:43:44.444475 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 04:43:44.444518 (+ 43us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 04:43:44.444694 (+ 176us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 04:43:44.444695 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 04:43:44.444695 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 04:43:44.444784 (+ 89us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 04:43:44.444826 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0810 04:43:44.445077 (+ 251us) server_negotiation.cc:299] Negotiation successful
0810 04:43:44.445139 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260810 04:43:44.445374 14803 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:43:44.437221 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.126:46073 (local address 127.22.167.67:35749)
0810 04:43:44.437380 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0810 04:43:44.437415 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0810 04:43:44.437860 (+ 445us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 04:43:44.438049 (+ 189us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 04:43:44.438052 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 04:43:44.438752 (+ 700us) client_negotiation.cc:189] Negotiated authn=SASL
0810 04:43:44.438989 (+ 237us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:43:44.438994 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:43:44.439798 (+ 804us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:43:44.439799 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:43:44.440733 (+ 934us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:43:44.440736 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:43:44.441115 (+ 379us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:43:44.441116 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:43:44.441224 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:43:44.441714 (+ 490us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 04:43:44.441722 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0810 04:43:44.443012 (+ 1290us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 04:43:44.444212 (+ 1200us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 04:43:44.444218 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 04:43:44.444220 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 04:43:44.444420 (+ 200us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 04:43:44.444559 (+ 139us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 04:43:44.444559 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 04:43:44.444560 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 04:43:44.444652 (+ 92us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 04:43:44.444824 (+ 172us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 04:43:44.444826 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 04:43:44.444956 (+ 130us) client_negotiation.cc:769] Sending connection context
0810 04:43:44.445048 (+ 92us) client_negotiation.cc:240] Negotiation successful
0810 04:43:44.445172 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":88,"threads_started":1}
I20260810 04:43:44.445780 14800 heartbeater.cc:347] Connected to a master server at 127.22.167.126:46073
I20260810 04:43:44.445870 14800 heartbeater.cc:464] Registering TS with master...
I20260810 04:43:44.446030 14800 heartbeater.cc:511] Master 127.22.167.126:46073 requested a full tablet report, sending...
I20260810 04:43:44.446393 14033 ts_manager.cc:194] Registered new tserver with Master: 3a46d62cc08a49938add205cef6d55aa (127.22.167.67:39865)
I20260810 04:43:44.446746 14033 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.167.67@KRBTEST.COM'} at 127.22.167.67:35749
I20260810 04:43:44.450145 23197 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260810 04:43:44.456564 14094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:43:44.451367 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38534 (local address 127.22.167.126:46073)
0810 04:43:44.451407 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0810 04:43:44.451409 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 04:43:44.451512 (+ 103us) server_negotiation.cc:407] Connection header received
0810 04:43:44.451571 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:43:44.451572 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:43:44.451580 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:43:44.451624 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0810 04:43:44.452140 (+ 516us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:43:44.452593 (+ 453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:43:44.453292 (+ 699us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:43:44.453485 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:43:44.453537 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:43:44.454561 (+ 1024us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 04:43:44.454563 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 04:43:44.454563 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 04:43:44.454570 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 04:43:44.455576 (+ 1006us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 04:43:44.455872 (+ 296us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 04:43:44.455873 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 04:43:44.455874 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 04:43:44.455927 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 04:43:44.456079 (+ 152us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 04:43:44.456079 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 04:43:44.456080 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 04:43:44.456221 (+ 141us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 04:43:44.456311 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0810 04:43:44.456408 (+ 97us) server_negotiation.cc:299] Negotiation successful
0810 04:43:44.456483 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260810 04:43:44.457782 14033 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38534:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 04:43:44.459319 14033 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 04:43:44.465708 14818 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:43:44.461504 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35824 (local address 127.22.167.65:44335)
0810 04:43:44.461803 (+ 299us) server_negotiation.cc:206] Beginning negotiation
0810 04:43:44.461808 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 04:43:44.461854 (+ 46us) server_negotiation.cc:407] Connection header received
0810 04:43:44.461907 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:43:44.461910 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:43:44.461926 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:43:44.461994 (+ 68us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 04:43:44.462235 (+ 241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:43:44.463322 (+ 1087us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:43:44.464580 (+ 1258us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:43:44.465318 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:43:44.465391 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:43:44.465451 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0810 04:43:44.465590 (+ 139us) server_negotiation.cc:299] Negotiation successful
0810 04:43:44.465628 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":263,"thread_start_us":88,"threads_started":1}
I20260810 04:43:44.465708 14814 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:43:44.461229 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.66:38979 (local address 127.0.0.1:47216)
0810 04:43:44.461349 (+ 120us) negotiation.cc:107] Waiting for socket to connect
0810 04:43:44.461390 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0810 04:43:44.461460 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 04:43:44.462048 (+ 588us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 04:43:44.462049 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 04:43:44.462067 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 04:43:44.462225 (+ 158us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:43:44.462228 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:43:44.463233 (+ 1005us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:43:44.463234 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:43:44.464503 (+ 1269us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:43:44.464505 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:43:44.465408 (+ 903us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:43:44.465409 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:43:44.465492 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:43:44.465496 (+ 4us) client_negotiation.cc:769] Sending connection context
0810 04:43:44.465537 (+ 41us) client_negotiation.cc:240] Negotiation successful
0810 04:43:44.465619 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":98,"mutex_wait_us":2,"thread_start_us":48,"threads_started":1}
I20260810 04:43:44.465710 14813 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:43:44.461230 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.65:44335 (local address 127.0.0.1:35824)
0810 04:43:44.461776 (+ 546us) negotiation.cc:107] Waiting for socket to connect
0810 04:43:44.461787 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0810 04:43:44.461852 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 04:43:44.462091 (+ 239us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 04:43:44.462092 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 04:43:44.462094 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 04:43:44.462158 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:43:44.462159 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:43:44.463421 (+ 1262us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:43:44.463422 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:43:44.464503 (+ 1081us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:43:44.464505 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:43:44.465392 (+ 887us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:43:44.465393 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:43:44.465480 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:43:44.465495 (+ 15us) client_negotiation.cc:769] Sending connection context
0810 04:43:44.465566 (+ 71us) client_negotiation.cc:240] Negotiation successful
0810 04:43:44.465615 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":513,"mutex_wait_us":5,"thread_start_us":54,"threads_started":1}
I20260810 04:43:44.466156 14817 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:43:44.461471 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47216 (local address 127.22.167.66:38979)
0810 04:43:44.461882 (+ 411us) server_negotiation.cc:206] Beginning negotiation
0810 04:43:44.461886 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 04:43:44.461896 (+ 10us) server_negotiation.cc:407] Connection header received
0810 04:43:44.461933 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:43:44.461935 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:43:44.461949 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:43:44.462306 (+ 357us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 04:43:44.462367 (+ 61us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:43:44.463141 (+ 774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:43:44.464588 (+ 1447us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:43:44.465318 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:43:44.465916 (+ 598us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:43:44.465987 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0810 04:43:44.466012 (+ 25us) server_negotiation.cc:299] Negotiation successful
0810 04:43:44.466057 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":380,"thread_start_us":83,"threads_started":1}
I20260810 04:43:44.466554 14810 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:43:44.461229 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.67:39865 (local address 127.0.0.1:36950)
0810 04:43:44.461703 (+ 474us) negotiation.cc:107] Waiting for socket to connect
0810 04:43:44.461711 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0810 04:43:44.461776 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 04:43:44.462027 (+ 251us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 04:43:44.462030 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 04:43:44.462033 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 04:43:44.462090 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:43:44.462092 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:43:44.464601 (+ 2509us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:43:44.464602 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:43:44.465277 (+ 675us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:43:44.465278 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:43:44.466090 (+ 812us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:43:44.466091 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:43:44.466389 (+ 298us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:43:44.466394 (+ 5us) client_negotiation.cc:769] Sending connection context
0810 04:43:44.466447 (+ 53us) client_negotiation.cc:240] Negotiation successful
0810 04:43:44.466485 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":463,"mutex_wait_us":5,"thread_start_us":46,"threads_started":1}
I20260810 04:43:44.466821 14816 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:43:44.461428 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36950 (local address 127.22.167.67:39865)
0810 04:43:44.461638 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0810 04:43:44.461642 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 04:43:44.461787 (+ 145us) server_negotiation.cc:407] Connection header received
0810 04:43:44.461832 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:43:44.461835 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:43:44.461851 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:43:44.461937 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 04:43:44.462213 (+ 276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:43:44.463334 (+ 1121us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:43:44.465354 (+ 2020us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:43:44.466020 (+ 666us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:43:44.466079 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:43:44.466175 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0810 04:43:44.466706 (+ 531us) server_negotiation.cc:299] Negotiation successful
0810 04:43:44.466750 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":175,"thread_start_us":105,"threads_started":1}
I20260810 04:43:44.466779 14238 tablet_service.cc:1467] Processing CreateTablet for tablet 3658dd055dba4012b05196ed90e44d45 (DEFAULT_TABLE table=test-table [id=5029120d5b634e148b09c0df13a52d78]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 04:43:44.466781 14456 tablet_service.cc:1467] Processing CreateTablet for tablet 3658dd055dba4012b05196ed90e44d45 (DEFAULT_TABLE table=test-table [id=5029120d5b634e148b09c0df13a52d78]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 04:43:44.467126 14238 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3658dd055dba4012b05196ed90e44d45. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 04:43:44.467126 14456 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3658dd055dba4012b05196ed90e44d45. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 04:43:44.467422 14685 tablet_service.cc:1467] Processing CreateTablet for tablet 3658dd055dba4012b05196ed90e44d45 (DEFAULT_TABLE table=test-table [id=5029120d5b634e148b09c0df13a52d78]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 04:43:44.467717 14685 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3658dd055dba4012b05196ed90e44d45. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 04:43:44.470036 14819 tablet_bootstrap.cc:492] T 3658dd055dba4012b05196ed90e44d45 P 3a46d62cc08a49938add205cef6d55aa: Bootstrap starting.
I20260810 04:43:44.470580 14822 tablet_bootstrap.cc:492] T 3658dd055dba4012b05196ed90e44d45 P ed99582b3b574c00991af0deb19f27c1: Bootstrap starting.
I20260810 04:43:44.470841 14819 tablet_bootstrap.cc:654] T 3658dd055dba4012b05196ed90e44d45 P 3a46d62cc08a49938add205cef6d55aa: Neither blocks nor log segments found. Creating new log.
I20260810 04:43:44.471244 14822 tablet_bootstrap.cc:654] T 3658dd055dba4012b05196ed90e44d45 P ed99582b3b574c00991af0deb19f27c1: Neither blocks nor log segments found. Creating new log.
I20260810 04:43:44.471415 14819 log.cc:824] T 3658dd055dba4012b05196ed90e44d45 P 3a46d62cc08a49938add205cef6d55aa: Log is configured to *not* fsync() on all Append() calls
I20260810 04:43:44.471794 14822 log.cc:824] T 3658dd055dba4012b05196ed90e44d45 P ed99582b3b574c00991af0deb19f27c1: Log is configured to *not* fsync() on all Append() calls
I20260810 04:43:44.472114 14824 tablet_bootstrap.cc:492] T 3658dd055dba4012b05196ed90e44d45 P 61a9f79165b8438299946a8f4b034178: Bootstrap starting.
I20260810 04:43:44.472251 14819 tablet_bootstrap.cc:492] T 3658dd055dba4012b05196ed90e44d45 P 3a46d62cc08a49938add205cef6d55aa: No bootstrap required, opened a new log
I20260810 04:43:44.472321 14819 ts_tablet_manager.cc:1397] T 3658dd055dba4012b05196ed90e44d45 P 3a46d62cc08a49938add205cef6d55aa: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260810 04:43:44.472476 14822 tablet_bootstrap.cc:492] T 3658dd055dba4012b05196ed90e44d45 P ed99582b3b574c00991af0deb19f27c1: No bootstrap required, opened a new log
I20260810 04:43:44.472536 14822 ts_tablet_manager.cc:1397] T 3658dd055dba4012b05196ed90e44d45 P ed99582b3b574c00991af0deb19f27c1: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260810 04:43:44.472786 14824 tablet_bootstrap.cc:654] T 3658dd055dba4012b05196ed90e44d45 P 61a9f79165b8438299946a8f4b034178: Neither blocks nor log segments found. Creating new log.
I20260810 04:43:44.473318 14824 log.cc:824] T 3658dd055dba4012b05196ed90e44d45 P 61a9f79165b8438299946a8f4b034178: Log is configured to *not* fsync() on all Append() calls
I20260810 04:43:44.474082 14824 tablet_bootstrap.cc:492] T 3658dd055dba4012b05196ed90e44d45 P 61a9f79165b8438299946a8f4b034178: No bootstrap required, opened a new log
I20260810 04:43:44.474177 14824 ts_tablet_manager.cc:1397] T 3658dd055dba4012b05196ed90e44d45 P 61a9f79165b8438299946a8f4b034178: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260810 04:43:44.474236 14822 raft_consensus.cc:348] T 3658dd055dba4012b05196ed90e44d45 P ed99582b3b574c00991af0deb19f27c1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "61a9f79165b8438299946a8f4b034178" member_type: VOTER last_known_addr { host: "127.22.167.66" port: 38979 } } peers { permanent_uuid: "ed99582b3b574c00991af0deb19f27c1" member_type: VOTER last_known_addr { host: "127.22.167.65" port: 44335 } } peers { permanent_uuid: "3a46d62cc08a49938add205cef6d55aa" member_type: VOTER last_known_addr { host: "127.22.167.67" port: 39865 } }
I20260810 04:43:44.474289 14819 raft_consensus.cc:348] T 3658dd055dba4012b05196ed90e44d45 P 3a46d62cc08a49938add205cef6d55aa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "61a9f79165b8438299946a8f4b034178" member_type: VOTER last_known_addr { host: "127.22.167.66" port: 38979 } } peers { permanent_uuid: "ed99582b3b574c00991af0deb19f27c1" member_type: VOTER last_known_addr { host: "127.22.167.65" port: 44335 } } peers { permanent_uuid: "3a46d62cc08a49938add205cef6d55aa" member_type: VOTER last_known_addr { host: "127.22.167.67" port: 39865 } }
I20260810 04:43:44.474406 14822 raft_consensus.cc:374] T 3658dd055dba4012b05196ed90e44d45 P ed99582b3b574c00991af0deb19f27c1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 04:43:44.474435 14819 raft_consensus.cc:374] T 3658dd055dba4012b05196ed90e44d45 P 3a46d62cc08a49938add205cef6d55aa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 04:43:44.474468 14822 raft_consensus.cc:729] T 3658dd055dba4012b05196ed90e44d45 P ed99582b3b574c00991af0deb19f27c1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ed99582b3b574c00991af0deb19f27c1, State: Initialized, Role: FOLLOWER
I20260810 04:43:44.474500 14819 raft_consensus.cc:729] T 3658dd055dba4012b05196ed90e44d45 P 3a46d62cc08a49938add205cef6d55aa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3a46d62cc08a49938add205cef6d55aa, State: Initialized, Role: FOLLOWER
I20260810 04:43:44.474625 14819 consensus_queue.cc:260] T 3658dd055dba4012b05196ed90e44d45 P 3a46d62cc08a49938add205cef6d55aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "61a9f79165b8438299946a8f4b034178" member_type: VOTER last_known_addr { host: "127.22.167.66" port: 38979 } } peers { permanent_uuid: "ed99582b3b574c00991af0deb19f27c1" member_type: VOTER last_known_addr { host: "127.22.167.65" port: 44335 } } peers { permanent_uuid: "3a46d62cc08a49938add205cef6d55aa" member_type: VOTER last_known_addr { host: "127.22.167.67" port: 39865 } }
I20260810 04:43:44.474627 14822 consensus_queue.cc:260] T 3658dd055dba4012b05196ed90e44d45 P ed99582b3b574c00991af0deb19f27c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "61a9f79165b8438299946a8f4b034178" member_type: VOTER last_known_addr { host: "127.22.167.66" port: 38979 } } peers { permanent_uuid: "ed99582b3b574c00991af0deb19f27c1" member_type: VOTER last_known_addr { host: "127.22.167.65" port: 44335 } } peers { permanent_uuid: "3a46d62cc08a49938add205cef6d55aa" member_type: VOTER last_known_addr { host: "127.22.167.67" port: 39865 } }
I20260810 04:43:44.474905 14822 ts_tablet_manager.cc:1428] T 3658dd055dba4012b05196ed90e44d45 P ed99582b3b574c00991af0deb19f27c1: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260810 04:43:44.474941 14340 heartbeater.cc:503] Master 127.22.167.126:46073 was elected leader, sending a full tablet report...
I20260810 04:43:44.475191 14819 ts_tablet_manager.cc:1428] T 3658dd055dba4012b05196ed90e44d45 P 3a46d62cc08a49938add205cef6d55aa: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260810 04:43:44.475334 14800 heartbeater.cc:503] Master 127.22.167.126:46073 was elected leader, sending a full tablet report...
I20260810 04:43:44.476058 14824 raft_consensus.cc:348] T 3658dd055dba4012b05196ed90e44d45 P 61a9f79165b8438299946a8f4b034178 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "61a9f79165b8438299946a8f4b034178" member_type: VOTER last_known_addr { host: "127.22.167.66" port: 38979 } } peers { permanent_uuid: "ed99582b3b574c00991af0deb19f27c1" member_type: VOTER last_known_addr { host: "127.22.167.65" port: 44335 } } peers { permanent_uuid: "3a46d62cc08a49938add205cef6d55aa" member_type: VOTER last_known_addr { host: "127.22.167.67" port: 39865 } }
I20260810 04:43:44.476241 14824 raft_consensus.cc:374] T 3658dd055dba4012b05196ed90e44d45 P 61a9f79165b8438299946a8f4b034178 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 04:43:44.476320 14824 raft_consensus.cc:729] T 3658dd055dba4012b05196ed90e44d45 P 61a9f79165b8438299946a8f4b034178 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 61a9f79165b8438299946a8f4b034178, State: Initialized, Role: FOLLOWER
I20260810 04:43:44.476436 14824 consensus_queue.cc:260] T 3658dd055dba4012b05196ed90e44d45 P 61a9f79165b8438299946a8f4b034178 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "61a9f79165b8438299946a8f4b034178" member_type: VOTER last_known_addr { host: "127.22.167.66" port: 38979 } } peers { permanent_uuid: "ed99582b3b574c00991af0deb19f27c1" member_type: VOTER last_known_addr { host: "127.22.167.65" port: 44335 } } peers { permanent_uuid: "3a46d62cc08a49938add205cef6d55aa" member_type: VOTER last_known_addr { host: "127.22.167.67" port: 39865 } }
I20260810 04:43:44.476719 14573 heartbeater.cc:503] Master 127.22.167.126:46073 was elected leader, sending a full tablet report...
I20260810 04:43:44.476707 14824 ts_tablet_manager.cc:1428] T 3658dd055dba4012b05196ed90e44d45 P 61a9f79165b8438299946a8f4b034178: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
W20260810 04:43:44.522018 14574 tablet.cc:2369] T 3658dd055dba4012b05196ed90e44d45 P 61a9f79165b8438299946a8f4b034178: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 04:43:44.579921 14837 raft_consensus.cc:482] T 3658dd055dba4012b05196ed90e44d45 P 61a9f79165b8438299946a8f4b034178 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 04:43:44.580044 14837 raft_consensus.cc:504] T 3658dd055dba4012b05196ed90e44d45 P 61a9f79165b8438299946a8f4b034178 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "61a9f79165b8438299946a8f4b034178" member_type: VOTER last_known_addr { host: "127.22.167.66" port: 38979 } } peers { permanent_uuid: "ed99582b3b574c00991af0deb19f27c1" member_type: VOTER last_known_addr { host: "127.22.167.65" port: 44335 } } peers { permanent_uuid: "3a46d62cc08a49938add205cef6d55aa" member_type: VOTER last_known_addr { host: "127.22.167.67" port: 39865 } }
I20260810 04:43:44.580428 14837 leader_election.cc:288] T 3658dd055dba4012b05196ed90e44d45 P 61a9f79165b8438299946a8f4b034178 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ed99582b3b574c00991af0deb19f27c1 (127.22.167.65:44335), 3a46d62cc08a49938add205cef6d55aa (127.22.167.67:39865)
I20260810 04:43:44.584059 14818 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:43:44.580652 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.167.66:44649 (local address 127.22.167.65:44335)
0810 04:43:44.580711 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0810 04:43:44.580713 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 04:43:44.581001 (+ 288us) server_negotiation.cc:407] Connection header received
0810 04:43:44.581066 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:43:44.581067 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:43:44.581077 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:43:44.581132 (+ 55us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 04:43:44.581250 (+ 118us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:43:44.581937 (+ 687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:43:44.583143 (+ 1206us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:43:44.583713 (+ 570us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:43:44.583786 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:43:44.583795 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0810 04:43:44.583949 (+ 154us) server_negotiation.cc:299] Negotiation successful
0810 04:43:44.583985 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260810 04:43:44.584059 14816 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:43:44.580685 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.167.66:47371 (local address 127.22.167.67:39865)
0810 04:43:44.580740 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0810 04:43:44.580742 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 04:43:44.580750 (+ 8us) server_negotiation.cc:407] Connection header received
0810 04:43:44.580776 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:43:44.580777 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:43:44.580787 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:43:44.580841 (+ 54us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 04:43:44.581033 (+ 192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:43:44.581781 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:43:44.583130 (+ 1349us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:43:44.583713 (+ 583us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:43:44.583786 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:43:44.583795 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0810 04:43:44.583949 (+ 154us) server_negotiation.cc:299] Negotiation successful
0810 04:43:44.583985 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260810 04:43:44.584077 14576 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:43:44.580593 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.67:39865 (local address 127.22.167.66:47371)
0810 04:43:44.580646 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0810 04:43:44.580658 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0810 04:43:44.580702 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 04:43:44.580859 (+ 157us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 04:43:44.580861 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 04:43:44.580863 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 04:43:44.580945 (+ 82us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:43:44.580947 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:43:44.581855 (+ 908us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:43:44.581856 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:43:44.583047 (+ 1191us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:43:44.583050 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:43:44.583788 (+ 738us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:43:44.583789 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:43:44.583885 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:43:44.583887 (+ 2us) client_negotiation.cc:769] Sending connection context
0810 04:43:44.583936 (+ 49us) client_negotiation.cc:240] Negotiation successful
0810 04:43:44.583987 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":31,"mutex_wait_us":7}
I20260810 04:43:44.584084 14838 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:43:44.580595 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.65:44335 (local address 127.22.167.66:44649)
0810 04:43:44.580926 (+ 331us) negotiation.cc:107] Waiting for socket to connect
0810 04:43:44.580936 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0810 04:43:44.580996 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 04:43:44.581138 (+ 142us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 04:43:44.581139 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 04:43:44.581140 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 04:43:44.581194 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:43:44.581195 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:43:44.581998 (+ 803us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:43:44.581998 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:43:44.583076 (+ 1078us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:43:44.583078 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:43:44.583789 (+ 711us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:43:44.583789 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:43:44.583880 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:43:44.583884 (+ 4us) client_negotiation.cc:769] Sending connection context
0810 04:43:44.583936 (+ 52us) client_negotiation.cc:240] Negotiation successful
0810 04:43:44.583987 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":312,"mutex_wait_us":159,"thread_start_us":110,"threads_started":1}
I20260810 04:43:44.584475 14724 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3658dd055dba4012b05196ed90e44d45" candidate_uuid: "61a9f79165b8438299946a8f4b034178" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3a46d62cc08a49938add205cef6d55aa" is_pre_election: true
I20260810 04:43:44.584669 14724 raft_consensus.cc:2454] T 3658dd055dba4012b05196ed90e44d45 P 3a46d62cc08a49938add205cef6d55aa [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 61a9f79165b8438299946a8f4b034178 in term 0.
I20260810 04:43:44.584749 14265 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3658dd055dba4012b05196ed90e44d45" candidate_uuid: "61a9f79165b8438299946a8f4b034178" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ed99582b3b574c00991af0deb19f27c1" is_pre_election: true
I20260810 04:43:44.584916 14265 raft_consensus.cc:2454] T 3658dd055dba4012b05196ed90e44d45 P ed99582b3b574c00991af0deb19f27c1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 61a9f79165b8438299946a8f4b034178 in term 0.
I20260810 04:43:44.584900 14392 leader_election.cc:302] T 3658dd055dba4012b05196ed90e44d45 P 61a9f79165b8438299946a8f4b034178 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a46d62cc08a49938add205cef6d55aa, 61a9f79165b8438299946a8f4b034178; no voters:
I20260810 04:43:44.585044 14837 raft_consensus.cc:2781] T 3658dd055dba4012b05196ed90e44d45 P 61a9f79165b8438299946a8f4b034178 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 04:43:44.585100 14837 raft_consensus.cc:482] T 3658dd055dba4012b05196ed90e44d45 P 61a9f79165b8438299946a8f4b034178 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 04:43:44.585122 14837 raft_consensus.cc:3037] T 3658dd055dba4012b05196ed90e44d45 P 61a9f79165b8438299946a8f4b034178 [term 0 FOLLOWER]: Advancing to term 1
I20260810 04:43:44.585825 14837 raft_consensus.cc:504] T 3658dd055dba4012b05196ed90e44d45 P 61a9f79165b8438299946a8f4b034178 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "61a9f79165b8438299946a8f4b034178" member_type: VOTER last_known_addr { host: "127.22.167.66" port: 38979 } } peers { permanent_uuid: "ed99582b3b574c00991af0deb19f27c1" member_type: VOTER last_known_addr { host: "127.22.167.65" port: 44335 } } peers { permanent_uuid: "3a46d62cc08a49938add205cef6d55aa" member_type: VOTER last_known_addr { host: "127.22.167.67" port: 39865 } }
I20260810 04:43:44.586005 14837 leader_election.cc:288] T 3658dd055dba4012b05196ed90e44d45 P 61a9f79165b8438299946a8f4b034178 [CANDIDATE]: Term 1 election: Requested vote from peers ed99582b3b574c00991af0deb19f27c1 (127.22.167.65:44335), 3a46d62cc08a49938add205cef6d55aa (127.22.167.67:39865)
I20260810 04:43:44.586093 14265 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3658dd055dba4012b05196ed90e44d45" candidate_uuid: "61a9f79165b8438299946a8f4b034178" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ed99582b3b574c00991af0deb19f27c1"
I20260810 04:43:44.586083 14724 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3658dd055dba4012b05196ed90e44d45" candidate_uuid: "61a9f79165b8438299946a8f4b034178" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3a46d62cc08a49938add205cef6d55aa"
I20260810 04:43:44.586182 14265 raft_consensus.cc:3037] T 3658dd055dba4012b05196ed90e44d45 P ed99582b3b574c00991af0deb19f27c1 [term 0 FOLLOWER]: Advancing to term 1
I20260810 04:43:44.586182 14724 raft_consensus.cc:3037] T 3658dd055dba4012b05196ed90e44d45 P 3a46d62cc08a49938add205cef6d55aa [term 0 FOLLOWER]: Advancing to term 1
I20260810 04:43:44.587055 14724 raft_consensus.cc:2454] T 3658dd055dba4012b05196ed90e44d45 P 3a46d62cc08a49938add205cef6d55aa [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 61a9f79165b8438299946a8f4b034178 in term 1.
I20260810 04:43:44.587055 14265 raft_consensus.cc:2454] T 3658dd055dba4012b05196ed90e44d45 P ed99582b3b574c00991af0deb19f27c1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 61a9f79165b8438299946a8f4b034178 in term 1.
I20260810 04:43:44.587209 14392 leader_election.cc:302] T 3658dd055dba4012b05196ed90e44d45 P 61a9f79165b8438299946a8f4b034178 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a46d62cc08a49938add205cef6d55aa, 61a9f79165b8438299946a8f4b034178; no voters:
I20260810 04:43:44.587294 14837 raft_consensus.cc:2781] T 3658dd055dba4012b05196ed90e44d45 P 61a9f79165b8438299946a8f4b034178 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 04:43:44.587441 14837 raft_consensus.cc:686] T 3658dd055dba4012b05196ed90e44d45 P 61a9f79165b8438299946a8f4b034178 [term 1 LEADER]: Becoming Leader. State: Replica: 61a9f79165b8438299946a8f4b034178, State: Running, Role: LEADER
I20260810 04:43:44.587520 14837 consensus_queue.cc:237] T 3658dd055dba4012b05196ed90e44d45 P 61a9f79165b8438299946a8f4b034178 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "61a9f79165b8438299946a8f4b034178" member_type: VOTER last_known_addr { host: "127.22.167.66" port: 38979 } } peers { permanent_uuid: "ed99582b3b574c00991af0deb19f27c1" member_type: VOTER last_known_addr { host: "127.22.167.65" port: 44335 } } peers { permanent_uuid: "3a46d62cc08a49938add205cef6d55aa" member_type: VOTER last_known_addr { host: "127.22.167.67" port: 39865 } }
I20260810 04:43:44.588327 14032 catalog_manager.cc:5495] T 3658dd055dba4012b05196ed90e44d45 P 61a9f79165b8438299946a8f4b034178 reported cstate change: term changed from 0 to 1, leader changed from <none> to 61a9f79165b8438299946a8f4b034178 (127.22.167.66). New cstate: current_term: 1 leader_uuid: "61a9f79165b8438299946a8f4b034178" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "61a9f79165b8438299946a8f4b034178" member_type: VOTER last_known_addr { host: "127.22.167.66" port: 38979 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ed99582b3b574c00991af0deb19f27c1" member_type: VOTER last_known_addr { host: "127.22.167.65" port: 44335 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3a46d62cc08a49938add205cef6d55aa" member_type: VOTER last_known_addr { host: "127.22.167.67" port: 39865 } health_report { overall_health: UNKNOWN } } }
I20260810 04:43:44.611650 14817 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:43:44.609662 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47222 (local address 127.22.167.66:38979)
0810 04:43:44.609700 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0810 04:43:44.609702 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 04:43:44.609710 (+ 8us) server_negotiation.cc:407] Connection header received
0810 04:43:44.609766 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:43:44.609766 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:43:44.609782 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:43:44.609840 (+ 58us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 04:43:44.609925 (+ 85us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:43:44.610419 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:43:44.611080 (+ 661us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:43:44.611254 (+ 174us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:43:44.611294 (+ 40us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:43:44.611390 (+ 96us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 04:43:44.611487 (+ 97us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 04:43:44.611538 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0810 04:43:44.611551 (+ 13us) server_negotiation.cc:299] Negotiation successful
0810 04:43:44.611582 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260810 04:43:44.613214 14265 raft_consensus.cc:1261] T 3658dd055dba4012b05196ed90e44d45 P ed99582b3b574c00991af0deb19f27c1 [term 1 FOLLOWER]: Refusing update from remote peer 61a9f79165b8438299946a8f4b034178: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 04:43:44.613484 14724 raft_consensus.cc:1261] T 3658dd055dba4012b05196ed90e44d45 P 3a46d62cc08a49938add205cef6d55aa [term 1 FOLLOWER]: Refusing update from remote peer 61a9f79165b8438299946a8f4b034178: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 04:43:44.613518 14837 consensus_queue.cc:1035] T 3658dd055dba4012b05196ed90e44d45 P 61a9f79165b8438299946a8f4b034178 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ed99582b3b574c00991af0deb19f27c1" member_type: VOTER last_known_addr { host: "127.22.167.65" port: 44335 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 04:43:44.613765 14837 consensus_queue.cc:1035] T 3658dd055dba4012b05196ed90e44d45 P 61a9f79165b8438299946a8f4b034178 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a46d62cc08a49938add205cef6d55aa" member_type: VOTER last_known_addr { host: "127.22.167.67" port: 39865 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 04:43:44.615763 14843 mvcc.cc:205] Tried to move back new op lower bound from 7316836452813438976 to 7316836452710977536. Current Snapshot: MvccSnapshot[applied={T|T < 7316836452813438976}]
I20260810 04:43:44.616526 14844 mvcc.cc:205] Tried to move back new op lower bound from 7316836452813438976 to 7316836452710977536. Current Snapshot: MvccSnapshot[applied={T|T < 7316836452813438976}]
I20260810 04:43:44.618285 14033 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38534:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 04:43:44.618369 14033 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38534:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 04:43:44.618992 14033 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 1623678ec1f5413687149d71bb0bce92: Sending DeleteTablet for 3 replicas of tablet 3658dd055dba4012b05196ed90e44d45
I20260810 04:43:44.619326 14685 tablet_service.cc:1514] Processing DeleteTablet for tablet 3658dd055dba4012b05196ed90e44d45 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 04:43:44 UTC) from {username='kudu', principal='kudu/127.22.167.126@KRBTEST.COM'} at 127.0.0.1:36950
I20260810 04:43:44.619328 14238 tablet_service.cc:1514] Processing DeleteTablet for tablet 3658dd055dba4012b05196ed90e44d45 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 04:43:44 UTC) from {username='kudu', principal='kudu/127.22.167.126@KRBTEST.COM'} at 127.0.0.1:35824
I20260810 04:43:44.619374 23197 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskemv_AM/build/release/bin/kudu with pid 14129
I20260810 04:43:44.619427 14456 tablet_service.cc:1514] Processing DeleteTablet for tablet 3658dd055dba4012b05196ed90e44d45 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 04:43:44 UTC) from {username='kudu', principal='kudu/127.22.167.126@KRBTEST.COM'} at 127.0.0.1:47216
I20260810 04:43:44.626715 14857 tablet_replica.cc:330] T 3658dd055dba4012b05196ed90e44d45 P 3a46d62cc08a49938add205cef6d55aa: stopping tablet replica
I20260810 04:43:44.626817 14857 raft_consensus.cc:2229] T 3658dd055dba4012b05196ed90e44d45 P 3a46d62cc08a49938add205cef6d55aa [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 04:43:44.626905 14857 raft_consensus.cc:2258] T 3658dd055dba4012b05196ed90e44d45 P 3a46d62cc08a49938add205cef6d55aa [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 04:43:44.627382 14859 tablet_replica.cc:330] T 3658dd055dba4012b05196ed90e44d45 P 61a9f79165b8438299946a8f4b034178: stopping tablet replica
I20260810 04:43:44.627449 14857 ts_tablet_manager.cc:1905] T 3658dd055dba4012b05196ed90e44d45 P 3a46d62cc08a49938add205cef6d55aa: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 04:43:44.627552 14859 raft_consensus.cc:2229] T 3658dd055dba4012b05196ed90e44d45 P 61a9f79165b8438299946a8f4b034178 [term 1 LEADER]: Raft consensus shutting down.
I20260810 04:43:44.627735 14859 raft_consensus.cc:2258] T 3658dd055dba4012b05196ed90e44d45 P 61a9f79165b8438299946a8f4b034178 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 04:43:44.628222 14859 ts_tablet_manager.cc:1905] T 3658dd055dba4012b05196ed90e44d45 P 61a9f79165b8438299946a8f4b034178: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 04:43:44.629415 14857 ts_tablet_manager.cc:1918] T 3658dd055dba4012b05196ed90e44d45 P 3a46d62cc08a49938add205cef6d55aa: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 04:43:44.629580 14857 log.cc:1196] T 3658dd055dba4012b05196ed90e44d45 P 3a46d62cc08a49938add205cef6d55aa: Deleting WAL directory at /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786336863388938-23197-0/minicluster-data/ts-2/wal/wals/3658dd055dba4012b05196ed90e44d45
I20260810 04:43:44.629848 14859 ts_tablet_manager.cc:1918] T 3658dd055dba4012b05196ed90e44d45 P 61a9f79165b8438299946a8f4b034178: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 04:43:44.629997 14859 log.cc:1196] T 3658dd055dba4012b05196ed90e44d45 P 61a9f79165b8438299946a8f4b034178: Deleting WAL directory at /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786336863388938-23197-0/minicluster-data/ts-1/wal/wals/3658dd055dba4012b05196ed90e44d45
I20260810 04:43:44.630031 14857 ts_tablet_manager.cc:1939] T 3658dd055dba4012b05196ed90e44d45 P 3a46d62cc08a49938add205cef6d55aa: Deleting consensus metadata
I20260810 04:43:44.630383 14859 ts_tablet_manager.cc:1939] T 3658dd055dba4012b05196ed90e44d45 P 61a9f79165b8438299946a8f4b034178: Deleting consensus metadata
I20260810 04:43:44.630622 14017 catalog_manager.cc:4841] TS 3a46d62cc08a49938add205cef6d55aa (127.22.167.67:39865): tablet 3658dd055dba4012b05196ed90e44d45 (table test-table [id=5029120d5b634e148b09c0df13a52d78]) successfully deleted
I20260810 04:43:44.630842 14011 catalog_manager.cc:4841] TS 61a9f79165b8438299946a8f4b034178 (127.22.167.66:38979): tablet 3658dd055dba4012b05196ed90e44d45 (table test-table [id=5029120d5b634e148b09c0df13a52d78]) successfully deleted
W20260810 04:43:44.634598 14015 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.22.167.65:44335 (error 108)
I20260810 04:43:44.634716 23197 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskemv_AM/build/release/bin/kudu with pid 14355
I20260810 04:43:44.636353 14810 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:43:44.634956 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.65:44335 (local address 127.0.0.1:35826)
0810 04:43:44.636278 (+ 1322us) negotiation.cc:107] Waiting for socket to connect
0810 04:43:44.636318 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.167.65:44335: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1303}
W20260810 04:43:44.638070 14015 catalog_manager.cc:4584] TS ed99582b3b574c00991af0deb19f27c1 (127.22.167.65:44335): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 3658dd055dba4012b05196ed90e44d45: Network error: Client connection negotiation failed: client connection to 127.22.167.65:44335: connect: Connection refused (error 111)
I20260810 04:43:44.642920 23197 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskemv_AM/build/release/bin/kudu with pid 14581
I20260810 04:43:44.652701 23197 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskemv_AM/build/release/bin/kudu with pid 13977
I20260810 04:43:44.664244 23197 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskemv_AM/build/release/bin/kudu
/tmp/dist-test-taskemv_AM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786336863388938-23197-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786336863388938-23197-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786336863388938-23197-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786336863388938-23197-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.167.126:46073
--webserver_interface=127.22.167.126
--webserver_port=35997
--keytab_file=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786336863388938-23197-0/krb5kdc/kudu_127.22.167.126.keytab
--principal=kudu/127.22.167.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.167.84:34205
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.167.126:46073
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--ipki_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786336863388938-23197-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786336863388938-23197-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786336863388938-23197-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 04:43:44.745944 14862 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 04:43:44.746152 14862 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 04:43:44.746186 14862 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 04:43:44.747277 14862 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 04:43:44.747310 14862 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 04:43:44.747323 14862 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 04:43:44.747334 14862 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 04:43:44.747344 14862 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 04:43:44.748703 14862 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.167.84:34205
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786336863388938-23197-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786336863388938-23197-0/minicluster-data/master-0/wal
--ipki_private_key_password_cmd=echo foo
--ipki_ca_key_size=768
--master_addresses=127.22.167.126:46073
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786336863388938-23197-0/krb5kdc/kudu_127.22.167.126.keytab
--principal=kudu/127.22.167.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.167.126:46073
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786336863388938-23197-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.167.126
--webserver_port=35997
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14862
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786336863388938-23197-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 03:54:01 UTC on ccycloud.quasar-icijmt.root.comops.site
build id 4577009
I20260810 04:43:44.748880 14862 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 04:43:44.749033 14862 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 04:43:44.758023 14877 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 04:43:44.758035 14881 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 04:43:44.758049 14884 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 04:43:44.758163 14862 server_base.cc:1034] running on GCE node
I20260810 04:43:44.758587 14862 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 04:43:44.758883 14862 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 04:43:44.760018 14862 hybrid_clock.cc:648] HybridClock initialized: now 1786337024760009 us; error 33 us; skew 500 ppm
Aug 10 04:43:44 dist-test-slave-bqcl krb5kdc[13951](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786337024, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.167.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 04:43:44.762425 14862 init.cc:377] Logged in from keytab as kudu/127.22.167.126@KRBTEST.COM (short username kudu)
I20260810 04:43:44.762877 14862 webserver.cc:460] Webserver started at http://127.22.167.126:35997/ using document root <none> and password file <none>
I20260810 04:43:44.763082 14862 fs_manager.cc:359] Metadata directory not provided
I20260810 04:43:44.763144 14862 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 04:43:44.764407 14862 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260810 04:43:44.764999 14895 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 04:43:44.765142 14862 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260810 04:43:44.765229 14862 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786336863388938-23197-0/minicluster-data/master-0/data,/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786336863388938-23197-0/minicluster-data/master-0/wal
uuid: "1623678ec1f5413687149d71bb0bce92"
format_stamp: "Formatted at 2026-08-10 04:43:43 on dist-test-slave-bqcl"
server_key: "3bba34883e284cb5c91d59bf7a18c494"
server_key_iv: "1ae4bae684dc57bb70292d045cdf211c"
server_key_version: "encryptionkey@0"
I20260810 04:43:44.765586 14862 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786336863388938-23197-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786336863388938-23197-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786336863388938-23197-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 04:43:44.779376 14862 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 04:43:44.781040 14862 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 04:43:44.781167 14862 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 04:43:44.785023 14862 rpc_server.cc:307] RPC server started. Bound to: 127.22.167.126:46073
I20260810 04:43:44.785066 14984 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.167.126:46073 every 8 connection(s)
I20260810 04:43:44.785432 14862 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786336863388938-23197-0/minicluster-data/master-0/data/info.pb
I20260810 04:43:44.787012 14985 sys_catalog.cc:263] Verifying existing consensus state
I20260810 04:43:44.787611 14985 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1623678ec1f5413687149d71bb0bce92: Bootstrap starting.
I20260810 04:43:44.789204 23197 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskemv_AM/build/release/bin/kudu as pid 14862
I20260810 04:43:44.789498 23197 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskemv_AM/build/release/bin/kudu
/tmp/dist-test-taskemv_AM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786336863388938-23197-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786336863388938-23197-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786336863388938-23197-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786336863388938-23197-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.167.65:44335
--local_ip_for_outbound_sockets=127.22.167.65
--tserver_master_addrs=127.22.167.126:46073
--webserver_port=42653
--webserver_interface=127.22.167.65
--keytab_file=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786336863388938-23197-0/krb5kdc/kudu_127.22.167.65.keytab
--principal=kudu/127.22.167.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.167.84:34205
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786336863388938-23197-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786336863388938-23197-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786336863388938-23197-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 04:43:44.796777 14985 log.cc:824] T 00000000000000000000000000000000 P 1623678ec1f5413687149d71bb0bce92: Log is configured to *not* fsync() on all Append() calls
I20260810 04:43:44.798642 14985 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1623678ec1f5413687149d71bb0bce92: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260810 04:43:44.798892 14985 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1623678ec1f5413687149d71bb0bce92: Bootstrap complete.
I20260810 04:43:44.800973 14985 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1623678ec1f5413687149d71bb0bce92 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1623678ec1f5413687149d71bb0bce92" member_type: VOTER last_known_addr { host: "127.22.167.126" port: 46073 } }
I20260810 04:43:44.801194 14985 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1623678ec1f5413687149d71bb0bce92 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1623678ec1f5413687149d71bb0bce92, State: Initialized, Role: FOLLOWER
I20260810 04:43:44.801328 14985 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1623678ec1f5413687149d71bb0bce92 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1623678ec1f5413687149d71bb0bce92" member_type: VOTER last_known_addr { host: "127.22.167.126" port: 46073 } }
I20260810 04:43:44.801386 14985 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1623678ec1f5413687149d71bb0bce92 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 04:43:44.801417 14985 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1623678ec1f5413687149d71bb0bce92 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 04:43:44.801463 14985 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1623678ec1f5413687149d71bb0bce92 [term 1 FOLLOWER]: Advancing to term 2
I20260810 04:43:44.802445 14985 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1623678ec1f5413687149d71bb0bce92 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1623678ec1f5413687149d71bb0bce92" member_type: VOTER last_known_addr { host: "127.22.167.126" port: 46073 } }
I20260810 04:43:44.802532 14985 leader_election.cc:302] T 00000000000000000000000000000000 P 1623678ec1f5413687149d71bb0bce92 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1623678ec1f5413687149d71bb0bce92; no voters:
I20260810 04:43:44.802678 14985 leader_election.cc:288] T 00000000000000000000000000000000 P 1623678ec1f5413687149d71bb0bce92 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260810 04:43:44.802721 14992 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1623678ec1f5413687149d71bb0bce92 [term 2 FOLLOWER]: Leader election won for term 2
I20260810 04:43:44.802882 14992 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1623678ec1f5413687149d71bb0bce92 [term 2 LEADER]: Becoming Leader. State: Replica: 1623678ec1f5413687149d71bb0bce92, State: Running, Role: LEADER
I20260810 04:43:44.802933 14985 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1623678ec1f5413687149d71bb0bce92 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 04:43:44.803076 14992 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1623678ec1f5413687149d71bb0bce92 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1623678ec1f5413687149d71bb0bce92" member_type: VOTER last_known_addr { host: "127.22.167.126" port: 46073 } }
I20260810 04:43:44.803342 14992 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1623678ec1f5413687149d71bb0bce92 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1623678ec1f5413687149d71bb0bce92. Latest consensus state: current_term: 2 leader_uuid: "1623678ec1f5413687149d71bb0bce92" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1623678ec1f5413687149d71bb0bce92" member_type: VOTER last_known_addr { host: "127.22.167.126" port: 46073 } } }
I20260810 04:43:44.803506 14995 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1623678ec1f5413687149d71bb0bce92 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "1623678ec1f5413687149d71bb0bce92" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1623678ec1f5413687149d71bb0bce92" member_type: VOTER last_known_addr { host: "127.22.167.126" port: 46073 } } }
I20260810 04:43:44.803658 14995 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1623678ec1f5413687149d71bb0bce92 [sys.catalog]: This master's current role is: LEADER
I20260810 04:43:44.803728 14992 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1623678ec1f5413687149d71bb0bce92 [sys.catalog]: This master's current role is: LEADER
I20260810 04:43:44.804050 15003 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 04:43:44.804792 15003 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 04:43:44.804893 15003 catalog_manager.cc:1259] Loaded cluster ID: 83957b148ce04e079ff7496e9b48b4fa
I20260810 04:43:44.804924 15003 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 04:43:44.805186 15003 catalog_manager.cc:1314] IPKI private key was stored in cleartext, attempting to encrypt it.
I20260810 04:43:44.809818 15003 catalog_manager.cc:1324] IPKI private key successfully encrypted.
I20260810 04:43:44.812089 15003 catalog_manager.cc:1504] Loading token signing keys...
I20260810 04:43:44.812273 15003 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 1623678ec1f5413687149d71bb0bce92: Loaded TSK: 0
I20260810 04:43:44.812507 15003 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260810 04:43:44.871423 14988 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 04:43:44.871614 14988 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 04:43:44.871645 14988 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 04:43:44.872740 14988 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 04:43:44.872781 14988 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 04:43:44.872815 14988 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.167.65
I20260810 04:43:44.874250 14988 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.167.84:34205
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786336863388938-23197-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786336863388938-23197-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786336863388938-23197-0/krb5kdc/kudu_127.22.167.65.keytab
--principal=kudu/127.22.167.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.167.65:44335
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786336863388938-23197-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.22.167.65
--webserver_port=42653
--tserver_master_addrs=127.22.167.126:46073
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14988
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.167.65
--log_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786336863388938-23197-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 03:54:01 UTC on ccycloud.quasar-icijmt.root.comops.site
build id 4577009
I20260810 04:43:44.874478 14988 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 04:43:44.874701 14988 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 04:43:44.879451 15041 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 04:43:44.879527 15043 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 04:43:44.879509 15040 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 04:43:44.879915 14988 server_base.cc:1034] running on GCE node
I20260810 04:43:44.880084 14988 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 04:43:44.880301 14988 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 04:43:44.881449 14988 hybrid_clock.cc:648] HybridClock initialized: now 1786337024881437 us; error 35 us; skew 500 ppm
Aug 10 04:43:44 dist-test-slave-bqcl krb5kdc[13951](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786337024, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.167.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 04:43:44.888284 14988 init.cc:377] Logged in from keytab as kudu/127.22.167.65@KRBTEST.COM (short username kudu)
I20260810 04:43:44.888713 14988 webserver.cc:460] Webserver started at http://127.22.167.65:42653/ using document root <none> and password file <none>
I20260810 04:43:44.888926 14988 fs_manager.cc:359] Metadata directory not provided
I20260810 04:43:44.888988 14988 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 04:43:44.890236 14988 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260810 04:43:44.890870 15056 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 04:43:44.891036 14988 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260810 04:43:44.891119 14988 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786336863388938-23197-0/minicluster-data/ts-0/data,/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786336863388938-23197-0/minicluster-data/ts-0/wal
uuid: "ed99582b3b574c00991af0deb19f27c1"
format_stamp: "Formatted at 2026-08-10 04:43:44 on dist-test-slave-bqcl"
server_key: "a79c210c6c8adf1172244207a1f9d808"
server_key_iv: "9e7a8eee06fb0a5364374b62d61ff33b"
server_key_version: "encryptionkey@0"
I20260810 04:43:44.891420 14988 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786336863388938-23197-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786336863388938-23197-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786336863388938-23197-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 04:43:44.917687 14988 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 04:43:44.919827 14988 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 04:43:44.919972 14988 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 04:43:44.920192 14988 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 04:43:44.920655 15066 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260810 04:43:44.922623 14988 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260810 04:43:44.922672 14988 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260810 04:43:44.922706 14988 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260810 04:43:44.923319 14988 ts_tablet_manager.cc:610] Registered 1 tablets
I20260810 04:43:44.923352 14988 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260810 04:43:44.923446 15066 tablet_bootstrap.cc:492] T 3658dd055dba4012b05196ed90e44d45 P ed99582b3b574c00991af0deb19f27c1: Bootstrap starting.
I20260810 04:43:44.929972 14988 rpc_server.cc:307] RPC server started. Bound to: 127.22.167.65:44335
I20260810 04:43:44.930086 15262 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.167.65:44335 every 8 connection(s)
I20260810 04:43:44.930433 14988 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786336863388938-23197-0/minicluster-data/ts-0/data/info.pb
I20260810 04:43:44.930589 23197 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskemv_AM/build/release/bin/kudu as pid 14988
I20260810 04:43:44.930866 23197 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskemv_AM/build/release/bin/kudu
/tmp/dist-test-taskemv_AM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786336863388938-23197-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786336863388938-23197-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786336863388938-23197-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786336863388938-23197-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.167.66:38979
--local_ip_for_outbound_sockets=127.22.167.66
--tserver_master_addrs=127.22.167.126:46073
--webserver_port=32775
--webserver_interface=127.22.167.66
--keytab_file=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786336863388938-23197-0/krb5kdc/kudu_127.22.167.66.keytab
--principal=kudu/127.22.167.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.167.84:34205
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786336863388938-23197-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786336863388938-23197-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786336863388938-23197-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 04:43:44.934509 15066 log.cc:824] T 3658dd055dba4012b05196ed90e44d45 P ed99582b3b574c00991af0deb19f27c1: Log is configured to *not* fsync() on all Append() calls
I20260810 04:43:44.936949 15066 tablet_bootstrap.cc:492] T 3658dd055dba4012b05196ed90e44d45 P ed99582b3b574c00991af0deb19f27c1: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260810 04:43:44.937211 15066 tablet_bootstrap.cc:492] T 3658dd055dba4012b05196ed90e44d45 P ed99582b3b574c00991af0deb19f27c1: Bootstrap complete.
I20260810 04:43:44.937994 15066 ts_tablet_manager.cc:1397] T 3658dd055dba4012b05196ed90e44d45 P ed99582b3b574c00991af0deb19f27c1: Time spent bootstrapping tablet: real 0.015s user 0.004s sys 0.007s
I20260810 04:43:44.939257 15066 raft_consensus.cc:348] T 3658dd055dba4012b05196ed90e44d45 P ed99582b3b574c00991af0deb19f27c1 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "61a9f79165b8438299946a8f4b034178" member_type: VOTER last_known_addr { host: "127.22.167.66" port: 38979 } } peers { permanent_uuid: "ed99582b3b574c00991af0deb19f27c1" member_type: VOTER last_known_addr { host: "127.22.167.65" port: 44335 } } peers { permanent_uuid: "3a46d62cc08a49938add205cef6d55aa" member_type: VOTER last_known_addr { host: "127.22.167.67" port: 39865 } }
I20260810 04:43:44.939520 15066 raft_consensus.cc:729] T 3658dd055dba4012b05196ed90e44d45 P ed99582b3b574c00991af0deb19f27c1 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: ed99582b3b574c00991af0deb19f27c1, State: Initialized, Role: FOLLOWER
I20260810 04:43:44.939706 15066 consensus_queue.cc:260] T 3658dd055dba4012b05196ed90e44d45 P ed99582b3b574c00991af0deb19f27c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "61a9f79165b8438299946a8f4b034178" member_type: VOTER last_known_addr { host: "127.22.167.66" port: 38979 } } peers { permanent_uuid: "ed99582b3b574c00991af0deb19f27c1" member_type: VOTER last_known_addr { host: "127.22.167.65" port: 44335 } } peers { permanent_uuid: "3a46d62cc08a49938add205cef6d55aa" member_type: VOTER last_known_addr { host: "127.22.167.67" port: 39865 } }
I20260810 04:43:44.940035 15066 ts_tablet_manager.cc:1428] T 3658dd055dba4012b05196ed90e44d45 P ed99582b3b574c00991af0deb19f27c1: Time spent starting tablet: real 0.002s user 0.004s sys 0.001s
Aug 10 04:43:44 dist-test-slave-bqcl krb5kdc[13951](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786337024, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.167.65@KRBTEST.COM for kudu/127.22.167.126@KRBTEST.COM
I20260810 04:43:44.944587 15270 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:43:44.933174 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.167.65:39249 (local address 127.22.167.126:46073)
0810 04:43:44.933338 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0810 04:43:44.933348 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 04:43:44.933674 (+ 326us) server_negotiation.cc:407] Connection header received
0810 04:43:44.934340 (+ 666us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:43:44.934349 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:43:44.934415 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:43:44.934574 (+ 159us) server_negotiation.cc:226] Negotiated authn=SASL
0810 04:43:44.935406 (+ 832us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:43:44.936739 (+ 1333us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:43:44.938284 (+ 1545us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:43:44.938878 (+ 594us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:43:44.938989 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:43:44.941603 (+ 2614us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 04:43:44.941607 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 04:43:44.941607 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 04:43:44.941617 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0810 04:43:44.943101 (+ 1484us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 04:43:44.943526 (+ 425us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 04:43:44.943528 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 04:43:44.943529 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 04:43:44.943611 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 04:43:44.943840 (+ 229us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 04:43:44.943840 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 04:43:44.943840 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 04:43:44.944028 (+ 188us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 04:43:44.944086 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0810 04:43:44.944359 (+ 273us) server_negotiation.cc:299] Negotiation successful
0810 04:43:44.944454 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":115,"thread_start_us":91,"threads_started":1}
I20260810 04:43:44.944679 15269 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:43:44.933095 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.126:46073 (local address 127.22.167.65:39249)
0810 04:43:44.933468 (+ 373us) negotiation.cc:107] Waiting for socket to connect
0810 04:43:44.933512 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0810 04:43:44.934091 (+ 579us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 04:43:44.934620 (+ 529us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 04:43:44.934624 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 04:43:44.934965 (+ 341us) client_negotiation.cc:189] Negotiated authn=SASL
0810 04:43:44.935268 (+ 303us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:43:44.935275 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:43:44.936862 (+ 1587us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:43:44.936863 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:43:44.938181 (+ 1318us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:43:44.938185 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:43:44.938993 (+ 808us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:43:44.938994 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:43:44.939150 (+ 156us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:43:44.939778 (+ 628us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 04:43:44.939787 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0810 04:43:44.941505 (+ 1718us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 04:43:44.943172 (+ 1667us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 04:43:44.943179 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 04:43:44.943181 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 04:43:44.943458 (+ 277us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 04:43:44.943663 (+ 205us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 04:43:44.943664 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 04:43:44.943664 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 04:43:44.943786 (+ 122us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 04:43:44.944080 (+ 294us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 04:43:44.944084 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 04:43:44.944223 (+ 139us) client_negotiation.cc:769] Sending connection context
0810 04:43:44.944324 (+ 101us) client_negotiation.cc:240] Negotiation successful
0810 04:43:44.944489 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":331,"thread_start_us":93,"threads_started":1}
I20260810 04:43:44.945323 15265 heartbeater.cc:347] Connected to a master server at 127.22.167.126:46073
I20260810 04:43:44.945418 15265 heartbeater.cc:464] Registering TS with master...
I20260810 04:43:44.945582 15265 heartbeater.cc:511] Master 127.22.167.126:46073 requested a full tablet report, sending...
I20260810 04:43:44.946314 14926 ts_manager.cc:194] Registered new tserver with Master: ed99582b3b574c00991af0deb19f27c1 (127.22.167.65:44335)
I20260810 04:43:44.947187 14926 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.167.65@KRBTEST.COM'} at 127.22.167.65:39249
I20260810 04:43:44.948056 15265 heartbeater.cc:503] Master 127.22.167.126:46073 was elected leader, sending a full tablet report...
Aug 10 04:43:44 dist-test-slave-bqcl krb5kdc[13951](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786337024, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.167.126@KRBTEST.COM for kudu/127.22.167.65@KRBTEST.COM
I20260810 04:43:44.954317 15276 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:43:44.947416 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35842 (local address 127.22.167.65:44335)
0810 04:43:44.947555 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0810 04:43:44.947559 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 04:43:44.947569 (+ 10us) server_negotiation.cc:407] Connection header received
0810 04:43:44.947597 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:43:44.947598 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:43:44.947609 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:43:44.947677 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0810 04:43:44.948052 (+ 375us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:43:44.948875 (+ 823us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:43:44.949572 (+ 697us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:43:44.949855 (+ 283us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:43:44.949912 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:43:44.951904 (+ 1992us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 04:43:44.951912 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 04:43:44.951912 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 04:43:44.951918 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0810 04:43:44.953214 (+ 1296us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 04:43:44.953605 (+ 391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 04:43:44.953607 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 04:43:44.953607 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 04:43:44.953654 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 04:43:44.953826 (+ 172us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 04:43:44.953827 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 04:43:44.953827 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 04:43:44.953937 (+ 110us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 04:43:44.953991 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0810 04:43:44.954190 (+ 199us) server_negotiation.cc:299] Negotiation successful
0810 04:43:44.954260 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":111,"thread_start_us":83,"threads_started":1}
I20260810 04:43:44.954356 15275 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:43:44.946805 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.65:44335 (local address 127.0.0.1:35842)
0810 04:43:44.946971 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0810 04:43:44.947005 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0810 04:43:44.947064 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 04:43:44.947691 (+ 627us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 04:43:44.947697 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 04:43:44.947939 (+ 242us) client_negotiation.cc:189] Negotiated authn=SASL
0810 04:43:44.947979 (+ 40us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:43:44.947981 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:43:44.948947 (+ 966us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:43:44.948948 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:43:44.949497 (+ 549us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:43:44.949499 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:43:44.949923 (+ 424us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:43:44.949924 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:43:44.950008 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:43:44.950456 (+ 448us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 04:43:44.950461 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0810 04:43:44.951829 (+ 1368us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 04:43:44.953290 (+ 1461us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 04:43:44.953297 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 04:43:44.953299 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 04:43:44.953537 (+ 238us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 04:43:44.953713 (+ 176us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 04:43:44.953713 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 04:43:44.953714 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 04:43:44.953777 (+ 63us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 04:43:44.953989 (+ 212us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 04:43:44.953994 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 04:43:44.954050 (+ 56us) client_negotiation.cc:769] Sending connection context
0810 04:43:44.954158 (+ 108us) client_negotiation.cc:240] Negotiation successful
0810 04:43:44.954266 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":85,"threads_started":1}
I20260810 04:43:44.954769 15144 tablet_service.cc:1514] Processing DeleteTablet for tablet 3658dd055dba4012b05196ed90e44d45 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 04:43:44 UTC) from {username='kudu', principal='kudu/127.22.167.126@KRBTEST.COM'} at 127.0.0.1:35842
I20260810 04:43:44.955217 15277 tablet_replica.cc:330] stopping tablet replica
I20260810 04:43:44.955307 15277 raft_consensus.cc:2229] T 3658dd055dba4012b05196ed90e44d45 P ed99582b3b574c00991af0deb19f27c1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 04:43:44.955369 15277 raft_consensus.cc:2258] T 3658dd055dba4012b05196ed90e44d45 P ed99582b3b574c00991af0deb19f27c1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 04:43:44.955638 15277 ts_tablet_manager.cc:1905] T 3658dd055dba4012b05196ed90e44d45 P ed99582b3b574c00991af0deb19f27c1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 04:43:44.957279 15277 ts_tablet_manager.cc:1918] T 3658dd055dba4012b05196ed90e44d45 P ed99582b3b574c00991af0deb19f27c1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 04:43:44.957345 15277 log.cc:1196] T 3658dd055dba4012b05196ed90e44d45 P ed99582b3b574c00991af0deb19f27c1: Deleting WAL directory at /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786336863388938-23197-0/minicluster-data/ts-0/wal/wals/3658dd055dba4012b05196ed90e44d45
I20260810 04:43:44.957538 15277 ts_tablet_manager.cc:1939] T 3658dd055dba4012b05196ed90e44d45 P ed99582b3b574c00991af0deb19f27c1: Deleting consensus metadata
I20260810 04:43:44.957962 14901 catalog_manager.cc:4841] TS ed99582b3b574c00991af0deb19f27c1 (127.22.167.65:44335): tablet 3658dd055dba4012b05196ed90e44d45 (table test-table [id=5029120d5b634e148b09c0df13a52d78]) successfully deleted
W20260810 04:43:45.015367 15267 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 04:43:45.015596 15267 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 04:43:45.015630 15267 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 04:43:45.016750 15267 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 04:43:45.016788 15267 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 04:43:45.016821 15267 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.167.66
I20260810 04:43:45.018222 15267 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.167.84:34205
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786336863388938-23197-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786336863388938-23197-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786336863388938-23197-0/krb5kdc/kudu_127.22.167.66.keytab
--principal=kudu/127.22.167.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.167.66:38979
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786336863388938-23197-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.22.167.66
--webserver_port=32775
--tserver_master_addrs=127.22.167.126:46073
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15267
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.167.66
--log_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786336863388938-23197-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 03:54:01 UTC on ccycloud.quasar-icijmt.root.comops.site
build id 4577009
I20260810 04:43:45.018433 15267 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 04:43:45.018663 15267 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 04:43:45.023305 15296 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 04:43:45.023324 15295 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 04:43:45.023412 15267 server_base.cc:1034] running on GCE node
W20260810 04:43:45.023348 15304 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 04:43:45.023725 15267 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 04:43:45.023962 15267 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 04:43:45.025103 15267 hybrid_clock.cc:648] HybridClock initialized: now 1786337025025088 us; error 27 us; skew 500 ppm
Aug 10 04:43:45 dist-test-slave-bqcl krb5kdc[13951](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786337025, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.167.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 04:43:45.027441 15267 init.cc:377] Logged in from keytab as kudu/127.22.167.66@KRBTEST.COM (short username kudu)
I20260810 04:43:45.027855 15267 webserver.cc:460] Webserver started at http://127.22.167.66:32775/ using document root <none> and password file <none>
I20260810 04:43:45.028045 15267 fs_manager.cc:359] Metadata directory not provided
I20260810 04:43:45.028103 15267 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 04:43:45.029302 15267 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260810 04:43:45.029794 15315 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 04:43:45.029943 15267 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260810 04:43:45.030045 15267 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786336863388938-23197-0/minicluster-data/ts-1/data,/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786336863388938-23197-0/minicluster-data/ts-1/wal
uuid: "61a9f79165b8438299946a8f4b034178"
format_stamp: "Formatted at 2026-08-10 04:43:44 on dist-test-slave-bqcl"
server_key: "91d51958193417a18cd51e9307e9fafc"
server_key_iv: "aa572f67f2d7ed0a2d36cdad795e0015"
server_key_version: "encryptionkey@0"
I20260810 04:43:45.030376 15267 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786336863388938-23197-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786336863388938-23197-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786336863388938-23197-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 04:43:45.050288 15267 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 04:43:45.051954 15267 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 04:43:45.052093 15267 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 04:43:45.052287 15267 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 04:43:45.052567 15267 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 04:43:45.052599 15267 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 04:43:45.052626 15267 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 04:43:45.052650 15267 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 04:43:45.058056 15267 rpc_server.cc:307] RPC server started. Bound to: 127.22.167.66:38979
I20260810 04:43:45.058096 15514 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.167.66:38979 every 8 connection(s)
I20260810 04:43:45.058423 15267 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786336863388938-23197-0/minicluster-data/ts-1/data/info.pb
Aug 10 04:43:45 dist-test-slave-bqcl krb5kdc[13951](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786337025, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.167.66@KRBTEST.COM for kudu/127.22.167.126@KRBTEST.COM
I20260810 04:43:45.066367 15270 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:43:45.059066 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.167.66:58617 (local address 127.22.167.126:46073)
0810 04:43:45.059142 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0810 04:43:45.059144 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 04:43:45.059289 (+ 145us) server_negotiation.cc:407] Connection header received
0810 04:43:45.059727 (+ 438us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:43:45.059727 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:43:45.059737 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:43:45.059781 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0810 04:43:45.060624 (+ 843us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:43:45.061112 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:43:45.062004 (+ 892us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:43:45.062238 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:43:45.062286 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:43:45.064088 (+ 1802us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 04:43:45.064089 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 04:43:45.064089 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 04:43:45.064095 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0810 04:43:45.065359 (+ 1264us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 04:43:45.065673 (+ 314us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 04:43:45.065674 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 04:43:45.065674 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 04:43:45.065721 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 04:43:45.065887 (+ 166us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 04:43:45.065887 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 04:43:45.065888 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 04:43:45.065976 (+ 88us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 04:43:45.066019 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0810 04:43:45.066238 (+ 219us) server_negotiation.cc:299] Negotiation successful
0810 04:43:45.066303 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260810 04:43:45.066517 15519 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:43:45.059035 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.126:46073 (local address 127.22.167.66:58617)
0810 04:43:45.059182 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0810 04:43:45.059217 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0810 04:43:45.059656 (+ 439us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 04:43:45.059841 (+ 185us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 04:43:45.059844 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 04:43:45.060378 (+ 534us) client_negotiation.cc:189] Negotiated authn=SASL
0810 04:43:45.060559 (+ 181us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:43:45.060563 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:43:45.061171 (+ 608us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:43:45.061172 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:43:45.061947 (+ 775us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:43:45.061948 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:43:45.062290 (+ 342us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:43:45.062290 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:43:45.062385 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:43:45.062751 (+ 366us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 04:43:45.062759 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0810 04:43:45.063996 (+ 1237us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 04:43:45.065421 (+ 1425us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 04:43:45.065424 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 04:43:45.065425 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 04:43:45.065615 (+ 190us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 04:43:45.065764 (+ 149us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 04:43:45.065764 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 04:43:45.065765 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 04:43:45.065848 (+ 83us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 04:43:45.066023 (+ 175us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 04:43:45.066026 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 04:43:45.066150 (+ 124us) client_negotiation.cc:769] Sending connection context
0810 04:43:45.066224 (+ 74us) client_negotiation.cc:240] Negotiation successful
0810 04:43:45.066331 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":88,"threads_started":1}
I20260810 04:43:45.066953 15516 heartbeater.cc:347] Connected to a master server at 127.22.167.126:46073
I20260810 04:43:45.067036 15516 heartbeater.cc:464] Registering TS with master...
I20260810 04:43:45.067034 23197 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskemv_AM/build/release/bin/kudu as pid 15267
I20260810 04:43:45.067200 15516 heartbeater.cc:511] Master 127.22.167.126:46073 requested a full tablet report, sending...
I20260810 04:43:45.067284 23197 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskemv_AM/build/release/bin/kudu
/tmp/dist-test-taskemv_AM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786336863388938-23197-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786336863388938-23197-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786336863388938-23197-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786336863388938-23197-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.167.67:39865
--local_ip_for_outbound_sockets=127.22.167.67
--tserver_master_addrs=127.22.167.126:46073
--webserver_port=35631
--webserver_interface=127.22.167.67
--keytab_file=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786336863388938-23197-0/krb5kdc/kudu_127.22.167.67.keytab
--principal=kudu/127.22.167.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.167.84:34205
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786336863388938-23197-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786336863388938-23197-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786336863388938-23197-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 04:43:45.067543 14926 ts_manager.cc:194] Registered new tserver with Master: 61a9f79165b8438299946a8f4b034178 (127.22.167.66:38979)
I20260810 04:43:45.067868 14926 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.167.66@KRBTEST.COM'} at 127.22.167.66:58617
W20260810 04:43:45.147320 15521 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 04:43:45.147508 15521 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 04:43:45.147531 15521 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 04:43:45.148588 15521 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 04:43:45.148622 15521 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 04:43:45.148645 15521 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.167.67
I20260810 04:43:45.150060 15521 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.167.84:34205
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786336863388938-23197-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786336863388938-23197-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786336863388938-23197-0/krb5kdc/kudu_127.22.167.67.keytab
--principal=kudu/127.22.167.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.167.67:39865
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786336863388938-23197-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.22.167.67
--webserver_port=35631
--tserver_master_addrs=127.22.167.126:46073
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15521
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.167.67
--log_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786336863388938-23197-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 03:54:01 UTC on ccycloud.quasar-icijmt.root.comops.site
build id 4577009
I20260810 04:43:45.150259 15521 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 04:43:45.150465 15521 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 04:43:45.155339 15529 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 04:43:45.155357 15527 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 04:43:45.155463 15521 server_base.cc:1034] running on GCE node
W20260810 04:43:45.155582 15532 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 04:43:45.155786 15521 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 04:43:45.156018 15521 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 04:43:45.157166 15521 hybrid_clock.cc:648] HybridClock initialized: now 1786337025157147 us; error 31 us; skew 500 ppm
Aug 10 04:43:45 dist-test-slave-bqcl krb5kdc[13951](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786337025, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.167.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 04:43:45.159425 15521 init.cc:377] Logged in from keytab as kudu/127.22.167.67@KRBTEST.COM (short username kudu)
I20260810 04:43:45.159808 15521 webserver.cc:460] Webserver started at http://127.22.167.67:35631/ using document root <none> and password file <none>
I20260810 04:43:45.160010 15521 fs_manager.cc:359] Metadata directory not provided
I20260810 04:43:45.160082 15521 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 04:43:45.161306 15521 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260810 04:43:45.161834 15541 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 04:43:45.161976 15521 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260810 04:43:45.162070 15521 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786336863388938-23197-0/minicluster-data/ts-2/data,/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786336863388938-23197-0/minicluster-data/ts-2/wal
uuid: "3a46d62cc08a49938add205cef6d55aa"
format_stamp: "Formatted at 2026-08-10 04:43:44 on dist-test-slave-bqcl"
server_key: "f5fb01f1cfad0883040e79fcf3eb13d1"
server_key_iv: "8a368d70585170c801565df9a6544025"
server_key_version: "encryptionkey@0"
I20260810 04:43:45.162442 15521 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786336863388938-23197-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786336863388938-23197-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786336863388938-23197-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 04:43:45.177049 15521 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 04:43:45.178699 15521 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 04:43:45.178826 15521 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 04:43:45.179033 15521 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 04:43:45.179306 15521 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 04:43:45.179339 15521 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 04:43:45.179368 15521 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 04:43:45.179385 15521 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 04:43:45.185130 15521 rpc_server.cc:307] RPC server started. Bound to: 127.22.167.67:39865
I20260810 04:43:45.185199 15736 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.167.67:39865 every 8 connection(s)
I20260810 04:43:45.185491 15521 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786336863388938-23197-0/minicluster-data/ts-2/data/info.pb
Aug 10 04:43:45 dist-test-slave-bqcl krb5kdc[13951](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786337025, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.167.67@KRBTEST.COM for kudu/127.22.167.126@KRBTEST.COM
I20260810 04:43:45.192760 23197 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskemv_AM/build/release/bin/kudu as pid 15521
I20260810 04:43:45.194514 15270 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:43:45.186181 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.167.67:35837 (local address 127.22.167.126:46073)
0810 04:43:45.186229 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0810 04:43:45.186231 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 04:43:45.186434 (+ 203us) server_negotiation.cc:407] Connection header received
0810 04:43:45.186928 (+ 494us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:43:45.186929 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:43:45.186942 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:43:45.186993 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0810 04:43:45.187898 (+ 905us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:43:45.188378 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:43:45.189381 (+ 1003us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:43:45.189571 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:43:45.189626 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:43:45.191860 (+ 2234us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 04:43:45.191862 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 04:43:45.191862 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 04:43:45.191870 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0810 04:43:45.193219 (+ 1349us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 04:43:45.193566 (+ 347us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 04:43:45.193567 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 04:43:45.193567 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 04:43:45.193626 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 04:43:45.193844 (+ 218us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 04:43:45.193845 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 04:43:45.193845 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 04:43:45.193955 (+ 110us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 04:43:45.194011 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0810 04:43:45.194341 (+ 330us) server_negotiation.cc:299] Negotiation successful
0810 04:43:45.194425 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260810 04:43:45.194664 15739 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:43:45.186165 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.126:46073 (local address 127.22.167.67:35837)
0810 04:43:45.186324 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0810 04:43:45.186358 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0810 04:43:45.186830 (+ 472us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 04:43:45.187062 (+ 232us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 04:43:45.187067 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 04:43:45.187617 (+ 550us) client_negotiation.cc:189] Negotiated authn=SASL
0810 04:43:45.187800 (+ 183us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:43:45.187805 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:43:45.188463 (+ 658us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:43:45.188464 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:43:45.189291 (+ 827us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:43:45.189294 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:43:45.189642 (+ 348us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:43:45.189643 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:43:45.189767 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:43:45.190275 (+ 508us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 04:43:45.190285 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0810 04:43:45.191775 (+ 1490us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 04:43:45.193280 (+ 1505us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 04:43:45.193288 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 04:43:45.193290 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 04:43:45.193500 (+ 210us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 04:43:45.193681 (+ 181us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 04:43:45.193681 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 04:43:45.193682 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 04:43:45.193789 (+ 107us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 04:43:45.194010 (+ 221us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 04:43:45.194017 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 04:43:45.194204 (+ 187us) client_negotiation.cc:769] Sending connection context
0810 04:43:45.194306 (+ 102us) client_negotiation.cc:240] Negotiation successful
0810 04:43:45.194455 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":83,"threads_started":1}
I20260810 04:43:45.195187 15737 heartbeater.cc:347] Connected to a master server at 127.22.167.126:46073
I20260810 04:43:45.195274 15737 heartbeater.cc:464] Registering TS with master...
I20260810 04:43:45.195441 15737 heartbeater.cc:511] Master 127.22.167.126:46073 requested a full tablet report, sending...
I20260810 04:43:45.195791 14926 ts_manager.cc:194] Registered new tserver with Master: 3a46d62cc08a49938add205cef6d55aa (127.22.167.67:39865)
I20260810 04:43:45.196205 14926 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.167.67@KRBTEST.COM'} at 127.22.167.67:35837
I20260810 04:43:45.198985 15740 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:43:45.193095 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38536 (local address 127.22.167.126:46073)
0810 04:43:45.193187 (+ 92us) server_negotiation.cc:206] Beginning negotiation
0810 04:43:45.193189 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 04:43:45.193273 (+ 84us) server_negotiation.cc:407] Connection header received
0810 04:43:45.193345 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:43:45.193346 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:43:45.193357 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:43:45.193418 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0810 04:43:45.193977 (+ 559us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:43:45.194628 (+ 651us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:43:45.195572 (+ 944us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:43:45.195780 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:43:45.195832 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:43:45.196893 (+ 1061us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 04:43:45.196894 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 04:43:45.196895 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 04:43:45.196901 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0810 04:43:45.197940 (+ 1039us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 04:43:45.198313 (+ 373us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 04:43:45.198314 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 04:43:45.198315 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 04:43:45.198369 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 04:43:45.198526 (+ 157us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 04:43:45.198526 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 04:43:45.198526 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 04:43:45.198632 (+ 106us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 04:43:45.198792 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0810 04:43:45.198828 (+ 36us) server_negotiation.cc:299] Negotiation successful
0810 04:43:45.198901 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75,"thread_start_us":45,"threads_started":1}
I20260810 04:43:45.199551 23197 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260810 04:43:45.201759 15740 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:43:45.199834 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38540 (local address 127.22.167.126:46073)
0810 04:43:45.199867 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0810 04:43:45.199869 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 04:43:45.199982 (+ 113us) server_negotiation.cc:407] Connection header received
0810 04:43:45.200020 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:43:45.200021 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:43:45.200048 (+ 27us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:43:45.200099 (+ 51us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 04:43:45.200174 (+ 75us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:43:45.200593 (+ 419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:43:45.201242 (+ 649us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:43:45.201407 (+ 165us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:43:45.201446 (+ 39us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:43:45.201555 (+ 109us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 04:43:45.201636 (+ 81us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 04:43:45.201688 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0810 04:43:45.201700 (+ 12us) server_negotiation.cc:299] Negotiation successful
0810 04:43:45.201715 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260810 04:43:45.202003 14926 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin'} at 127.0.0.1:38540:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 04:43:45.203660 14926 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 04:43:45.205590 15144 tablet_service.cc:1467] Processing CreateTablet for tablet 416edda2678e425d9c0c87ed4f56cf2a (DEFAULT_TABLE table=test-table [id=f69bc2ad255a489c9c111542f5d88ef0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 04:43:45.205796 15144 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 416edda2678e425d9c0c87ed4f56cf2a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 04:43:45.207312 15066 tablet_bootstrap.cc:492] T 416edda2678e425d9c0c87ed4f56cf2a P ed99582b3b574c00991af0deb19f27c1: Bootstrap starting.
I20260810 04:43:45.207589 15066 tablet_bootstrap.cc:654] T 416edda2678e425d9c0c87ed4f56cf2a P ed99582b3b574c00991af0deb19f27c1: Neither blocks nor log segments found. Creating new log.
I20260810 04:43:45.208518 15066 tablet_bootstrap.cc:492] T 416edda2678e425d9c0c87ed4f56cf2a P ed99582b3b574c00991af0deb19f27c1: No bootstrap required, opened a new log
I20260810 04:43:45.208562 15066 ts_tablet_manager.cc:1397] T 416edda2678e425d9c0c87ed4f56cf2a P ed99582b3b574c00991af0deb19f27c1: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260810 04:43:45.208698 15066 raft_consensus.cc:348] T 416edda2678e425d9c0c87ed4f56cf2a P ed99582b3b574c00991af0deb19f27c1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "61a9f79165b8438299946a8f4b034178" member_type: VOTER last_known_addr { host: "127.22.167.66" port: 38979 } } peers { permanent_uuid: "3a46d62cc08a49938add205cef6d55aa" member_type: VOTER last_known_addr { host: "127.22.167.67" port: 39865 } } peers { permanent_uuid: "ed99582b3b574c00991af0deb19f27c1" member_type: VOTER last_known_addr { host: "127.22.167.65" port: 44335 } }
I20260810 04:43:45.208765 15066 raft_consensus.cc:374] T 416edda2678e425d9c0c87ed4f56cf2a P ed99582b3b574c00991af0deb19f27c1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 04:43:45.208806 15066 raft_consensus.cc:729] T 416edda2678e425d9c0c87ed4f56cf2a P ed99582b3b574c00991af0deb19f27c1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ed99582b3b574c00991af0deb19f27c1, State: Initialized, Role: FOLLOWER
I20260810 04:43:45.208851 15066 consensus_queue.cc:260] T 416edda2678e425d9c0c87ed4f56cf2a P ed99582b3b574c00991af0deb19f27c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "61a9f79165b8438299946a8f4b034178" member_type: VOTER last_known_addr { host: "127.22.167.66" port: 38979 } } peers { permanent_uuid: "3a46d62cc08a49938add205cef6d55aa" member_type: VOTER last_known_addr { host: "127.22.167.67" port: 39865 } } peers { permanent_uuid: "ed99582b3b574c00991af0deb19f27c1" member_type: VOTER last_known_addr { host: "127.22.167.65" port: 44335 } }
I20260810 04:43:45.208940 15066 ts_tablet_manager.cc:1428] T 416edda2678e425d9c0c87ed4f56cf2a P ed99582b3b574c00991af0deb19f27c1: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260810 04:43:45.209154 15275 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:43:45.205304 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.66:38979 (local address 127.0.0.1:47238)
0810 04:43:45.205426 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0810 04:43:45.205436 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0810 04:43:45.205474 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 04:43:45.205786 (+ 312us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 04:43:45.205787 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 04:43:45.205789 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 04:43:45.205826 (+ 37us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:43:45.205827 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:43:45.207061 (+ 1234us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:43:45.207061 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:43:45.208041 (+ 980us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:43:45.208044 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:43:45.208931 (+ 887us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:43:45.208932 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:43:45.209004 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:43:45.209024 (+ 20us) client_negotiation.cc:769] Sending connection context
0810 04:43:45.209053 (+ 29us) client_negotiation.cc:240] Negotiation successful
0810 04:43:45.209090 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":105}
I20260810 04:43:45.209546 15745 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:43:45.205404 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47238 (local address 127.22.167.66:38979)
0810 04:43:45.205592 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0810 04:43:45.205596 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 04:43:45.205606 (+ 10us) server_negotiation.cc:407] Connection header received
0810 04:43:45.205643 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:43:45.205645 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:43:45.205659 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:43:45.205771 (+ 112us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 04:43:45.205890 (+ 119us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:43:45.206946 (+ 1056us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:43:45.208265 (+ 1319us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:43:45.208835 (+ 570us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:43:45.209321 (+ 486us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:43:45.209393 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0810 04:43:45.209421 (+ 28us) server_negotiation.cc:299] Negotiation successful
0810 04:43:45.209466 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":153,"thread_start_us":88,"threads_started":1}
I20260810 04:43:45.209790 15743 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:43:45.205318 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.67:39865 (local address 127.0.0.1:36966)
0810 04:43:45.206246 (+ 928us) negotiation.cc:107] Waiting for socket to connect
0810 04:43:45.206255 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0810 04:43:45.206307 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 04:43:45.206551 (+ 244us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 04:43:45.206552 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 04:43:45.206554 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 04:43:45.206606 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:43:45.206608 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:43:45.207639 (+ 1031us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:43:45.207640 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:43:45.208672 (+ 1032us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:43:45.208676 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:43:45.209587 (+ 911us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:43:45.209588 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:43:45.209651 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:43:45.209655 (+ 4us) client_negotiation.cc:769] Sending connection context
0810 04:43:45.209703 (+ 48us) client_negotiation.cc:240] Negotiation successful
0810 04:43:45.209742 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":911,"thread_start_us":83,"threads_started":1}
I20260810 04:43:45.210006 15747 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:43:45.205499 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36966 (local address 127.22.167.67:39865)
0810 04:43:45.205614 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0810 04:43:45.205618 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 04:43:45.206365 (+ 747us) server_negotiation.cc:407] Connection header received
0810 04:43:45.206404 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:43:45.206406 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:43:45.206420 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:43:45.206487 (+ 67us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 04:43:45.206669 (+ 182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:43:45.207530 (+ 861us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:43:45.208768 (+ 1238us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:43:45.209458 (+ 690us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:43:45.209520 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:43:45.209566 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0810 04:43:45.209886 (+ 320us) server_negotiation.cc:299] Negotiation successful
0810 04:43:45.209931 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":88,"thread_start_us":58,"threads_started":1}
I20260810 04:43:45.210213 15404 tablet_service.cc:1467] Processing CreateTablet for tablet 416edda2678e425d9c0c87ed4f56cf2a (DEFAULT_TABLE table=test-table [id=f69bc2ad255a489c9c111542f5d88ef0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 04:43:45.210573 15404 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 416edda2678e425d9c0c87ed4f56cf2a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 04:43:45.210646 15627 tablet_service.cc:1467] Processing CreateTablet for tablet 416edda2678e425d9c0c87ed4f56cf2a (DEFAULT_TABLE table=test-table [id=f69bc2ad255a489c9c111542f5d88ef0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 04:43:45.210919 15627 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 416edda2678e425d9c0c87ed4f56cf2a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 04:43:45.213140 15752 tablet_bootstrap.cc:492] T 416edda2678e425d9c0c87ed4f56cf2a P 61a9f79165b8438299946a8f4b034178: Bootstrap starting.
I20260810 04:43:45.213423 15753 tablet_bootstrap.cc:492] T 416edda2678e425d9c0c87ed4f56cf2a P 3a46d62cc08a49938add205cef6d55aa: Bootstrap starting.
I20260810 04:43:45.213845 15752 tablet_bootstrap.cc:654] T 416edda2678e425d9c0c87ed4f56cf2a P 61a9f79165b8438299946a8f4b034178: Neither blocks nor log segments found. Creating new log.
I20260810 04:43:45.214006 15753 tablet_bootstrap.cc:654] T 416edda2678e425d9c0c87ed4f56cf2a P 3a46d62cc08a49938add205cef6d55aa: Neither blocks nor log segments found. Creating new log.
I20260810 04:43:45.214412 15752 log.cc:824] T 416edda2678e425d9c0c87ed4f56cf2a P 61a9f79165b8438299946a8f4b034178: Log is configured to *not* fsync() on all Append() calls
I20260810 04:43:45.214536 15753 log.cc:824] T 416edda2678e425d9c0c87ed4f56cf2a P 3a46d62cc08a49938add205cef6d55aa: Log is configured to *not* fsync() on all Append() calls
I20260810 04:43:45.215247 15753 tablet_bootstrap.cc:492] T 416edda2678e425d9c0c87ed4f56cf2a P 3a46d62cc08a49938add205cef6d55aa: No bootstrap required, opened a new log
I20260810 04:43:45.215247 15752 tablet_bootstrap.cc:492] T 416edda2678e425d9c0c87ed4f56cf2a P 61a9f79165b8438299946a8f4b034178: No bootstrap required, opened a new log
I20260810 04:43:45.215322 15752 ts_tablet_manager.cc:1397] T 416edda2678e425d9c0c87ed4f56cf2a P 61a9f79165b8438299946a8f4b034178: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260810 04:43:45.215322 15753 ts_tablet_manager.cc:1397] T 416edda2678e425d9c0c87ed4f56cf2a P 3a46d62cc08a49938add205cef6d55aa: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260810 04:43:45.217163 15752 raft_consensus.cc:348] T 416edda2678e425d9c0c87ed4f56cf2a P 61a9f79165b8438299946a8f4b034178 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "61a9f79165b8438299946a8f4b034178" member_type: VOTER last_known_addr { host: "127.22.167.66" port: 38979 } } peers { permanent_uuid: "3a46d62cc08a49938add205cef6d55aa" member_type: VOTER last_known_addr { host: "127.22.167.67" port: 39865 } } peers { permanent_uuid: "ed99582b3b574c00991af0deb19f27c1" member_type: VOTER last_known_addr { host: "127.22.167.65" port: 44335 } }
I20260810 04:43:45.217163 15753 raft_consensus.cc:348] T 416edda2678e425d9c0c87ed4f56cf2a P 3a46d62cc08a49938add205cef6d55aa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "61a9f79165b8438299946a8f4b034178" member_type: VOTER last_known_addr { host: "127.22.167.66" port: 38979 } } peers { permanent_uuid: "3a46d62cc08a49938add205cef6d55aa" member_type: VOTER last_known_addr { host: "127.22.167.67" port: 39865 } } peers { permanent_uuid: "ed99582b3b574c00991af0deb19f27c1" member_type: VOTER last_known_addr { host: "127.22.167.65" port: 44335 } }
I20260810 04:43:45.217330 15753 raft_consensus.cc:374] T 416edda2678e425d9c0c87ed4f56cf2a P 3a46d62cc08a49938add205cef6d55aa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 04:43:45.217330 15752 raft_consensus.cc:374] T 416edda2678e425d9c0c87ed4f56cf2a P 61a9f79165b8438299946a8f4b034178 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 04:43:45.217376 15752 raft_consensus.cc:729] T 416edda2678e425d9c0c87ed4f56cf2a P 61a9f79165b8438299946a8f4b034178 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 61a9f79165b8438299946a8f4b034178, State: Initialized, Role: FOLLOWER
I20260810 04:43:45.217376 15753 raft_consensus.cc:729] T 416edda2678e425d9c0c87ed4f56cf2a P 3a46d62cc08a49938add205cef6d55aa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3a46d62cc08a49938add205cef6d55aa, State: Initialized, Role: FOLLOWER
I20260810 04:43:45.217499 15753 consensus_queue.cc:260] T 416edda2678e425d9c0c87ed4f56cf2a P 3a46d62cc08a49938add205cef6d55aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "61a9f79165b8438299946a8f4b034178" member_type: VOTER last_known_addr { host: "127.22.167.66" port: 38979 } } peers { permanent_uuid: "3a46d62cc08a49938add205cef6d55aa" member_type: VOTER last_known_addr { host: "127.22.167.67" port: 39865 } } peers { permanent_uuid: "ed99582b3b574c00991af0deb19f27c1" member_type: VOTER last_known_addr { host: "127.22.167.65" port: 44335 } }
I20260810 04:43:45.217499 15752 consensus_queue.cc:260] T 416edda2678e425d9c0c87ed4f56cf2a P 61a9f79165b8438299946a8f4b034178 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "61a9f79165b8438299946a8f4b034178" member_type: VOTER last_known_addr { host: "127.22.167.66" port: 38979 } } peers { permanent_uuid: "3a46d62cc08a49938add205cef6d55aa" member_type: VOTER last_known_addr { host: "127.22.167.67" port: 39865 } } peers { permanent_uuid: "ed99582b3b574c00991af0deb19f27c1" member_type: VOTER last_known_addr { host: "127.22.167.65" port: 44335 } }
I20260810 04:43:45.217780 15752 ts_tablet_manager.cc:1428] T 416edda2678e425d9c0c87ed4f56cf2a P 61a9f79165b8438299946a8f4b034178: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260810 04:43:45.217780 15753 ts_tablet_manager.cc:1428] T 416edda2678e425d9c0c87ed4f56cf2a P 3a46d62cc08a49938add205cef6d55aa: Time spent starting tablet: real 0.002s user 0.000s sys 0.003s
I20260810 04:43:45.217788 15737 heartbeater.cc:503] Master 127.22.167.126:46073 was elected leader, sending a full tablet report...
I20260810 04:43:45.217866 15516 heartbeater.cc:503] Master 127.22.167.126:46073 was elected leader, sending a full tablet report...
W20260810 04:43:45.308974 15518 tablet.cc:2369] T 416edda2678e425d9c0c87ed4f56cf2a P 61a9f79165b8438299946a8f4b034178: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 04:43:45.350189 15759 raft_consensus.cc:482] T 416edda2678e425d9c0c87ed4f56cf2a P 3a46d62cc08a49938add205cef6d55aa [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 04:43:45.350360 15759 raft_consensus.cc:504] T 416edda2678e425d9c0c87ed4f56cf2a P 3a46d62cc08a49938add205cef6d55aa [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "61a9f79165b8438299946a8f4b034178" member_type: VOTER last_known_addr { host: "127.22.167.66" port: 38979 } } peers { permanent_uuid: "3a46d62cc08a49938add205cef6d55aa" member_type: VOTER last_known_addr { host: "127.22.167.67" port: 39865 } } peers { permanent_uuid: "ed99582b3b574c00991af0deb19f27c1" member_type: VOTER last_known_addr { host: "127.22.167.65" port: 44335 } }
I20260810 04:43:45.350812 15759 leader_election.cc:288] T 416edda2678e425d9c0c87ed4f56cf2a P 3a46d62cc08a49938add205cef6d55aa [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 61a9f79165b8438299946a8f4b034178 (127.22.167.66:38979), ed99582b3b574c00991af0deb19f27c1 (127.22.167.65:44335)
I20260810 04:43:45.354311 15745 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:43:45.351044 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.167.67:39531 (local address 127.22.167.66:38979)
0810 04:43:45.351098 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0810 04:43:45.351099 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0810 04:43:45.351107 (+ 8us) server_negotiation.cc:407] Connection header received
0810 04:43:45.351134 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:43:45.351135 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:43:45.351144 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:43:45.351215 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 04:43:45.351373 (+ 158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:43:45.352102 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:43:45.353440 (+ 1338us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:43:45.353983 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:43:45.354060 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:43:45.354069 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0810 04:43:45.354206 (+ 137us) server_negotiation.cc:299] Negotiation successful
0810 04:43:45.354246 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260810 04:43:45.354311 15739 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:43:45.350955 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.66:38979 (local address 127.22.167.67:39531)
0810 04:43:45.351019 (+ 64us) negotiation.cc:107] Waiting for socket to connect
0810 04:43:45.351032 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0810 04:43:45.351074 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 04:43:45.351235 (+ 161us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 04:43:45.351236 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 04:43:45.351238 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 04:43:45.351307 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:43:45.351309 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:43:45.352188 (+ 879us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:43:45.352189 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:43:45.353353 (+ 1164us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:43:45.353356 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:43:45.354054 (+ 698us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:43:45.354055 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:43:45.354115 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:43:45.354118 (+ 3us) client_negotiation.cc:769] Sending connection context
0810 04:43:45.354195 (+ 77us) client_negotiation.cc:240] Negotiation successful
0810 04:43:45.354238 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":36,"mutex_wait_us":10}
I20260810 04:43:45.354465 15764 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:43:45.350973 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.65:44335 (local address 127.22.167.67:45811)
0810 04:43:45.351157 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0810 04:43:45.351164 (+ 7us) client_negotiation.cc:174] Beginning negotiation
0810 04:43:45.351206 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 04:43:45.351516 (+ 310us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 04:43:45.351517 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 04:43:45.351519 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 04:43:45.351587 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:43:45.351589 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:43:45.352374 (+ 785us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:43:45.352375 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:43:45.353441 (+ 1066us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:43:45.353443 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:43:45.354211 (+ 768us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:43:45.354212 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:43:45.354292 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:43:45.354296 (+ 4us) client_negotiation.cc:769] Sending connection context
0810 04:43:45.354346 (+ 50us) client_negotiation.cc:240] Negotiation successful
0810 04:43:45.354385 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":171,"thread_start_us":118,"threads_started":1}
I20260810 04:43:45.354458 15276 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:43:45.351031 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.167.67:45811 (local address 127.22.167.65:44335)
0810 04:43:45.351076 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0810 04:43:45.351079 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 04:43:45.351378 (+ 299us) server_negotiation.cc:407] Connection header received
0810 04:43:45.351409 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:43:45.351410 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:43:45.351441 (+ 31us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:43:45.351499 (+ 58us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 04:43:45.351648 (+ 149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:43:45.352304 (+ 656us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:43:45.353506 (+ 1202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:43:45.354108 (+ 602us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:43:45.354193 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:43:45.354278 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0810 04:43:45.354351 (+ 73us) server_negotiation.cc:299] Negotiation successful
0810 04:43:45.354394 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260810 04:43:45.354660 15433 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "416edda2678e425d9c0c87ed4f56cf2a" candidate_uuid: "3a46d62cc08a49938add205cef6d55aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "61a9f79165b8438299946a8f4b034178" is_pre_election: true
I20260810 04:43:45.354758 15181 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "416edda2678e425d9c0c87ed4f56cf2a" candidate_uuid: "3a46d62cc08a49938add205cef6d55aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ed99582b3b574c00991af0deb19f27c1" is_pre_election: true
I20260810 04:43:45.354823 15433 raft_consensus.cc:2454] T 416edda2678e425d9c0c87ed4f56cf2a P 61a9f79165b8438299946a8f4b034178 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a46d62cc08a49938add205cef6d55aa in term 0.
I20260810 04:43:45.354912 15181 raft_consensus.cc:2454] T 416edda2678e425d9c0c87ed4f56cf2a P ed99582b3b574c00991af0deb19f27c1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a46d62cc08a49938add205cef6d55aa in term 0.
I20260810 04:43:45.355075 15545 leader_election.cc:302] T 416edda2678e425d9c0c87ed4f56cf2a P 3a46d62cc08a49938add205cef6d55aa [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a46d62cc08a49938add205cef6d55aa, 61a9f79165b8438299946a8f4b034178; no voters:
I20260810 04:43:45.355230 15759 raft_consensus.cc:2781] T 416edda2678e425d9c0c87ed4f56cf2a P 3a46d62cc08a49938add205cef6d55aa [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 04:43:45.355297 15759 raft_consensus.cc:482] T 416edda2678e425d9c0c87ed4f56cf2a P 3a46d62cc08a49938add205cef6d55aa [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 04:43:45.355321 15759 raft_consensus.cc:3037] T 416edda2678e425d9c0c87ed4f56cf2a P 3a46d62cc08a49938add205cef6d55aa [term 0 FOLLOWER]: Advancing to term 1
I20260810 04:43:45.355988 15759 raft_consensus.cc:504] T 416edda2678e425d9c0c87ed4f56cf2a P 3a46d62cc08a49938add205cef6d55aa [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "61a9f79165b8438299946a8f4b034178" member_type: VOTER last_known_addr { host: "127.22.167.66" port: 38979 } } peers { permanent_uuid: "3a46d62cc08a49938add205cef6d55aa" member_type: VOTER last_known_addr { host: "127.22.167.67" port: 39865 } } peers { permanent_uuid: "ed99582b3b574c00991af0deb19f27c1" member_type: VOTER last_known_addr { host: "127.22.167.65" port: 44335 } }
I20260810 04:43:45.356096 15759 leader_election.cc:288] T 416edda2678e425d9c0c87ed4f56cf2a P 3a46d62cc08a49938add205cef6d55aa [CANDIDATE]: Term 1 election: Requested vote from peers 61a9f79165b8438299946a8f4b034178 (127.22.167.66:38979), ed99582b3b574c00991af0deb19f27c1 (127.22.167.65:44335)
I20260810 04:43:45.356269 15181 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "416edda2678e425d9c0c87ed4f56cf2a" candidate_uuid: "3a46d62cc08a49938add205cef6d55aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ed99582b3b574c00991af0deb19f27c1"
I20260810 04:43:45.356346 15181 raft_consensus.cc:3037] T 416edda2678e425d9c0c87ed4f56cf2a P ed99582b3b574c00991af0deb19f27c1 [term 0 FOLLOWER]: Advancing to term 1
I20260810 04:43:45.357010 15181 raft_consensus.cc:2454] T 416edda2678e425d9c0c87ed4f56cf2a P ed99582b3b574c00991af0deb19f27c1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3a46d62cc08a49938add205cef6d55aa in term 1.
I20260810 04:43:45.357110 15433 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "416edda2678e425d9c0c87ed4f56cf2a" candidate_uuid: "3a46d62cc08a49938add205cef6d55aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "61a9f79165b8438299946a8f4b034178"
I20260810 04:43:45.357185 15548 leader_election.cc:302] T 416edda2678e425d9c0c87ed4f56cf2a P 3a46d62cc08a49938add205cef6d55aa [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a46d62cc08a49938add205cef6d55aa, ed99582b3b574c00991af0deb19f27c1; no voters:
I20260810 04:43:45.357184 15433 raft_consensus.cc:3037] T 416edda2678e425d9c0c87ed4f56cf2a P 61a9f79165b8438299946a8f4b034178 [term 0 FOLLOWER]: Advancing to term 1
I20260810 04:43:45.357273 15759 raft_consensus.cc:2781] T 416edda2678e425d9c0c87ed4f56cf2a P 3a46d62cc08a49938add205cef6d55aa [term 1 FOLLOWER]: Leader election won for term 1
I20260810 04:43:45.357421 15759 raft_consensus.cc:686] T 416edda2678e425d9c0c87ed4f56cf2a P 3a46d62cc08a49938add205cef6d55aa [term 1 LEADER]: Becoming Leader. State: Replica: 3a46d62cc08a49938add205cef6d55aa, State: Running, Role: LEADER
I20260810 04:43:45.357530 15759 consensus_queue.cc:237] T 416edda2678e425d9c0c87ed4f56cf2a P 3a46d62cc08a49938add205cef6d55aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "61a9f79165b8438299946a8f4b034178" member_type: VOTER last_known_addr { host: "127.22.167.66" port: 38979 } } peers { permanent_uuid: "3a46d62cc08a49938add205cef6d55aa" member_type: VOTER last_known_addr { host: "127.22.167.67" port: 39865 } } peers { permanent_uuid: "ed99582b3b574c00991af0deb19f27c1" member_type: VOTER last_known_addr { host: "127.22.167.65" port: 44335 } }
I20260810 04:43:45.357973 15433 raft_consensus.cc:2454] T 416edda2678e425d9c0c87ed4f56cf2a P 61a9f79165b8438299946a8f4b034178 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3a46d62cc08a49938add205cef6d55aa in term 1.
I20260810 04:43:45.358490 14926 catalog_manager.cc:5495] T 416edda2678e425d9c0c87ed4f56cf2a P 3a46d62cc08a49938add205cef6d55aa reported cstate change: term changed from 0 to 1, leader changed from <none> to 3a46d62cc08a49938add205cef6d55aa (127.22.167.67). New cstate: current_term: 1 leader_uuid: "3a46d62cc08a49938add205cef6d55aa" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "61a9f79165b8438299946a8f4b034178" member_type: VOTER last_known_addr { host: "127.22.167.66" port: 38979 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3a46d62cc08a49938add205cef6d55aa" member_type: VOTER last_known_addr { host: "127.22.167.67" port: 39865 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ed99582b3b574c00991af0deb19f27c1" member_type: VOTER last_known_addr { host: "127.22.167.65" port: 44335 } health_report { overall_health: UNKNOWN } } }
I20260810 04:43:45.390789 15747 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:43:45.388724 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36974 (local address 127.22.167.67:39865)
0810 04:43:45.388786 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0810 04:43:45.388788 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 04:43:45.388795 (+ 7us) server_negotiation.cc:407] Connection header received
0810 04:43:45.388830 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:43:45.388830 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:43:45.388847 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:43:45.388907 (+ 60us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 04:43:45.388993 (+ 86us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:43:45.389454 (+ 461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:43:45.390118 (+ 664us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:43:45.390315 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:43:45.390363 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:43:45.390505 (+ 142us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 04:43:45.390609 (+ 104us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 04:43:45.390665 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0810 04:43:45.390681 (+ 16us) server_negotiation.cc:299] Negotiation successful
0810 04:43:45.390697 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260810 04:43:45.392542 15433 raft_consensus.cc:1261] T 416edda2678e425d9c0c87ed4f56cf2a P 61a9f79165b8438299946a8f4b034178 [term 1 FOLLOWER]: Refusing update from remote peer 3a46d62cc08a49938add205cef6d55aa: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 04:43:45.392545 15181 raft_consensus.cc:1261] T 416edda2678e425d9c0c87ed4f56cf2a P ed99582b3b574c00991af0deb19f27c1 [term 1 FOLLOWER]: Refusing update from remote peer 3a46d62cc08a49938add205cef6d55aa: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 04:43:45.392933 15765 consensus_queue.cc:1035] T 416edda2678e425d9c0c87ed4f56cf2a P 3a46d62cc08a49938add205cef6d55aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "61a9f79165b8438299946a8f4b034178" member_type: VOTER last_known_addr { host: "127.22.167.66" port: 38979 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 04:43:45.393067 15759 consensus_queue.cc:1035] T 416edda2678e425d9c0c87ed4f56cf2a P 3a46d62cc08a49938add205cef6d55aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "ed99582b3b574c00991af0deb19f27c1" member_type: VOTER last_known_addr { host: "127.22.167.65" port: 44335 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 04:43:45.395983 15776 mvcc.cc:205] Tried to move back new op lower bound from 7316836456005165056 to 7316836455865032704. Current Snapshot: MvccSnapshot[applied={T|T < 7316836456005165056}]
I20260810 04:43:45.397267 14926 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin'} at 127.0.0.1:38540:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 04:43:45.397348 14926 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin'} at 127.0.0.1:38540:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 04:43:45.397996 14926 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 1623678ec1f5413687149d71bb0bce92: Sending DeleteTablet for 3 replicas of tablet 416edda2678e425d9c0c87ed4f56cf2a
I20260810 04:43:45.398346 15627 tablet_service.cc:1514] Processing DeleteTablet for tablet 416edda2678e425d9c0c87ed4f56cf2a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 04:43:45 UTC) from {username='kudu', principal='kudu/127.22.167.126@KRBTEST.COM'} at 127.0.0.1:36966
I20260810 04:43:45.398381 15144 tablet_service.cc:1514] Processing DeleteTablet for tablet 416edda2678e425d9c0c87ed4f56cf2a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 04:43:45 UTC) from {username='kudu', principal='kudu/127.22.167.126@KRBTEST.COM'} at 127.0.0.1:35842
I20260810 04:43:45.398365 15404 tablet_service.cc:1514] Processing DeleteTablet for tablet 416edda2678e425d9c0c87ed4f56cf2a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 04:43:45 UTC) from {username='kudu', principal='kudu/127.22.167.126@KRBTEST.COM'} at 127.0.0.1:47238
I20260810 04:43:45.398459 15277 tablet_replica.cc:330] T 416edda2678e425d9c0c87ed4f56cf2a P ed99582b3b574c00991af0deb19f27c1: stopping tablet replica
I20260810 04:43:45.398505 15277 raft_consensus.cc:2229] T 416edda2678e425d9c0c87ed4f56cf2a P ed99582b3b574c00991af0deb19f27c1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 04:43:45.398541 15277 raft_consensus.cc:2258] T 416edda2678e425d9c0c87ed4f56cf2a P ed99582b3b574c00991af0deb19f27c1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 04:43:45.398638 23197 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskemv_AM/build/release/bin/kudu with pid 14988
I20260810 04:43:45.398658 15790 tablet_replica.cc:330] T 416edda2678e425d9c0c87ed4f56cf2a P 61a9f79165b8438299946a8f4b034178: stopping tablet replica
I20260810 04:43:45.401364 15790 raft_consensus.cc:2229] T 416edda2678e425d9c0c87ed4f56cf2a P 61a9f79165b8438299946a8f4b034178 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 04:43:45.401495 15790 raft_consensus.cc:2258] T 416edda2678e425d9c0c87ed4f56cf2a P 61a9f79165b8438299946a8f4b034178 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 04:43:45.408224 15790 ts_tablet_manager.cc:1905] T 416edda2678e425d9c0c87ed4f56cf2a P 61a9f79165b8438299946a8f4b034178: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 04:43:45.411836 15790 ts_tablet_manager.cc:1918] T 416edda2678e425d9c0c87ed4f56cf2a P 61a9f79165b8438299946a8f4b034178: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 04:43:45.411901 15790 log.cc:1196] T 416edda2678e425d9c0c87ed4f56cf2a P 61a9f79165b8438299946a8f4b034178: Deleting WAL directory at /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786336863388938-23197-0/minicluster-data/ts-1/wal/wals/416edda2678e425d9c0c87ed4f56cf2a
I20260810 04:43:45.412133 15790 ts_tablet_manager.cc:1939] T 416edda2678e425d9c0c87ed4f56cf2a P 61a9f79165b8438299946a8f4b034178: Deleting consensus metadata
I20260810 04:43:45.415225 14898 catalog_manager.cc:4841] TS 61a9f79165b8438299946a8f4b034178 (127.22.167.66:38979): tablet 416edda2678e425d9c0c87ed4f56cf2a (table test-table [id=f69bc2ad255a489c9c111542f5d88ef0]) successfully deleted
I20260810 04:43:45.417501 15789 tablet_replica.cc:330] T 416edda2678e425d9c0c87ed4f56cf2a P 3a46d62cc08a49938add205cef6d55aa: stopping tablet replica
I20260810 04:43:45.417629 15789 raft_consensus.cc:2229] T 416edda2678e425d9c0c87ed4f56cf2a P 3a46d62cc08a49938add205cef6d55aa [term 1 LEADER]: Raft consensus shutting down.
I20260810 04:43:45.417737 15789 raft_consensus.cc:2258] T 416edda2678e425d9c0c87ed4f56cf2a P 3a46d62cc08a49938add205cef6d55aa [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 04:43:45.418210 15789 ts_tablet_manager.cc:1905] T 416edda2678e425d9c0c87ed4f56cf2a P 3a46d62cc08a49938add205cef6d55aa: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 04:43:45.419723 15789 ts_tablet_manager.cc:1918] T 416edda2678e425d9c0c87ed4f56cf2a P 3a46d62cc08a49938add205cef6d55aa: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 04:43:45.419800 15789 log.cc:1196] T 416edda2678e425d9c0c87ed4f56cf2a P 3a46d62cc08a49938add205cef6d55aa: Deleting WAL directory at /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786336863388938-23197-0/minicluster-data/ts-2/wal/wals/416edda2678e425d9c0c87ed4f56cf2a
I20260810 04:43:45.420147 15789 ts_tablet_manager.cc:1939] T 416edda2678e425d9c0c87ed4f56cf2a P 3a46d62cc08a49938add205cef6d55aa: Deleting consensus metadata
I20260810 04:43:45.420581 14902 catalog_manager.cc:4841] TS 3a46d62cc08a49938add205cef6d55aa (127.22.167.67:39865): tablet 416edda2678e425d9c0c87ed4f56cf2a (table test-table [id=f69bc2ad255a489c9c111542f5d88ef0]) successfully deleted
W20260810 04:43:45.421833 14901 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.22.167.65:44335 (error 108)
I20260810 04:43:45.422113 23197 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskemv_AM/build/release/bin/kudu with pid 15267
I20260810 04:43:45.448205 15743 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:43:45.422267 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.65:44335 (local address 127.0.0.1:35850)
0810 04:43:45.438097 (+ 15830us) negotiation.cc:107] Waiting for socket to connect
0810 04:43:45.448154 (+ 10057us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.167.65:44335: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":15790}
W20260810 04:43:45.448397 14901 catalog_manager.cc:4584] TS ed99582b3b574c00991af0deb19f27c1 (127.22.167.65:44335): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 416edda2678e425d9c0c87ed4f56cf2a: Network error: Client connection negotiation failed: client connection to 127.22.167.65:44335: connect: Connection refused (error 111)
I20260810 04:43:45.452195 23197 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskemv_AM/build/release/bin/kudu with pid 15521
I20260810 04:43:45.463608 23197 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskemv_AM/build/release/bin/kudu with pid 14862
I20260810 04:43:45.472823 23197 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskemv_AM/build/release/bin/kudu
/tmp/dist-test-taskemv_AM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786336863388938-23197-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786336863388938-23197-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786336863388938-23197-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786336863388938-23197-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.167.126:46073
--webserver_interface=127.22.167.126
--webserver_port=35997
--keytab_file=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786336863388938-23197-0/krb5kdc/kudu_127.22.167.126.keytab
--principal=kudu/127.22.167.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.167.84:34205
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.167.126:46073
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786336863388938-23197-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786336863388938-23197-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786336863388938-23197-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 04:43:45.552541 15793 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 04:43:45.552736 15793 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 04:43:45.552765 15793 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 04:43:45.553869 15793 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 04:43:45.553908 15793 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 04:43:45.553926 15793 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 04:43:45.553942 15793 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 04:43:45.553958 15793 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 04:43:45.555405 15793 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.167.84:34205
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786336863388938-23197-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786336863388938-23197-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.22.167.126:46073
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786336863388938-23197-0/krb5kdc/kudu_127.22.167.126.keytab
--principal=kudu/127.22.167.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.167.126:46073
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786336863388938-23197-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.167.126
--webserver_port=35997
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15793
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786336863388938-23197-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 03:54:01 UTC on ccycloud.quasar-icijmt.root.comops.site
build id 4577009
I20260810 04:43:45.555645 15793 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 04:43:45.555842 15793 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 04:43:45.560351 15806 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 04:43:45.560364 15805 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 04:43:45.560339 15808 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 04:43:45.560613 15793 server_base.cc:1034] running on GCE node
I20260810 04:43:45.560865 15793 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 04:43:45.561136 15793 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 04:43:45.562278 15793 hybrid_clock.cc:648] HybridClock initialized: now 1786337025562255 us; error 32 us; skew 500 ppm
Aug 10 04:43:45 dist-test-slave-bqcl krb5kdc[13951](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786337025, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.167.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 04:43:45.564574 15793 init.cc:377] Logged in from keytab as kudu/127.22.167.126@KRBTEST.COM (short username kudu)
I20260810 04:43:45.564992 15793 webserver.cc:460] Webserver started at http://127.22.167.126:35997/ using document root <none> and password file <none>
I20260810 04:43:45.565208 15793 fs_manager.cc:359] Metadata directory not provided
I20260810 04:43:45.565268 15793 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 04:43:45.566536 15793 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260810 04:43:45.567178 15820 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 04:43:45.567381 15793 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260810 04:43:45.567466 15793 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786336863388938-23197-0/minicluster-data/master-0/data,/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786336863388938-23197-0/minicluster-data/master-0/wal
uuid: "1623678ec1f5413687149d71bb0bce92"
format_stamp: "Formatted at 2026-08-10 04:43:43 on dist-test-slave-bqcl"
server_key: "3bba34883e284cb5c91d59bf7a18c494"
server_key_iv: "1ae4bae684dc57bb70292d045cdf211c"
server_key_version: "encryptionkey@0"
I20260810 04:43:45.567760 15793 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786336863388938-23197-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786336863388938-23197-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786336863388938-23197-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 04:43:45.579960 15793 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 04:43:45.581620 15793 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 04:43:45.581749 15793 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 04:43:45.585628 15793 rpc_server.cc:307] RPC server started. Bound to: 127.22.167.126:46073
I20260810 04:43:45.585656 15903 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.167.126:46073 every 8 connection(s)
I20260810 04:43:45.585996 15793 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786336863388938-23197-0/minicluster-data/master-0/data/info.pb
I20260810 04:43:45.587532 15904 sys_catalog.cc:263] Verifying existing consensus state
I20260810 04:43:45.587687 23197 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskemv_AM/build/release/bin/kudu as pid 15793
I20260810 04:43:45.587965 23197 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskemv_AM/build/release/bin/kudu
/tmp/dist-test-taskemv_AM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786336863388938-23197-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786336863388938-23197-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786336863388938-23197-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786336863388938-23197-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.167.65:44335
--local_ip_for_outbound_sockets=127.22.167.65
--tserver_master_addrs=127.22.167.126:46073
--webserver_port=42653
--webserver_interface=127.22.167.65
--keytab_file=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786336863388938-23197-0/krb5kdc/kudu_127.22.167.65.keytab
--principal=kudu/127.22.167.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.167.84:34205
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786336863388938-23197-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786336863388938-23197-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786336863388938-23197-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 04:43:45.588397 15904 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1623678ec1f5413687149d71bb0bce92: Bootstrap starting.
I20260810 04:43:45.597440 15904 log.cc:824] T 00000000000000000000000000000000 P 1623678ec1f5413687149d71bb0bce92: Log is configured to *not* fsync() on all Append() calls
I20260810 04:43:45.599671 15904 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1623678ec1f5413687149d71bb0bce92: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=7 ignored=0} mutations{seen=9 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260810 04:43:45.599931 15904 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1623678ec1f5413687149d71bb0bce92: Bootstrap complete.
I20260810 04:43:45.601984 15904 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1623678ec1f5413687149d71bb0bce92 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1623678ec1f5413687149d71bb0bce92" member_type: VOTER last_known_addr { host: "127.22.167.126" port: 46073 } }
I20260810 04:43:45.602246 15904 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1623678ec1f5413687149d71bb0bce92 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1623678ec1f5413687149d71bb0bce92, State: Initialized, Role: FOLLOWER
I20260810 04:43:45.602404 15904 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1623678ec1f5413687149d71bb0bce92 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1623678ec1f5413687149d71bb0bce92" member_type: VOTER last_known_addr { host: "127.22.167.126" port: 46073 } }
I20260810 04:43:45.602465 15904 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1623678ec1f5413687149d71bb0bce92 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 04:43:45.602504 15904 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1623678ec1f5413687149d71bb0bce92 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 04:43:45.602552 15904 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1623678ec1f5413687149d71bb0bce92 [term 2 FOLLOWER]: Advancing to term 3
I20260810 04:43:45.603473 15904 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1623678ec1f5413687149d71bb0bce92 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1623678ec1f5413687149d71bb0bce92" member_type: VOTER last_known_addr { host: "127.22.167.126" port: 46073 } }
I20260810 04:43:45.603576 15904 leader_election.cc:302] T 00000000000000000000000000000000 P 1623678ec1f5413687149d71bb0bce92 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1623678ec1f5413687149d71bb0bce92; no voters:
I20260810 04:43:45.603734 15904 leader_election.cc:288] T 00000000000000000000000000000000 P 1623678ec1f5413687149d71bb0bce92 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260810 04:43:45.603777 15911 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1623678ec1f5413687149d71bb0bce92 [term 3 FOLLOWER]: Leader election won for term 3
I20260810 04:43:45.603885 15911 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1623678ec1f5413687149d71bb0bce92 [term 3 LEADER]: Becoming Leader. State: Replica: 1623678ec1f5413687149d71bb0bce92, State: Running, Role: LEADER
I20260810 04:43:45.603969 15911 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1623678ec1f5413687149d71bb0bce92 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1623678ec1f5413687149d71bb0bce92" member_type: VOTER last_known_addr { host: "127.22.167.126" port: 46073 } }
I20260810 04:43:45.603993 15904 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1623678ec1f5413687149d71bb0bce92 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 04:43:45.604238 15911 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1623678ec1f5413687149d71bb0bce92 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1623678ec1f5413687149d71bb0bce92. Latest consensus state: current_term: 3 leader_uuid: "1623678ec1f5413687149d71bb0bce92" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1623678ec1f5413687149d71bb0bce92" member_type: VOTER last_known_addr { host: "127.22.167.126" port: 46073 } } }
I20260810 04:43:45.604315 15911 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1623678ec1f5413687149d71bb0bce92 [sys.catalog]: This master's current role is: LEADER
I20260810 04:43:45.604494 15917 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 04:43:45.605330 15917 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 04:43:45.605441 15914 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1623678ec1f5413687149d71bb0bce92 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "1623678ec1f5413687149d71bb0bce92" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1623678ec1f5413687149d71bb0bce92" member_type: VOTER last_known_addr { host: "127.22.167.126" port: 46073 } } }
I20260810 04:43:45.605556 15914 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1623678ec1f5413687149d71bb0bce92 [sys.catalog]: This master's current role is: LEADER
I20260810 04:43:45.605449 15917 catalog_manager.cc:1259] Loaded cluster ID: 83957b148ce04e079ff7496e9b48b4fa
I20260810 04:43:45.605644 15917 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
F20260810 04:43:45.605973 15917 catalog_manager.cc:1466] Initializing Kudu internal certificate authority failed: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7faad0f779ad google::LogMessage::Fail() at ??:0
@ 0x7faad0f7a88a google::LogMessage::SendToLog() at ??:0
@ 0x7faad0f774de google::LogMessage::Flush() at ??:0
@ 0x7faad0f78a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7faad8e980f1 _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_.isra.591 at ??:0
@ 0x7faad8ea18b9 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7faad2e450d3 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7faad2e3a954 kudu::Thread::SuperviseThread() at ??:0
@ 0x7faad3c9e6db start_thread at ??:0
@ 0x7faacfa7561f clone at ??:0
W20260810 04:43:45.678103 15909 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 04:43:45.678333 15909 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 04:43:45.678363 15909 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 04:43:45.679453 15909 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 04:43:45.679490 15909 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 04:43:45.679524 15909 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.167.65
I20260810 04:43:45.680924 15909 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.167.84:34205
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786336863388938-23197-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786336863388938-23197-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786336863388938-23197-0/krb5kdc/kudu_127.22.167.65.keytab
--principal=kudu/127.22.167.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.167.65:44335
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786336863388938-23197-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.22.167.65
--webserver_port=42653
--tserver_master_addrs=127.22.167.126:46073
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15909
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.167.65
--log_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786336863388938-23197-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 03:54:01 UTC on ccycloud.quasar-icijmt.root.comops.site
build id 4577009
I20260810 04:43:45.681147 15909 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 04:43:45.681354 15909 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 04:43:45.686084 15973 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 04:43:45.686107 15969 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 04:43:45.686244 15909 server_base.cc:1034] running on GCE node
W20260810 04:43:45.686309 15970 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 04:43:45.686497 15909 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 04:43:45.686751 15909 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 04:43:45.687896 15909 hybrid_clock.cc:648] HybridClock initialized: now 1786337025687880 us; error 29 us; skew 500 ppm
Aug 10 04:43:45 dist-test-slave-bqcl krb5kdc[13951](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786337025, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.167.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 04:43:45.690369 15909 init.cc:377] Logged in from keytab as kudu/127.22.167.65@KRBTEST.COM (short username kudu)
I20260810 04:43:45.690793 15909 webserver.cc:460] Webserver started at http://127.22.167.65:42653/ using document root <none> and password file <none>
I20260810 04:43:45.691012 15909 fs_manager.cc:359] Metadata directory not provided
I20260810 04:43:45.691061 15909 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 04:43:45.692294 15909 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260810 04:43:45.692904 15982 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 04:43:45.693112 15909 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260810 04:43:45.693207 15909 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786336863388938-23197-0/minicluster-data/ts-0/data,/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786336863388938-23197-0/minicluster-data/ts-0/wal
uuid: "ed99582b3b574c00991af0deb19f27c1"
format_stamp: "Formatted at 2026-08-10 04:43:44 on dist-test-slave-bqcl"
server_key: "a79c210c6c8adf1172244207a1f9d808"
server_key_iv: "9e7a8eee06fb0a5364374b62d61ff33b"
server_key_version: "encryptionkey@0"
I20260810 04:43:45.693612 15909 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786336863388938-23197-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786336863388938-23197-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786336863388938-23197-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 04:43:45.726722 15909 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 04:43:45.729092 15909 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 04:43:45.729257 15909 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 04:43:45.729552 15909 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 04:43:45.730077 15996 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260810 04:43:45.731549 15909 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260810 04:43:45.731613 15909 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260810 04:43:45.731649 15909 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260810 04:43:45.732555 15909 ts_tablet_manager.cc:610] Registered 1 tablets
I20260810 04:43:45.732606 15909 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260810 04:43:45.732656 15996 tablet_bootstrap.cc:492] T 416edda2678e425d9c0c87ed4f56cf2a P ed99582b3b574c00991af0deb19f27c1: Bootstrap starting.
I20260810 04:43:45.739715 15909 rpc_server.cc:307] RPC server started. Bound to: 127.22.167.65:44335
I20260810 04:43:45.739959 16194 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.167.65:44335 every 8 connection(s)
I20260810 04:43:45.740159 15909 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786336863388938-23197-0/minicluster-data/ts-0/data/info.pb
I20260810 04:43:45.742647 15996 log.cc:824] T 416edda2678e425d9c0c87ed4f56cf2a P ed99582b3b574c00991af0deb19f27c1: Log is configured to *not* fsync() on all Append() calls
I20260810 04:43:45.744478 23197 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskemv_AM/build/release/bin/kudu as pid 15909
I20260810 04:43:45.744706 15996 tablet_bootstrap.cc:492] T 416edda2678e425d9c0c87ed4f56cf2a P ed99582b3b574c00991af0deb19f27c1: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260810 04:43:45.744752 23197 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskemv_AM/build/release/bin/kudu
/tmp/dist-test-taskemv_AM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786336863388938-23197-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786336863388938-23197-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786336863388938-23197-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786336863388938-23197-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.167.66:38979
--local_ip_for_outbound_sockets=127.22.167.66
--tserver_master_addrs=127.22.167.126:46073
--webserver_port=32775
--webserver_interface=127.22.167.66
--keytab_file=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786336863388938-23197-0/krb5kdc/kudu_127.22.167.66.keytab
--principal=kudu/127.22.167.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.167.84:34205
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786336863388938-23197-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786336863388938-23197-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786336863388938-23197-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 04:43:45.744975 15996 tablet_bootstrap.cc:492] T 416edda2678e425d9c0c87ed4f56cf2a P ed99582b3b574c00991af0deb19f27c1: Bootstrap complete.
I20260810 04:43:45.746066 15996 ts_tablet_manager.cc:1397] T 416edda2678e425d9c0c87ed4f56cf2a P ed99582b3b574c00991af0deb19f27c1: Time spent bootstrapping tablet: real 0.013s user 0.003s sys 0.007s
I20260810 04:43:45.747174 15996 raft_consensus.cc:348] T 416edda2678e425d9c0c87ed4f56cf2a P ed99582b3b574c00991af0deb19f27c1 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "61a9f79165b8438299946a8f4b034178" member_type: VOTER last_known_addr { host: "127.22.167.66" port: 38979 } } peers { permanent_uuid: "3a46d62cc08a49938add205cef6d55aa" member_type: VOTER last_known_addr { host: "127.22.167.67" port: 39865 } } peers { permanent_uuid: "ed99582b3b574c00991af0deb19f27c1" member_type: VOTER last_known_addr { host: "127.22.167.65" port: 44335 } }
I20260810 04:43:45.747400 15996 raft_consensus.cc:729] T 416edda2678e425d9c0c87ed4f56cf2a P ed99582b3b574c00991af0deb19f27c1 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: ed99582b3b574c00991af0deb19f27c1, State: Initialized, Role: FOLLOWER
I20260810 04:43:45.747685 15996 consensus_queue.cc:260] T 416edda2678e425d9c0c87ed4f56cf2a P ed99582b3b574c00991af0deb19f27c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "61a9f79165b8438299946a8f4b034178" member_type: VOTER last_known_addr { host: "127.22.167.66" port: 38979 } } peers { permanent_uuid: "3a46d62cc08a49938add205cef6d55aa" member_type: VOTER last_known_addr { host: "127.22.167.67" port: 39865 } } peers { permanent_uuid: "ed99582b3b574c00991af0deb19f27c1" member_type: VOTER last_known_addr { host: "127.22.167.65" port: 44335 } }
I20260810 04:43:45.747965 15996 ts_tablet_manager.cc:1428] T 416edda2678e425d9c0c87ed4f56cf2a P ed99582b3b574c00991af0deb19f27c1: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
W20260810 04:43:45.803450 16197 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 04:43:45.740845 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.126:46073 (local address 127.22.167.65:43061)
0810 04:43:45.741067 (+ 222us) negotiation.cc:107] Waiting for socket to connect
0810 04:43:45.741123 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0810 04:43:45.741814 (+ 691us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 04:43:45.803180 (+ 61366us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.167.126:46073: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":164,"thread_start_us":102,"threads_started":1}
W20260810 04:43:45.803709 16195 heartbeater.cc:644] Failed to heartbeat to 127.22.167.126:46073 (0 consecutive failures): Network error: Failed to ping master at 127.22.167.126:46073: Client connection negotiation failed: client connection to 127.22.167.126:46073: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260810 04:43:45.804009 16197 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:43:45.803909 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.126:46073 (local address 127.22.167.65:55481)
0810 04:43:45.803953 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0810 04:43:45.803985 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.167.126:46073: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
I20260810 04:43:45.804276 16197 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:43:45.804202 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.126:46073 (local address 127.22.167.65:44003)
0810 04:43:45.804235 (+ 33us) negotiation.cc:107] Waiting for socket to connect
0810 04:43:45.804254 (+ 19us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.167.126:46073: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":20}
W20260810 04:43:45.804373 16195 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260810 04:43:45.829967 16200 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 04:43:45.830188 16200 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 04:43:45.830216 16200 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 04:43:45.831303 16200 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 04:43:45.831336 16200 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 04:43:45.831360 16200 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.167.66
I20260810 04:43:45.832734 16200 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.167.84:34205
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786336863388938-23197-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786336863388938-23197-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786336863388938-23197-0/krb5kdc/kudu_127.22.167.66.keytab
--principal=kudu/127.22.167.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.167.66:38979
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786336863388938-23197-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.22.167.66
--webserver_port=32775
--tserver_master_addrs=127.22.167.126:46073
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16200
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.167.66
--log_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786336863388938-23197-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 03:54:01 UTC on ccycloud.quasar-icijmt.root.comops.site
build id 4577009
I20260810 04:43:45.832914 16200 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 04:43:45.833073 16200 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 04:43:45.837641 16212 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 04:43:45.837670 16214 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 04:43:45.837728 16211 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 04:43:45.837749 16200 server_base.cc:1034] running on GCE node
I20260810 04:43:45.837978 16200 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 04:43:45.838219 16200 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 04:43:45.839346 16200 hybrid_clock.cc:648] HybridClock initialized: now 1786337025839338 us; error 26 us; skew 500 ppm
Aug 10 04:43:45 dist-test-slave-bqcl krb5kdc[13951](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786337025, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.167.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 04:43:45.841672 16200 init.cc:377] Logged in from keytab as kudu/127.22.167.66@KRBTEST.COM (short username kudu)
I20260810 04:43:45.842100 16200 webserver.cc:460] Webserver started at http://127.22.167.66:32775/ using document root <none> and password file <none>
I20260810 04:43:45.842348 16200 fs_manager.cc:359] Metadata directory not provided
I20260810 04:43:45.842398 16200 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 04:43:45.843628 16200 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260810 04:43:45.844148 16225 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 04:43:45.844297 16200 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260810 04:43:45.844377 16200 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786336863388938-23197-0/minicluster-data/ts-1/data,/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786336863388938-23197-0/minicluster-data/ts-1/wal
uuid: "61a9f79165b8438299946a8f4b034178"
format_stamp: "Formatted at 2026-08-10 04:43:44 on dist-test-slave-bqcl"
server_key: "91d51958193417a18cd51e9307e9fafc"
server_key_iv: "aa572f67f2d7ed0a2d36cdad795e0015"
server_key_version: "encryptionkey@0"
I20260810 04:43:45.844661 16200 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786336863388938-23197-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786336863388938-23197-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786336863388938-23197-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 04:43:45.867197 16200 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 04:43:45.869280 16200 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 04:43:45.869415 16200 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 04:43:45.869622 16200 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 04:43:45.869912 16200 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 04:43:45.869946 16200 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 04:43:45.869975 16200 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 04:43:45.869998 16200 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 04:43:45.875737 16200 rpc_server.cc:307] RPC server started. Bound to: 127.22.167.66:38979
I20260810 04:43:45.875800 16449 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.167.66:38979 every 8 connection(s)
I20260810 04:43:45.876103 16200 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786336863388938-23197-0/minicluster-data/ts-1/data/info.pb
I20260810 04:43:45.877278 16455 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:43:45.876716 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.126:46073 (local address 127.22.167.66:56285)
0810 04:43:45.876858 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0810 04:43:45.876934 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.167.126:46073: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":75,"threads_started":1}
W20260810 04:43:45.877466 16452 heartbeater.cc:644] Failed to heartbeat to 127.22.167.126:46073 (0 consecutive failures): Network error: Failed to ping master at 127.22.167.126:46073: Client connection negotiation failed: client connection to 127.22.167.126:46073: connect: Connection refused (error 111)
I20260810 04:43:45.877707 16455 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:43:45.877629 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.126:46073 (local address 127.22.167.66:39369)
0810 04:43:45.877666 (+ 37us) negotiation.cc:107] Waiting for socket to connect
0810 04:43:45.877690 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.167.126:46073: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":24}
I20260810 04:43:45.877897 16455 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:43:45.877850 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.126:46073 (local address 127.22.167.66:54477)
0810 04:43:45.877870 (+ 20us) negotiation.cc:107] Waiting for socket to connect
0810 04:43:45.877886 (+ 16us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.167.126:46073: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":11}
W20260810 04:43:45.877977 16452 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260810 04:43:45.881439 23197 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskemv_AM/build/release/bin/kudu as pid 16200
I20260810 04:43:45.881659 23197 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskemv_AM/build/release/bin/kudu
/tmp/dist-test-taskemv_AM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786336863388938-23197-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786336863388938-23197-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786336863388938-23197-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786336863388938-23197-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.167.67:39865
--local_ip_for_outbound_sockets=127.22.167.67
--tserver_master_addrs=127.22.167.126:46073
--webserver_port=35631
--webserver_interface=127.22.167.67
--keytab_file=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786336863388938-23197-0/krb5kdc/kudu_127.22.167.67.keytab
--principal=kudu/127.22.167.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.167.84:34205
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786336863388938-23197-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786336863388938-23197-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786336863388938-23197-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 04:43:45.961818 16456 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 04:43:45.962010 16456 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 04:43:45.962033 16456 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 04:43:45.963151 16456 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 04:43:45.963191 16456 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 04:43:45.963217 16456 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.167.67
I20260810 04:43:45.964570 16456 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.167.84:34205
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786336863388938-23197-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786336863388938-23197-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786336863388938-23197-0/krb5kdc/kudu_127.22.167.67.keytab
--principal=kudu/127.22.167.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.167.67:39865
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786336863388938-23197-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.22.167.67
--webserver_port=35631
--tserver_master_addrs=127.22.167.126:46073
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16456
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.167.67
--log_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786336863388938-23197-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 03:54:01 UTC on ccycloud.quasar-icijmt.root.comops.site
build id 4577009
I20260810 04:43:45.964740 16456 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 04:43:45.964910 16456 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 04:43:45.969578 16462 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 04:43:45.969653 16469 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 04:43:45.969691 16464 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 04:43:45.969722 16456 server_base.cc:1034] running on GCE node
I20260810 04:43:45.970006 16456 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 04:43:45.970261 16456 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 04:43:45.971395 16456 hybrid_clock.cc:648] HybridClock initialized: now 1786337025971379 us; error 33 us; skew 500 ppm
Aug 10 04:43:45 dist-test-slave-bqcl krb5kdc[13951](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786337025, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.167.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 04:43:45.973799 16456 init.cc:377] Logged in from keytab as kudu/127.22.167.67@KRBTEST.COM (short username kudu)
I20260810 04:43:45.974201 16456 webserver.cc:460] Webserver started at http://127.22.167.67:35631/ using document root <none> and password file <none>
I20260810 04:43:45.974377 16456 fs_manager.cc:359] Metadata directory not provided
I20260810 04:43:45.974417 16456 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 04:43:45.975603 16456 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260810 04:43:45.976118 16477 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 04:43:45.976258 16456 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260810 04:43:45.976336 16456 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786336863388938-23197-0/minicluster-data/ts-2/data,/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786336863388938-23197-0/minicluster-data/ts-2/wal
uuid: "3a46d62cc08a49938add205cef6d55aa"
format_stamp: "Formatted at 2026-08-10 04:43:44 on dist-test-slave-bqcl"
server_key: "f5fb01f1cfad0883040e79fcf3eb13d1"
server_key_iv: "8a368d70585170c801565df9a6544025"
server_key_version: "encryptionkey@0"
I20260810 04:43:45.976672 16456 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786336863388938-23197-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786336863388938-23197-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786336863388938-23197-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 04:43:45.997200 16456 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 04:43:45.999321 16456 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 04:43:45.999447 16456 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 04:43:45.999663 16456 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 04:43:45.999948 16456 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 04:43:45.999981 16456 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 04:43:46.000002 16456 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 04:43:46.000026 16456 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 04:43:46.005388 16456 rpc_server.cc:307] RPC server started. Bound to: 127.22.167.67:39865
I20260810 04:43:46.005455 16698 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.167.67:39865 every 8 connection(s)
I20260810 04:43:46.005743 16456 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786336863388938-23197-0/minicluster-data/ts-2/data/info.pb
I20260810 04:43:46.006740 16703 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:43:46.006366 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.126:46073 (local address 127.22.167.67:41655)
0810 04:43:46.006510 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0810 04:43:46.006586 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.167.126:46073: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":75,"threads_started":1}
I20260810 04:43:46.006968 23197 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskemv_AM/build/release/bin/kudu as pid 16456
W20260810 04:43:46.007164 16700 heartbeater.cc:644] Failed to heartbeat to 127.22.167.126:46073 (0 consecutive failures): Network error: Failed to ping master at 127.22.167.126:46073: Client connection negotiation failed: client connection to 127.22.167.126:46073: connect: Connection refused (error 111)
I20260810 04:43:46.007454 16703 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:43:46.007362 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.126:46073 (local address 127.22.167.67:50609)
0810 04:43:46.007404 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0810 04:43:46.007430 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.167.126:46073: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":25}
I20260810 04:43:46.007704 16703 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:43:46.007632 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.126:46073 (local address 127.22.167.67:56283)
0810 04:43:46.007659 (+ 27us) negotiation.cc:107] Waiting for socket to connect
0810 04:43:46.007682 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.167.126:46073: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":14}
W20260810 04:43:46.007799 16700 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260810 04:43:46.804883 16707 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:43:46.804635 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.126:46073 (local address 127.22.167.65:51593)
0810 04:43:46.804776 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0810 04:43:46.804818 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.167.126:46073: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":64,"threads_started":1}
I20260810 04:43:46.878470 16709 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:43:46.878270 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.126:46073 (local address 127.22.167.66:50787)
0810 04:43:46.878388 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0810 04:43:46.878419 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.167.126:46073: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":73,"threads_started":1}
I20260810 04:43:47.008262 16711 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:43:47.008051 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.126:46073 (local address 127.22.167.67:35085)
0810 04:43:47.008179 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0810 04:43:47.008214 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.167.126:46073: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":66,"threads_started":1}
I20260810 04:43:47.601043 16712 raft_consensus.cc:482] T 416edda2678e425d9c0c87ed4f56cf2a P ed99582b3b574c00991af0deb19f27c1 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 04:43:47.601176 16712 raft_consensus.cc:504] T 416edda2678e425d9c0c87ed4f56cf2a P ed99582b3b574c00991af0deb19f27c1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "61a9f79165b8438299946a8f4b034178" member_type: VOTER last_known_addr { host: "127.22.167.66" port: 38979 } } peers { permanent_uuid: "3a46d62cc08a49938add205cef6d55aa" member_type: VOTER last_known_addr { host: "127.22.167.67" port: 39865 } } peers { permanent_uuid: "ed99582b3b574c00991af0deb19f27c1" member_type: VOTER last_known_addr { host: "127.22.167.65" port: 44335 } }
I20260810 04:43:47.601507 16712 leader_election.cc:288] T 416edda2678e425d9c0c87ed4f56cf2a P ed99582b3b574c00991af0deb19f27c1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 61a9f79165b8438299946a8f4b034178 (127.22.167.66:38979), 3a46d62cc08a49938add205cef6d55aa (127.22.167.67:39865)
Aug 10 04:43:47 dist-test-slave-bqcl krb5kdc[13951](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786337025, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.167.65@KRBTEST.COM for kudu/127.22.167.66@KRBTEST.COM
Aug 10 04:43:47 dist-test-slave-bqcl krb5kdc[13951](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786337025, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.167.65@KRBTEST.COM for kudu/127.22.167.67@KRBTEST.COM
I20260810 04:43:47.611569 16716 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:43:47.601784 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.167.65:43125 (local address 127.22.167.67:39865)
0810 04:43:47.601973 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0810 04:43:47.601980 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 04:43:47.601999 (+ 19us) server_negotiation.cc:407] Connection header received
0810 04:43:47.602586 (+ 587us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:43:47.602590 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:43:47.602653 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:43:47.602808 (+ 155us) server_negotiation.cc:226] Negotiated authn=SASL
0810 04:43:47.603480 (+ 672us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:43:47.604346 (+ 866us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:43:47.605354 (+ 1008us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:43:47.605751 (+ 397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:43:47.605840 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:43:47.608569 (+ 2729us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 04:43:47.608572 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 04:43:47.608573 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 04:43:47.608582 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0810 04:43:47.610175 (+ 1593us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 04:43:47.610537 (+ 362us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 04:43:47.610540 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 04:43:47.610540 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 04:43:47.610633 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 04:43:47.610837 (+ 204us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 04:43:47.610837 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 04:43:47.610837 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 04:43:47.611066 (+ 229us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 04:43:47.611143 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0810 04:43:47.611335 (+ 192us) server_negotiation.cc:299] Negotiation successful
0810 04:43:47.611452 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":149,"thread_start_us":69,"threads_started":1}
I20260810 04:43:47.611578 16714 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:43:47.601637 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.66:38979 (local address 127.22.167.65:57923)
0810 04:43:47.601762 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0810 04:43:47.601772 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0810 04:43:47.601830 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 04:43:47.602852 (+ 1022us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 04:43:47.602855 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 04:43:47.603162 (+ 307us) client_negotiation.cc:189] Negotiated authn=SASL
0810 04:43:47.603352 (+ 190us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:43:47.603357 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:43:47.604435 (+ 1078us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:43:47.604435 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:43:47.605274 (+ 839us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:43:47.605276 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:43:47.605841 (+ 565us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:43:47.605842 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:43:47.605946 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:43:47.606519 (+ 573us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 04:43:47.606524 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0810 04:43:47.607936 (+ 1412us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 04:43:47.609889 (+ 1953us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 04:43:47.609897 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 04:43:47.609899 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 04:43:47.610195 (+ 296us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 04:43:47.610452 (+ 257us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 04:43:47.610453 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 04:43:47.610453 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 04:43:47.610585 (+ 132us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 04:43:47.611033 (+ 448us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 04:43:47.611040 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 04:43:47.611181 (+ 141us) client_negotiation.cc:769] Sending connection context
0810 04:43:47.611270 (+ 89us) client_negotiation.cc:240] Negotiation successful
0810 04:43:47.611452 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":110,"mutex_wait_us":49,"thread_start_us":69,"threads_started":1}
I20260810 04:43:47.611593 16715 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:43:47.601792 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.167.65:57923 (local address 127.22.167.66:38979)
0810 04:43:47.601949 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0810 04:43:47.601963 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0810 04:43:47.601988 (+ 25us) server_negotiation.cc:407] Connection header received
0810 04:43:47.602586 (+ 598us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:43:47.602590 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:43:47.602653 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:43:47.602808 (+ 155us) server_negotiation.cc:226] Negotiated authn=SASL
0810 04:43:47.603472 (+ 664us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:43:47.604325 (+ 853us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:43:47.605361 (+ 1036us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:43:47.605760 (+ 399us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:43:47.605858 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:43:47.608032 (+ 2174us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 04:43:47.608035 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 04:43:47.608036 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 04:43:47.608045 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0810 04:43:47.609808 (+ 1763us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 04:43:47.610280 (+ 472us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 04:43:47.610283 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 04:43:47.610284 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 04:43:47.610396 (+ 112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 04:43:47.610661 (+ 265us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 04:43:47.610662 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 04:43:47.610663 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 04:43:47.610968 (+ 305us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 04:43:47.611028 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0810 04:43:47.611340 (+ 312us) server_negotiation.cc:299] Negotiation successful
0810 04:43:47.611456 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":118,"thread_start_us":54,"threads_started":1}
I20260810 04:43:47.611655 16713 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:43:47.601662 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.67:39865 (local address 127.22.167.65:43125)
0810 04:43:47.601813 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0810 04:43:47.601826 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0810 04:43:47.601891 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 04:43:47.602937 (+ 1046us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 04:43:47.602938 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 04:43:47.603168 (+ 230us) client_negotiation.cc:189] Negotiated authn=SASL
0810 04:43:47.603352 (+ 184us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:43:47.603357 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:43:47.604449 (+ 1092us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:43:47.604449 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:43:47.605274 (+ 825us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:43:47.605276 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:43:47.605834 (+ 558us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:43:47.605835 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:43:47.605947 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:43:47.606476 (+ 529us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 04:43:47.606484 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0810 04:43:47.608342 (+ 1858us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 04:43:47.610247 (+ 1905us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 04:43:47.610248 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 04:43:47.610249 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 04:43:47.610452 (+ 203us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 04:43:47.610711 (+ 259us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 04:43:47.610712 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 04:43:47.610712 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 04:43:47.610777 (+ 65us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 04:43:47.611132 (+ 355us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 04:43:47.611133 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 04:43:47.611177 (+ 44us) client_negotiation.cc:769] Sending connection context
0810 04:43:47.611270 (+ 93us) client_negotiation.cc:240] Negotiation successful
0810 04:43:47.611530 (+ 260us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":135,"mutex_wait_us":170,"thread_start_us":49,"threads_started":1}
W20260810 04:43:47.612561 15983 leader_election.cc:341] T 416edda2678e425d9c0c87ed4f56cf2a P ed99582b3b574c00991af0deb19f27c1 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 61a9f79165b8438299946a8f4b034178 (127.22.167.66:38979): Not found: Tablet not found: 416edda2678e425d9c0c87ed4f56cf2a
W20260810 04:43:47.612905 15990 leader_election.cc:341] T 416edda2678e425d9c0c87ed4f56cf2a P ed99582b3b574c00991af0deb19f27c1 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 3a46d62cc08a49938add205cef6d55aa (127.22.167.67:39865): Not found: Tablet not found: 416edda2678e425d9c0c87ed4f56cf2a
I20260810 04:43:47.612950 15990 leader_election.cc:302] T 416edda2678e425d9c0c87ed4f56cf2a P ed99582b3b574c00991af0deb19f27c1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ed99582b3b574c00991af0deb19f27c1; no voters: 3a46d62cc08a49938add205cef6d55aa, 61a9f79165b8438299946a8f4b034178
I20260810 04:43:47.613067 16712 raft_consensus.cc:2726] T 416edda2678e425d9c0c87ed4f56cf2a P ed99582b3b574c00991af0deb19f27c1 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260810 04:43:47.805430 16713 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:43:47.805297 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.126:46073 (local address 127.22.167.65:33841)
0810 04:43:47.805351 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0810 04:43:47.805387 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.167.126:46073: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
I20260810 04:43:47.879086 16717 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:43:47.878864 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.126:46073 (local address 127.22.167.66:41569)
0810 04:43:47.878984 (+ 120us) negotiation.cc:107] Waiting for socket to connect
0810 04:43:47.879029 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.167.126:46073: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":74,"threads_started":1}
I20260810 04:43:48.008831 16719 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:43:48.008633 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.126:46073 (local address 127.22.167.67:39375)
0810 04:43:48.008739 (+ 106us) negotiation.cc:107] Waiting for socket to connect
0810 04:43:48.008786 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.167.126:46073: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":93,"thread_start_us":61,"threads_started":1}
I20260810 04:43:48.806073 16722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:43:48.805847 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.126:46073 (local address 127.22.167.65:39597)
0810 04:43:48.805985 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0810 04:43:48.806022 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.167.126:46073: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":75,"threads_started":1}
I20260810 04:43:48.879648 16723 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:43:48.879420 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.126:46073 (local address 127.22.167.66:48877)
0810 04:43:48.879552 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0810 04:43:48.879608 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.167.126:46073: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":63,"threads_started":1}
I20260810 04:43:49.009403 16724 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:43:49.009221 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.126:46073 (local address 127.22.167.67:32897)
0810 04:43:49.009335 (+ 114us) negotiation.cc:107] Waiting for socket to connect
0810 04:43:49.009365 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.167.126:46073: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":100,"thread_start_us":62,"threads_started":1}
I20260810 04:43:49.616997 16726 raft_consensus.cc:482] T 416edda2678e425d9c0c87ed4f56cf2a P ed99582b3b574c00991af0deb19f27c1 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 04:43:49.617098 16726 raft_consensus.cc:504] T 416edda2678e425d9c0c87ed4f56cf2a P ed99582b3b574c00991af0deb19f27c1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "61a9f79165b8438299946a8f4b034178" member_type: VOTER last_known_addr { host: "127.22.167.66" port: 38979 } } peers { permanent_uuid: "3a46d62cc08a49938add205cef6d55aa" member_type: VOTER last_known_addr { host: "127.22.167.67" port: 39865 } } peers { permanent_uuid: "ed99582b3b574c00991af0deb19f27c1" member_type: VOTER last_known_addr { host: "127.22.167.65" port: 44335 } }
I20260810 04:43:49.617254 16726 leader_election.cc:288] T 416edda2678e425d9c0c87ed4f56cf2a P ed99582b3b574c00991af0deb19f27c1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 61a9f79165b8438299946a8f4b034178 (127.22.167.66:38979), 3a46d62cc08a49938add205cef6d55aa (127.22.167.67:39865)
W20260810 04:43:49.617683 15983 leader_election.cc:341] T 416edda2678e425d9c0c87ed4f56cf2a P ed99582b3b574c00991af0deb19f27c1 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 61a9f79165b8438299946a8f4b034178 (127.22.167.66:38979): Not found: Tablet not found: 416edda2678e425d9c0c87ed4f56cf2a
W20260810 04:43:49.617806 15990 leader_election.cc:341] T 416edda2678e425d9c0c87ed4f56cf2a P ed99582b3b574c00991af0deb19f27c1 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 3a46d62cc08a49938add205cef6d55aa (127.22.167.67:39865): Not found: Tablet not found: 416edda2678e425d9c0c87ed4f56cf2a
I20260810 04:43:49.617887 15990 leader_election.cc:302] T 416edda2678e425d9c0c87ed4f56cf2a P ed99582b3b574c00991af0deb19f27c1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ed99582b3b574c00991af0deb19f27c1; no voters: 3a46d62cc08a49938add205cef6d55aa, 61a9f79165b8438299946a8f4b034178
I20260810 04:43:49.617990 16726 raft_consensus.cc:2726] T 416edda2678e425d9c0c87ed4f56cf2a P ed99582b3b574c00991af0deb19f27c1 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260810 04:43:49.806737 16727 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:43:49.806492 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.126:46073 (local address 127.22.167.65:39277)
0810 04:43:49.806634 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0810 04:43:49.806672 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.167.126:46073: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":90,"threads_started":1}
I20260810 04:43:49.880237 16730 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:43:49.880037 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.126:46073 (local address 127.22.167.66:59773)
0810 04:43:49.880147 (+ 110us) negotiation.cc:107] Waiting for socket to connect
0810 04:43:49.880182 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.167.126:46073: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":96,"thread_start_us":62,"threads_started":1}
I20260810 04:43:50.010032 16731 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:43:50.009827 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.126:46073 (local address 127.22.167.67:50175)
0810 04:43:50.009940 (+ 113us) negotiation.cc:107] Waiting for socket to connect
0810 04:43:50.009977 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.167.126:46073: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":96,"thread_start_us":60,"threads_started":1}
I20260810 04:43:50.807356 16732 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:43:50.807153 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.126:46073 (local address 127.22.167.65:37569)
0810 04:43:50.807273 (+ 120us) negotiation.cc:107] Waiting for socket to connect
0810 04:43:50.807308 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.167.126:46073: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":68,"threads_started":1}
I20260810 04:43:50.880800 16733 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:43:50.880570 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.126:46073 (local address 127.22.167.66:39337)
0810 04:43:50.880720 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0810 04:43:50.880757 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.167.126:46073: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":86,"threads_started":1}
I20260810 04:43:51.010640 16737 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:43:51.010432 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.126:46073 (local address 127.22.167.67:60545)
0810 04:43:51.010549 (+ 117us) negotiation.cc:107] Waiting for socket to connect
0810 04:43:51.010582 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.167.126:46073: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":101,"thread_start_us":62,"threads_started":1}
I20260810 04:43:51.807943 16739 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:43:51.807733 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.126:46073 (local address 127.22.167.65:33569)
0810 04:43:51.807860 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0810 04:43:51.807892 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.167.126:46073: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":70,"threads_started":1}
I20260810 04:43:51.881348 16740 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:43:51.881160 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.126:46073 (local address 127.22.167.66:56421)
0810 04:43:51.881274 (+ 114us) negotiation.cc:107] Waiting for socket to connect
0810 04:43:51.881307 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.167.126:46073: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":100,"thread_start_us":62,"threads_started":1}
I20260810 04:43:52.011188 16742 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:43:52.010996 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.126:46073 (local address 127.22.167.67:54775)
0810 04:43:52.011105 (+ 109us) negotiation.cc:107] Waiting for socket to connect
0810 04:43:52.011149 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.167.126:46073: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":94,"thread_start_us":61,"threads_started":1}
I20260810 04:43:52.718266 16747 raft_consensus.cc:482] T 416edda2678e425d9c0c87ed4f56cf2a P ed99582b3b574c00991af0deb19f27c1 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 04:43:52.718357 16747 raft_consensus.cc:504] T 416edda2678e425d9c0c87ed4f56cf2a P ed99582b3b574c00991af0deb19f27c1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "61a9f79165b8438299946a8f4b034178" member_type: VOTER last_known_addr { host: "127.22.167.66" port: 38979 } } peers { permanent_uuid: "3a46d62cc08a49938add205cef6d55aa" member_type: VOTER last_known_addr { host: "127.22.167.67" port: 39865 } } peers { permanent_uuid: "ed99582b3b574c00991af0deb19f27c1" member_type: VOTER last_known_addr { host: "127.22.167.65" port: 44335 } }
I20260810 04:43:52.718519 16747 leader_election.cc:288] T 416edda2678e425d9c0c87ed4f56cf2a P ed99582b3b574c00991af0deb19f27c1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 61a9f79165b8438299946a8f4b034178 (127.22.167.66:38979), 3a46d62cc08a49938add205cef6d55aa (127.22.167.67:39865)
W20260810 04:43:52.718874 15983 leader_election.cc:341] T 416edda2678e425d9c0c87ed4f56cf2a P ed99582b3b574c00991af0deb19f27c1 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 61a9f79165b8438299946a8f4b034178 (127.22.167.66:38979): Not found: Tablet not found: 416edda2678e425d9c0c87ed4f56cf2a
W20260810 04:43:52.718953 15990 leader_election.cc:341] T 416edda2678e425d9c0c87ed4f56cf2a P ed99582b3b574c00991af0deb19f27c1 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 3a46d62cc08a49938add205cef6d55aa (127.22.167.67:39865): Not found: Tablet not found: 416edda2678e425d9c0c87ed4f56cf2a
I20260810 04:43:52.719022 15990 leader_election.cc:302] T 416edda2678e425d9c0c87ed4f56cf2a P ed99582b3b574c00991af0deb19f27c1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ed99582b3b574c00991af0deb19f27c1; no voters: 3a46d62cc08a49938add205cef6d55aa, 61a9f79165b8438299946a8f4b034178
I20260810 04:43:52.719098 16747 raft_consensus.cc:2726] T 416edda2678e425d9c0c87ed4f56cf2a P ed99582b3b574c00991af0deb19f27c1 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260810 04:43:52.808590 16748 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:43:52.808366 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.126:46073 (local address 127.22.167.65:48467)
0810 04:43:52.808488 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0810 04:43:52.808538 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.167.126:46073: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":66,"threads_started":1}
I20260810 04:43:52.881897 16749 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:43:52.881713 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.126:46073 (local address 127.22.167.66:41869)
0810 04:43:52.881825 (+ 112us) negotiation.cc:107] Waiting for socket to connect
0810 04:43:52.881855 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.167.126:46073: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":99,"thread_start_us":59,"threads_started":1}
I20260810 04:43:53.011802 16751 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:43:53.011598 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.126:46073 (local address 127.22.167.67:39181)
0810 04:43:53.011727 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0810 04:43:53.011761 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.167.126:46073: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":80,"threads_started":1}
I20260810 04:43:53.809244 16753 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:43:53.809024 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.126:46073 (local address 127.22.167.65:34733)
0810 04:43:53.809156 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0810 04:43:53.809193 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.167.126:46073: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":66,"threads_started":1}
I20260810 04:43:53.882485 16754 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:43:53.882295 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.126:46073 (local address 127.22.167.66:51729)
0810 04:43:53.882413 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0810 04:43:53.882445 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.167.126:46073: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":103,"thread_start_us":61,"threads_started":1}
I20260810 04:43:54.012393 16755 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:43:54.012169 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.126:46073 (local address 127.22.167.67:39787)
0810 04:43:54.012312 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0810 04:43:54.012349 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.167.126:46073: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":61,"threads_started":1}
I20260810 04:43:54.327021 16756 raft_consensus.cc:482] T 416edda2678e425d9c0c87ed4f56cf2a P ed99582b3b574c00991af0deb19f27c1 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 04:43:54.327152 16756 raft_consensus.cc:504] T 416edda2678e425d9c0c87ed4f56cf2a P ed99582b3b574c00991af0deb19f27c1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "61a9f79165b8438299946a8f4b034178" member_type: VOTER last_known_addr { host: "127.22.167.66" port: 38979 } } peers { permanent_uuid: "3a46d62cc08a49938add205cef6d55aa" member_type: VOTER last_known_addr { host: "127.22.167.67" port: 39865 } } peers { permanent_uuid: "ed99582b3b574c00991af0deb19f27c1" member_type: VOTER last_known_addr { host: "127.22.167.65" port: 44335 } }
I20260810 04:43:54.327298 16756 leader_election.cc:288] T 416edda2678e425d9c0c87ed4f56cf2a P ed99582b3b574c00991af0deb19f27c1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 61a9f79165b8438299946a8f4b034178 (127.22.167.66:38979), 3a46d62cc08a49938add205cef6d55aa (127.22.167.67:39865)
W20260810 04:43:54.327658 15983 leader_election.cc:341] T 416edda2678e425d9c0c87ed4f56cf2a P ed99582b3b574c00991af0deb19f27c1 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 61a9f79165b8438299946a8f4b034178 (127.22.167.66:38979): Not found: Tablet not found: 416edda2678e425d9c0c87ed4f56cf2a
W20260810 04:43:54.327751 15990 leader_election.cc:341] T 416edda2678e425d9c0c87ed4f56cf2a P ed99582b3b574c00991af0deb19f27c1 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 3a46d62cc08a49938add205cef6d55aa (127.22.167.67:39865): Not found: Tablet not found: 416edda2678e425d9c0c87ed4f56cf2a
I20260810 04:43:54.327824 15990 leader_election.cc:302] T 416edda2678e425d9c0c87ed4f56cf2a P ed99582b3b574c00991af0deb19f27c1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ed99582b3b574c00991af0deb19f27c1; no voters: 3a46d62cc08a49938add205cef6d55aa, 61a9f79165b8438299946a8f4b034178
I20260810 04:43:54.327931 16756 raft_consensus.cc:2726] T 416edda2678e425d9c0c87ed4f56cf2a P ed99582b3b574c00991af0deb19f27c1 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260810 04:43:54.809872 16757 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:43:54.809668 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.126:46073 (local address 127.22.167.65:48707)
0810 04:43:54.809788 (+ 120us) negotiation.cc:107] Waiting for socket to connect
0810 04:43:54.809823 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.167.126:46073: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":104,"thread_start_us":71,"threads_started":1}
I20260810 04:43:54.883066 16759 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:43:54.882880 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.126:46073 (local address 127.22.167.66:52739)
0810 04:43:54.882986 (+ 106us) negotiation.cc:107] Waiting for socket to connect
0810 04:43:54.883021 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.167.126:46073: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":92,"thread_start_us":58,"threads_started":1}
I20260810 04:43:55.013010 16760 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:43:55.012801 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.126:46073 (local address 127.22.167.67:48233)
0810 04:43:55.012936 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0810 04:43:55.012971 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.167.126:46073: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":78,"threads_started":1}
I20260810 04:43:55.810534 16761 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:43:55.810319 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.126:46073 (local address 127.22.167.65:47873)
0810 04:43:55.810448 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0810 04:43:55.810485 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.167.126:46073: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":64,"threads_started":1}
I20260810 04:43:55.883634 16768 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:43:55.883441 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.126:46073 (local address 127.22.167.66:35771)
0810 04:43:55.883556 (+ 115us) negotiation.cc:107] Waiting for socket to connect
0810 04:43:55.883591 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.167.126:46073: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":101,"thread_start_us":62,"threads_started":1}
I20260810 04:43:56.013622 16770 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:43:56.013421 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.126:46073 (local address 127.22.167.67:54517)
0810 04:43:56.013540 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0810 04:43:56.013576 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.167.126:46073: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":104,"thread_start_us":62,"threads_started":1}
I20260810 04:43:56.811141 16771 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:43:56.810939 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.126:46073 (local address 127.22.167.65:56841)
0810 04:43:56.811057 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0810 04:43:56.811090 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.167.126:46073: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":102,"thread_start_us":67,"threads_started":1}
I20260810 04:43:56.884178 16774 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:43:56.883978 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.126:46073 (local address 127.22.167.66:48881)
0810 04:43:56.884097 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0810 04:43:56.884134 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.167.126:46073: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":102,"thread_start_us":61,"threads_started":1}
I20260810 04:43:57.014228 16775 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:43:57.013992 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.126:46073 (local address 127.22.167.67:49163)
0810 04:43:57.014149 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0810 04:43:57.014189 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.167.126:46073: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":80,"threads_started":1}
I20260810 04:43:57.811765 16776 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:43:57.811563 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.126:46073 (local address 127.22.167.65:42635)
0810 04:43:57.811674 (+ 111us) negotiation.cc:107] Waiting for socket to connect
0810 04:43:57.811715 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.167.126:46073: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":96,"thread_start_us":62,"threads_started":1}
I20260810 04:43:57.884796 16779 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:43:57.884557 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.126:46073 (local address 127.22.167.66:45605)
0810 04:43:57.884714 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0810 04:43:57.884752 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.167.126:46073: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":81,"threads_started":1}
I20260810 04:43:58.014793 16780 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:43:58.014556 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.126:46073 (local address 127.22.167.67:54005)
0810 04:43:58.014713 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0810 04:43:58.014751 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.167.126:46073: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":77,"threads_started":1}
I20260810 04:43:58.812381 16781 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:43:58.812176 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.126:46073 (local address 127.22.167.65:46547)
0810 04:43:58.812300 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0810 04:43:58.812333 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.167.126:46073: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":64,"threads_started":1}
I20260810 04:43:58.885419 16782 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:43:58.885209 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.126:46073 (local address 127.22.167.66:59333)
0810 04:43:58.885346 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0810 04:43:58.885379 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.167.126:46073: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":86,"threads_started":1}
I20260810 04:43:59.015475 16783 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:43:59.015246 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.126:46073 (local address 127.22.167.67:44411)
0810 04:43:59.015403 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0810 04:43:59.015433 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.167.126:46073: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":96,"threads_started":1}
I20260810 04:43:59.680255 16788 raft_consensus.cc:482] T 416edda2678e425d9c0c87ed4f56cf2a P ed99582b3b574c00991af0deb19f27c1 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 04:43:59.680351 16788 raft_consensus.cc:504] T 416edda2678e425d9c0c87ed4f56cf2a P ed99582b3b574c00991af0deb19f27c1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "61a9f79165b8438299946a8f4b034178" member_type: VOTER last_known_addr { host: "127.22.167.66" port: 38979 } } peers { permanent_uuid: "3a46d62cc08a49938add205cef6d55aa" member_type: VOTER last_known_addr { host: "127.22.167.67" port: 39865 } } peers { permanent_uuid: "ed99582b3b574c00991af0deb19f27c1" member_type: VOTER last_known_addr { host: "127.22.167.65" port: 44335 } }
I20260810 04:43:59.680516 16788 leader_election.cc:288] T 416edda2678e425d9c0c87ed4f56cf2a P ed99582b3b574c00991af0deb19f27c1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 61a9f79165b8438299946a8f4b034178 (127.22.167.66:38979), 3a46d62cc08a49938add205cef6d55aa (127.22.167.67:39865)
W20260810 04:43:59.680892 15983 leader_election.cc:341] T 416edda2678e425d9c0c87ed4f56cf2a P ed99582b3b574c00991af0deb19f27c1 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 61a9f79165b8438299946a8f4b034178 (127.22.167.66:38979): Not found: Tablet not found: 416edda2678e425d9c0c87ed4f56cf2a
W20260810 04:43:59.680996 15990 leader_election.cc:341] T 416edda2678e425d9c0c87ed4f56cf2a P ed99582b3b574c00991af0deb19f27c1 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 3a46d62cc08a49938add205cef6d55aa (127.22.167.67:39865): Not found: Tablet not found: 416edda2678e425d9c0c87ed4f56cf2a
I20260810 04:43:59.681071 15990 leader_election.cc:302] T 416edda2678e425d9c0c87ed4f56cf2a P ed99582b3b574c00991af0deb19f27c1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ed99582b3b574c00991af0deb19f27c1; no voters: 3a46d62cc08a49938add205cef6d55aa, 61a9f79165b8438299946a8f4b034178
I20260810 04:43:59.681151 16788 raft_consensus.cc:2726] T 416edda2678e425d9c0c87ed4f56cf2a P ed99582b3b574c00991af0deb19f27c1 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260810 04:43:59.813030 16789 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:43:59.812804 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.126:46073 (local address 127.22.167.65:38013)
0810 04:43:59.812936 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0810 04:43:59.812976 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.167.126:46073: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":74,"threads_started":1}
I20260810 04:43:59.885972 16791 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:43:59.885765 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.126:46073 (local address 127.22.167.66:59561)
0810 04:43:59.885891 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0810 04:43:59.885929 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.167.126:46073: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":76,"threads_started":1}
I20260810 04:44:00.016170 16792 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:44:00.015933 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.126:46073 (local address 127.22.167.67:34349)
0810 04:44:00.016079 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0810 04:44:00.016121 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.167.126:46073: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":87,"threads_started":1}
I20260810 04:44:00.813742 16793 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:44:00.813509 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.126:46073 (local address 127.22.167.65:34341)
0810 04:44:00.813648 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0810 04:44:00.813689 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.167.126:46073: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":62,"threads_started":1}
I20260810 04:44:00.886663 16794 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:44:00.886400 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.126:46073 (local address 127.22.167.66:34925)
0810 04:44:00.886541 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0810 04:44:00.886604 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.167.126:46073: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":65,"threads_started":1}
I20260810 04:44:01.011705 23197 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskemv_AM/build/release/bin/kudu with pid 15909
I20260810 04:44:01.018827 16795 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:44:01.016756 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.126:46073 (local address 127.22.167.67:55501)
0810 04:44:01.018728 (+ 1972us) negotiation.cc:107] Waiting for socket to connect
0810 04:44:01.018773 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.167.126:46073: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1953,"thread_start_us":1838,"threads_started":1}
I20260810 04:44:01.026268 23197 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskemv_AM/build/release/bin/kudu with pid 16200
I20260810 04:44:01.047060 23197 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskemv_AM/build/release/bin/kudu with pid 16456
2026-08-10T04:44:01Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedIPKICert (18463 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedTSK
Loading random data
Initializing database '/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786336863388938-23197-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 04:44:01 dist-test-slave-bqcl krb5kdc[16801](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 04:44:01 dist-test-slave-bqcl krb5kdc[16801](info): set up 2 sockets
Aug 10 04:44:01 dist-test-slave-bqcl krb5kdc[16801](info): commencing operation
krb5kdc: starting...
W20260810 04:44:02.208271 23197 mini_kdc.cc:121] Time spent starting KDC: real 1.137s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786336863388938-23197-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786336863388938-23197-0/krb5kdc/test-user.keytab.
Aug 10 04:44:02 dist-test-slave-bqcl krb5kdc[16801](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786337042, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T04:44:02Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T04:44:02Z Disabled control of system clock
WARNING: no policy specified for kudu/127.22.167.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.167.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.167.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786336863388938-23197-0/krb5kdc/kudu_127.22.167.126.keytab.
Entry for principal kudu/127.22.167.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786336863388938-23197-0/krb5kdc/kudu_127.22.167.126.keytab.
I20260810 04:44:02.325090 23197 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskemv_AM/build/release/bin/kudu
/tmp/dist-test-taskemv_AM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786336863388938-23197-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786336863388938-23197-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786336863388938-23197-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786336863388938-23197-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.167.126:38209
--webserver_interface=127.22.167.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786336863388938-23197-0/krb5kdc/kudu_127.22.167.126.keytab
--principal=kudu/127.22.167.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.167.84:33205
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.167.126:38209
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786336863388938-23197-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786336863388938-23197-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786336863388938-23197-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 04:44:02.407346 16830 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 04:44:02.407567 16830 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 04:44:02.407589 16830 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 04:44:02.408666 16830 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260810 04:44:02.408704 16830 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260810 04:44:02.408718 16830 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260810 04:44:02.408730 16830 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 04:44:02.408756 16830 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 04:44:02.408771 16830 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 04:44:02.408789 16830 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 04:44:02.408802 16830 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 04:44:02.410226 16830 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.167.84:33205
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786336863388938-23197-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786336863388938-23197-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.22.167.126:38209
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786336863388938-23197-0/krb5kdc/kudu_127.22.167.126.keytab
--principal=kudu/127.22.167.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.167.126:38209
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786336863388938-23197-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.167.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16830
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786336863388938-23197-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 03:54:01 UTC on ccycloud.quasar-icijmt.root.comops.site
build id 4577009
I20260810 04:44:02.410424 16830 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 04:44:02.410609 16830 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 04:44:02.415436 16837 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 04:44:02.415421 16841 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 04:44:02.415416 16830 server_base.cc:1034] running on GCE node
W20260810 04:44:02.415398 16836 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 04:44:02.415915 16830 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 04:44:02.416204 16830 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 04:44:02.417341 16830 hybrid_clock.cc:648] HybridClock initialized: now 1786337042417331 us; error 29 us; skew 500 ppm
Aug 10 04:44:02 dist-test-slave-bqcl krb5kdc[16801](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786337042, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.167.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 04:44:02.419782 16830 init.cc:377] Logged in from keytab as kudu/127.22.167.126@KRBTEST.COM (short username kudu)
I20260810 04:44:02.420202 16830 webserver.cc:460] Webserver started at http://127.22.167.126:35293/ using document root <none> and password file <none>
I20260810 04:44:02.420392 16830 fs_manager.cc:359] Metadata directory not provided
I20260810 04:44:02.420439 16830 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 04:44:02.420567 16830 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 04:44:02.421518 16830 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786336863388938-23197-0/minicluster-data/master-0/data/instance:
uuid: "f49432ad9f9f4c31b0380574d0265634"
format_stamp: "Formatted at 2026-08-10 04:44:02 on dist-test-slave-bqcl"
server_key: "934a4a74932f95d0458d6d58d3085223"
server_key_iv: "995b6f0cd97aa1961bf7aa35fd595378"
server_key_version: "encryptionkey@0"
I20260810 04:44:02.421846 16830 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786336863388938-23197-0/minicluster-data/master-0/wal/instance:
uuid: "f49432ad9f9f4c31b0380574d0265634"
format_stamp: "Formatted at 2026-08-10 04:44:02 on dist-test-slave-bqcl"
server_key: "934a4a74932f95d0458d6d58d3085223"
server_key_iv: "995b6f0cd97aa1961bf7aa35fd595378"
server_key_version: "encryptionkey@0"
I20260810 04:44:02.423110 16830 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260810 04:44:02.423763 16851 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 04:44:02.423910 16830 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260810 04:44:02.423996 16830 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786336863388938-23197-0/minicluster-data/master-0/data,/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786336863388938-23197-0/minicluster-data/master-0/wal
uuid: "f49432ad9f9f4c31b0380574d0265634"
format_stamp: "Formatted at 2026-08-10 04:44:02 on dist-test-slave-bqcl"
server_key: "934a4a74932f95d0458d6d58d3085223"
server_key_iv: "995b6f0cd97aa1961bf7aa35fd595378"
server_key_version: "encryptionkey@0"
I20260810 04:44:02.424063 16830 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786336863388938-23197-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786336863388938-23197-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786336863388938-23197-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 04:44:02.435783 16830 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 04:44:02.437834 16830 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 04:44:02.437961 16830 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 04:44:02.441798 16830 rpc_server.cc:307] RPC server started. Bound to: 127.22.167.126:38209
I20260810 04:44:02.441833 16938 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.167.126:38209 every 8 connection(s)
I20260810 04:44:02.442164 16830 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786336863388938-23197-0/minicluster-data/master-0/data/info.pb
I20260810 04:44:02.442979 16939 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 04:44:02.445210 16939 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f49432ad9f9f4c31b0380574d0265634: Bootstrap starting.
I20260810 04:44:02.445791 16939 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f49432ad9f9f4c31b0380574d0265634: Neither blocks nor log segments found. Creating new log.
I20260810 04:44:02.446069 16939 log.cc:824] T 00000000000000000000000000000000 P f49432ad9f9f4c31b0380574d0265634: Log is configured to *not* fsync() on all Append() calls
I20260810 04:44:02.446720 16939 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f49432ad9f9f4c31b0380574d0265634: No bootstrap required, opened a new log
I20260810 04:44:02.448244 16939 raft_consensus.cc:348] T 00000000000000000000000000000000 P f49432ad9f9f4c31b0380574d0265634 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f49432ad9f9f4c31b0380574d0265634" member_type: VOTER last_known_addr { host: "127.22.167.126" port: 38209 } }
I20260810 04:44:02.448380 16939 raft_consensus.cc:374] T 00000000000000000000000000000000 P f49432ad9f9f4c31b0380574d0265634 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 04:44:02.448403 16939 raft_consensus.cc:729] T 00000000000000000000000000000000 P f49432ad9f9f4c31b0380574d0265634 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f49432ad9f9f4c31b0380574d0265634, State: Initialized, Role: FOLLOWER
I20260810 04:44:02.448539 16939 consensus_queue.cc:260] T 00000000000000000000000000000000 P f49432ad9f9f4c31b0380574d0265634 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f49432ad9f9f4c31b0380574d0265634" member_type: VOTER last_known_addr { host: "127.22.167.126" port: 38209 } }
I20260810 04:44:02.448599 16939 raft_consensus.cc:388] T 00000000000000000000000000000000 P f49432ad9f9f4c31b0380574d0265634 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 04:44:02.448632 16939 raft_consensus.cc:482] T 00000000000000000000000000000000 P f49432ad9f9f4c31b0380574d0265634 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 04:44:02.448685 16939 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f49432ad9f9f4c31b0380574d0265634 [term 0 FOLLOWER]: Advancing to term 1
I20260810 04:44:02.449182 16939 raft_consensus.cc:504] T 00000000000000000000000000000000 P f49432ad9f9f4c31b0380574d0265634 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f49432ad9f9f4c31b0380574d0265634" member_type: VOTER last_known_addr { host: "127.22.167.126" port: 38209 } }
I20260810 04:44:02.449275 16939 leader_election.cc:302] T 00000000000000000000000000000000 P f49432ad9f9f4c31b0380574d0265634 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f49432ad9f9f4c31b0380574d0265634; no voters:
I20260810 04:44:02.449455 16939 leader_election.cc:288] T 00000000000000000000000000000000 P f49432ad9f9f4c31b0380574d0265634 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 04:44:02.449491 16948 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f49432ad9f9f4c31b0380574d0265634 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 04:44:02.449575 23197 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskemv_AM/build/release/bin/kudu as pid 16830
I20260810 04:44:02.449627 16948 raft_consensus.cc:686] T 00000000000000000000000000000000 P f49432ad9f9f4c31b0380574d0265634 [term 1 LEADER]: Becoming Leader. State: Replica: f49432ad9f9f4c31b0380574d0265634, State: Running, Role: LEADER
I20260810 04:44:02.449673 23197 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786336863388938-23197-0/minicluster-data/master-0/wal/instance
I20260810 04:44:02.449720 16939 sys_catalog.cc:564] T 00000000000000000000000000000000 P f49432ad9f9f4c31b0380574d0265634 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 04:44:02.449864 23197 external_mini_cluster.cc:1428] Setting key b960605eb905bffa6fa74772f9227809
I20260810 04:44:02.449702 16948 consensus_queue.cc:237] T 00000000000000000000000000000000 P f49432ad9f9f4c31b0380574d0265634 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f49432ad9f9f4c31b0380574d0265634" member_type: VOTER last_known_addr { host: "127.22.167.126" port: 38209 } }
I20260810 04:44:02.450778 16950 sys_catalog.cc:455] T 00000000000000000000000000000000 P f49432ad9f9f4c31b0380574d0265634 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f49432ad9f9f4c31b0380574d0265634. Latest consensus state: current_term: 1 leader_uuid: "f49432ad9f9f4c31b0380574d0265634" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f49432ad9f9f4c31b0380574d0265634" member_type: VOTER last_known_addr { host: "127.22.167.126" port: 38209 } } }
I20260810 04:44:02.450875 16950 sys_catalog.cc:458] T 00000000000000000000000000000000 P f49432ad9f9f4c31b0380574d0265634 [sys.catalog]: This master's current role is: LEADER
I20260810 04:44:02.450913 16951 sys_catalog.cc:455] T 00000000000000000000000000000000 P f49432ad9f9f4c31b0380574d0265634 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f49432ad9f9f4c31b0380574d0265634" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f49432ad9f9f4c31b0380574d0265634" member_type: VOTER last_known_addr { host: "127.22.167.126" port: 38209 } } }
I20260810 04:44:02.450990 16951 sys_catalog.cc:458] T 00000000000000000000000000000000 P f49432ad9f9f4c31b0380574d0265634 [sys.catalog]: This master's current role is: LEADER
W20260810 04:44:02.451612 16971 catalog_manager.cc:1558] T 00000000000000000000000000000000 P f49432ad9f9f4c31b0380574d0265634: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260810 04:44:02.451663 16971 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260810 04:44:02.451949 16968 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 04:44:02.452131 16968 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 04:44:02.453661 16968 catalog_manager.cc:1347] Generated new cluster ID: bd36be342a8e4f998163ba9d6f1c530e
I20260810 04:44:02.453723 16968 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 10 04:44:02 dist-test-slave-bqcl krb5kdc[16801](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786337042, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.167.126@KRBTEST.COM
I20260810 04:44:02.461066 16959 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:44:02.450344 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54380 (local address 127.22.167.126:38209)
0810 04:44:02.450553 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0810 04:44:02.450564 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0810 04:44:02.450598 (+ 34us) server_negotiation.cc:407] Connection header received
0810 04:44:02.450761 (+ 163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:44:02.450768 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:44:02.450838 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:44:02.450980 (+ 142us) server_negotiation.cc:226] Negotiated authn=SASL
0810 04:44:02.451934 (+ 954us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:44:02.453172 (+ 1238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:44:02.454206 (+ 1034us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:44:02.454890 (+ 684us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:44:02.455007 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:44:02.457664 (+ 2657us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 04:44:02.457668 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 04:44:02.457669 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 04:44:02.457680 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0810 04:44:02.459542 (+ 1862us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 04:44:02.459925 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 04:44:02.459927 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 04:44:02.459927 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 04:44:02.460040 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 04:44:02.460232 (+ 192us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 04:44:02.460233 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 04:44:02.460233 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 04:44:02.460495 (+ 262us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 04:44:02.460639 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0810 04:44:02.460739 (+ 100us) server_negotiation.cc:299] Negotiation successful
0810 04:44:02.460863 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":164,"thread_start_us":103,"threads_started":1}
I20260810 04:44:02.464939 16968 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 04:44:02.465623 16968 catalog_manager.cc:1504] Loading token signing keys...
I20260810 04:44:02.481207 16968 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f49432ad9f9f4c31b0380574d0265634: Generated new TSK 0
I20260810 04:44:02.481403 16968 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.22.167.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.167.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.167.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786336863388938-23197-0/krb5kdc/kudu_127.22.167.65.keytab.
Entry for principal kudu/127.22.167.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786336863388938-23197-0/krb5kdc/kudu_127.22.167.65.keytab.
I20260810 04:44:02.516760 23197 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskemv_AM/build/release/bin/kudu
/tmp/dist-test-taskemv_AM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786336863388938-23197-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786336863388938-23197-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786336863388938-23197-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786336863388938-23197-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.167.65:0
--local_ip_for_outbound_sockets=127.22.167.65
--webserver_interface=127.22.167.65
--webserver_port=0
--tserver_master_addrs=127.22.167.126:38209
--keytab_file=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786336863388938-23197-0/krb5kdc/kudu_127.22.167.65.keytab
--principal=kudu/127.22.167.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.167.84:33205
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786336863388938-23197-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786336863388938-23197-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786336863388938-23197-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 04:44:02.623518 16985 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 04:44:02.623721 16985 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 04:44:02.623752 16985 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 04:44:02.624840 16985 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 04:44:02.624881 16985 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 04:44:02.624913 16985 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.167.65
I20260810 04:44:02.626405 16985 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.167.84:33205
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786336863388938-23197-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786336863388938-23197-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786336863388938-23197-0/krb5kdc/kudu_127.22.167.65.keytab
--principal=kudu/127.22.167.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.167.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786336863388938-23197-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.22.167.65
--webserver_port=0
--tserver_master_addrs=127.22.167.126:38209
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16985
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.167.65
--log_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786336863388938-23197-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 03:54:01 UTC on ccycloud.quasar-icijmt.root.comops.site
build id 4577009
I20260810 04:44:02.626667 16985 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 04:44:02.627086 16985 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 04:44:02.632085 16990 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 04:44:02.632143 16994 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 04:44:02.632179 16985 server_base.cc:1034] running on GCE node
W20260810 04:44:02.632107 16996 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 04:44:02.632465 16985 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 04:44:02.632668 16985 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 04:44:02.633800 16985 hybrid_clock.cc:648] HybridClock initialized: now 1786337042633788 us; error 28 us; skew 500 ppm
Aug 10 04:44:02 dist-test-slave-bqcl krb5kdc[16801](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786337042, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.167.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 04:44:02.636452 16985 init.cc:377] Logged in from keytab as kudu/127.22.167.65@KRBTEST.COM (short username kudu)
I20260810 04:44:02.636848 16985 webserver.cc:460] Webserver started at http://127.22.167.65:38707/ using document root <none> and password file <none>
I20260810 04:44:02.637063 16985 fs_manager.cc:359] Metadata directory not provided
I20260810 04:44:02.637125 16985 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 04:44:02.637243 16985 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 04:44:02.638226 16985 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786336863388938-23197-0/minicluster-data/ts-0/data/instance:
uuid: "7e7834fc8be34efcb685d3a20756d41b"
format_stamp: "Formatted at 2026-08-10 04:44:02 on dist-test-slave-bqcl"
server_key: "eb7825fd5d6485fca2190b72dbfdb5f6"
server_key_iv: "9228f8ec69eec8ecfc92f4d36d095375"
server_key_version: "encryptionkey@0"
I20260810 04:44:02.638521 16985 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786336863388938-23197-0/minicluster-data/ts-0/wal/instance:
uuid: "7e7834fc8be34efcb685d3a20756d41b"
format_stamp: "Formatted at 2026-08-10 04:44:02 on dist-test-slave-bqcl"
server_key: "eb7825fd5d6485fca2190b72dbfdb5f6"
server_key_iv: "9228f8ec69eec8ecfc92f4d36d095375"
server_key_version: "encryptionkey@0"
I20260810 04:44:02.639711 16985 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260810 04:44:02.640326 17019 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 04:44:02.640489 16985 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260810 04:44:02.640585 16985 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786336863388938-23197-0/minicluster-data/ts-0/data,/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786336863388938-23197-0/minicluster-data/ts-0/wal
uuid: "7e7834fc8be34efcb685d3a20756d41b"
format_stamp: "Formatted at 2026-08-10 04:44:02 on dist-test-slave-bqcl"
server_key: "eb7825fd5d6485fca2190b72dbfdb5f6"
server_key_iv: "9228f8ec69eec8ecfc92f4d36d095375"
server_key_version: "encryptionkey@0"
I20260810 04:44:02.640641 16985 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786336863388938-23197-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786336863388938-23197-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786336863388938-23197-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 04:44:02.649365 16985 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 04:44:02.651595 16985 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 04:44:02.651733 16985 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 04:44:02.651943 16985 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 04:44:02.652213 16985 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 04:44:02.652246 16985 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 04:44:02.652266 16985 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 04:44:02.652289 16985 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 04:44:02.657625 16985 rpc_server.cc:307] RPC server started. Bound to: 127.22.167.65:37211
I20260810 04:44:02.657708 17227 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.167.65:37211 every 8 connection(s)
I20260810 04:44:02.657956 16985 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786336863388938-23197-0/minicluster-data/ts-0/data/info.pb
I20260810 04:44:02.662019 23197 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskemv_AM/build/release/bin/kudu as pid 16985
I20260810 04:44:02.662106 23197 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786336863388938-23197-0/minicluster-data/ts-0/wal/instance
I20260810 04:44:02.662326 23197 external_mini_cluster.cc:1428] Setting key c1520fd7774eafd688332158f1d79fdc
Aug 10 04:44:02 dist-test-slave-bqcl krb5kdc[16801](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786337042, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.167.65@KRBTEST.COM for kudu/127.22.167.126@KRBTEST.COM
I20260810 04:44:02.666872 16959 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:44:02.658599 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.167.65:60877 (local address 127.22.167.126:38209)
0810 04:44:02.658639 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0810 04:44:02.658642 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 04:44:02.658827 (+ 185us) server_negotiation.cc:407] Connection header received
0810 04:44:02.659313 (+ 486us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:44:02.659315 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:44:02.659331 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:44:02.659382 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0810 04:44:02.660280 (+ 898us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:44:02.660780 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:44:02.661743 (+ 963us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:44:02.661971 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:44:02.662025 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:44:02.664243 (+ 2218us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 04:44:02.664244 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 04:44:02.664245 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 04:44:02.664252 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 04:44:02.665511 (+ 1259us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 04:44:02.665965 (+ 454us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 04:44:02.665966 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 04:44:02.665966 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 04:44:02.666015 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 04:44:02.666279 (+ 264us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 04:44:02.666280 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 04:44:02.666280 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 04:44:02.666384 (+ 104us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 04:44:02.666435 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0810 04:44:02.666740 (+ 305us) server_negotiation.cc:299] Negotiation successful
0810 04:44:02.666808 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260810 04:44:02.667366 17232 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:44:02.658563 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.126:38209 (local address 127.22.167.65:60877)
0810 04:44:02.658716 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0810 04:44:02.658751 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0810 04:44:02.659215 (+ 464us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 04:44:02.659451 (+ 236us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 04:44:02.659455 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 04:44:02.660000 (+ 545us) client_negotiation.cc:189] Negotiated authn=SASL
0810 04:44:02.660183 (+ 183us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:44:02.660189 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:44:02.660862 (+ 673us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:44:02.660864 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:44:02.661653 (+ 789us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:44:02.661655 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:44:02.662043 (+ 388us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:44:02.662043 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:44:02.662209 (+ 166us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:44:02.662702 (+ 493us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 04:44:02.662711 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0810 04:44:02.664154 (+ 1443us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 04:44:02.665588 (+ 1434us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 04:44:02.665595 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 04:44:02.665597 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 04:44:02.665900 (+ 303us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 04:44:02.666064 (+ 164us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 04:44:02.666065 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 04:44:02.666065 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 04:44:02.666223 (+ 158us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 04:44:02.666440 (+ 217us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 04:44:02.666447 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 04:44:02.666603 (+ 156us) client_negotiation.cc:769] Sending connection context
0810 04:44:02.666715 (+ 112us) client_negotiation.cc:240] Negotiation successful
0810 04:44:02.666875 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":101,"thread_start_us":69,"threads_started":1}
I20260810 04:44:02.668922 17228 heartbeater.cc:347] Connected to a master server at 127.22.167.126:38209
I20260810 04:44:02.669036 17228 heartbeater.cc:464] Registering TS with master...
I20260810 04:44:02.669235 17228 heartbeater.cc:511] Master 127.22.167.126:38209 requested a full tablet report, sending...
I20260810 04:44:02.669893 16884 ts_manager.cc:194] Registered new tserver with Master: 7e7834fc8be34efcb685d3a20756d41b (127.22.167.65:37211)
I20260810 04:44:02.670830 16884 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.167.65@KRBTEST.COM'} at 127.22.167.65:60877
WARNING: no policy specified for kudu/127.22.167.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.167.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.167.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786336863388938-23197-0/krb5kdc/kudu_127.22.167.66.keytab.
Entry for principal kudu/127.22.167.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786336863388938-23197-0/krb5kdc/kudu_127.22.167.66.keytab.
I20260810 04:44:02.688232 23197 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskemv_AM/build/release/bin/kudu
/tmp/dist-test-taskemv_AM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786336863388938-23197-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786336863388938-23197-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786336863388938-23197-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786336863388938-23197-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.167.66:0
--local_ip_for_outbound_sockets=127.22.167.66
--webserver_interface=127.22.167.66
--webserver_port=0
--tserver_master_addrs=127.22.167.126:38209
--keytab_file=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786336863388938-23197-0/krb5kdc/kudu_127.22.167.66.keytab
--principal=kudu/127.22.167.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.167.84:33205
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786336863388938-23197-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786336863388938-23197-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786336863388938-23197-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 04:44:02.772394 17239 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 04:44:02.772593 17239 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 04:44:02.772614 17239 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 04:44:02.773689 17239 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 04:44:02.773721 17239 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 04:44:02.773746 17239 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.167.66
I20260810 04:44:02.775125 17239 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.167.84:33205
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786336863388938-23197-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786336863388938-23197-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786336863388938-23197-0/krb5kdc/kudu_127.22.167.66.keytab
--principal=kudu/127.22.167.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.167.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786336863388938-23197-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.22.167.66
--webserver_port=0
--tserver_master_addrs=127.22.167.126:38209
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17239
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.167.66
--log_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786336863388938-23197-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 03:54:01 UTC on ccycloud.quasar-icijmt.root.comops.site
build id 4577009
I20260810 04:44:02.775315 17239 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 04:44:02.775483 17239 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 04:44:02.780222 17246 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 04:44:02.780259 17248 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 04:44:02.780448 17245 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 04:44:02.780431 17239 server_base.cc:1034] running on GCE node
I20260810 04:44:02.780658 17239 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 04:44:02.780885 17239 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 04:44:02.782027 17239 hybrid_clock.cc:648] HybridClock initialized: now 1786337042782011 us; error 27 us; skew 500 ppm
Aug 10 04:44:02 dist-test-slave-bqcl krb5kdc[16801](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786337042, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.167.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 04:44:02.784514 17239 init.cc:377] Logged in from keytab as kudu/127.22.167.66@KRBTEST.COM (short username kudu)
I20260810 04:44:02.784957 17239 webserver.cc:460] Webserver started at http://127.22.167.66:35009/ using document root <none> and password file <none>
I20260810 04:44:02.785151 17239 fs_manager.cc:359] Metadata directory not provided
I20260810 04:44:02.785211 17239 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 04:44:02.785356 17239 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 04:44:02.786420 17239 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786336863388938-23197-0/minicluster-data/ts-1/data/instance:
uuid: "6ee21ed53fbc49d284bbecab757bb9db"
format_stamp: "Formatted at 2026-08-10 04:44:02 on dist-test-slave-bqcl"
server_key: "b47c454c9f14a9d83f9fcc747b87c8e1"
server_key_iv: "454b1c4173200cffd387da3c134da7d4"
server_key_version: "encryptionkey@0"
I20260810 04:44:02.786751 17239 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786336863388938-23197-0/minicluster-data/ts-1/wal/instance:
uuid: "6ee21ed53fbc49d284bbecab757bb9db"
format_stamp: "Formatted at 2026-08-10 04:44:02 on dist-test-slave-bqcl"
server_key: "b47c454c9f14a9d83f9fcc747b87c8e1"
server_key_iv: "454b1c4173200cffd387da3c134da7d4"
server_key_version: "encryptionkey@0"
I20260810 04:44:02.787971 17239 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260810 04:44:02.788625 17257 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 04:44:02.788772 17239 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260810 04:44:02.788854 17239 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786336863388938-23197-0/minicluster-data/ts-1/data,/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786336863388938-23197-0/minicluster-data/ts-1/wal
uuid: "6ee21ed53fbc49d284bbecab757bb9db"
format_stamp: "Formatted at 2026-08-10 04:44:02 on dist-test-slave-bqcl"
server_key: "b47c454c9f14a9d83f9fcc747b87c8e1"
server_key_iv: "454b1c4173200cffd387da3c134da7d4"
server_key_version: "encryptionkey@0"
I20260810 04:44:02.788915 17239 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786336863388938-23197-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786336863388938-23197-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786336863388938-23197-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 04:44:02.802963 17239 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 04:44:02.804934 17239 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 04:44:02.805063 17239 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 04:44:02.805274 17239 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 04:44:02.805548 17239 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 04:44:02.805581 17239 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 04:44:02.805608 17239 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 04:44:02.805631 17239 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 04:44:02.811108 17239 rpc_server.cc:307] RPC server started. Bound to: 127.22.167.66:45913
I20260810 04:44:02.811172 17454 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.167.66:45913 every 8 connection(s)
I20260810 04:44:02.811437 17239 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786336863388938-23197-0/minicluster-data/ts-1/data/info.pb
I20260810 04:44:02.812896 23197 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskemv_AM/build/release/bin/kudu as pid 17239
I20260810 04:44:02.812990 23197 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786336863388938-23197-0/minicluster-data/ts-1/wal/instance
I20260810 04:44:02.813185 23197 external_mini_cluster.cc:1428] Setting key 9e566f66b53e83f215b5e65e51ade2cb
Aug 10 04:44:02 dist-test-slave-bqcl krb5kdc[16801](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786337042, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.167.66@KRBTEST.COM for kudu/127.22.167.126@KRBTEST.COM
I20260810 04:44:02.820463 16959 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:44:02.812072 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.167.66:59233 (local address 127.22.167.126:38209)
0810 04:44:02.812124 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0810 04:44:02.812126 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 04:44:02.812310 (+ 184us) server_negotiation.cc:407] Connection header received
0810 04:44:02.812775 (+ 465us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:44:02.812776 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:44:02.812788 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:44:02.812866 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0810 04:44:02.813852 (+ 986us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:44:02.814522 (+ 670us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:44:02.815484 (+ 962us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:44:02.815676 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:44:02.815731 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:44:02.817865 (+ 2134us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 04:44:02.817866 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 04:44:02.817867 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 04:44:02.817873 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0810 04:44:02.819123 (+ 1250us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 04:44:02.819577 (+ 454us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 04:44:02.819578 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 04:44:02.819578 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 04:44:02.819628 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 04:44:02.819881 (+ 253us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 04:44:02.819882 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 04:44:02.819882 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 04:44:02.819980 (+ 98us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 04:44:02.820026 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0810 04:44:02.820339 (+ 313us) server_negotiation.cc:299] Negotiation successful
0810 04:44:02.820401 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260810 04:44:02.820739 17459 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:44:02.812036 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.126:38209 (local address 127.22.167.66:59233)
0810 04:44:02.812198 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0810 04:44:02.812239 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0810 04:44:02.812696 (+ 457us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 04:44:02.812923 (+ 227us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 04:44:02.812927 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 04:44:02.813576 (+ 649us) client_negotiation.cc:189] Negotiated authn=SASL
0810 04:44:02.813777 (+ 201us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:44:02.813784 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:44:02.814603 (+ 819us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:44:02.814603 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:44:02.815396 (+ 793us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:44:02.815399 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:44:02.815743 (+ 344us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:44:02.815744 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:44:02.815868 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:44:02.816292 (+ 424us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 04:44:02.816300 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0810 04:44:02.817780 (+ 1480us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 04:44:02.819203 (+ 1423us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 04:44:02.819212 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 04:44:02.819214 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 04:44:02.819507 (+ 293us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 04:44:02.819685 (+ 178us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 04:44:02.819686 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 04:44:02.819687 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 04:44:02.819824 (+ 137us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 04:44:02.820034 (+ 210us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 04:44:02.820040 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 04:44:02.820216 (+ 176us) client_negotiation.cc:769] Sending connection context
0810 04:44:02.820331 (+ 115us) client_negotiation.cc:240] Negotiation successful
0810 04:44:02.820493 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":76,"threads_started":1}
I20260810 04:44:02.821240 17455 heartbeater.cc:347] Connected to a master server at 127.22.167.126:38209
I20260810 04:44:02.821347 17455 heartbeater.cc:464] Registering TS with master...
I20260810 04:44:02.821537 17455 heartbeater.cc:511] Master 127.22.167.126:38209 requested a full tablet report, sending...
I20260810 04:44:02.821895 16884 ts_manager.cc:194] Registered new tserver with Master: 6ee21ed53fbc49d284bbecab757bb9db (127.22.167.66:45913)
I20260810 04:44:02.822283 16884 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.167.66@KRBTEST.COM'} at 127.22.167.66:59233
WARNING: no policy specified for kudu/127.22.167.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.167.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.167.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786336863388938-23197-0/krb5kdc/kudu_127.22.167.67.keytab.
Entry for principal kudu/127.22.167.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786336863388938-23197-0/krb5kdc/kudu_127.22.167.67.keytab.
I20260810 04:44:02.838788 23197 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskemv_AM/build/release/bin/kudu
/tmp/dist-test-taskemv_AM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786336863388938-23197-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786336863388938-23197-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786336863388938-23197-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786336863388938-23197-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.167.67:0
--local_ip_for_outbound_sockets=127.22.167.67
--webserver_interface=127.22.167.67
--webserver_port=0
--tserver_master_addrs=127.22.167.126:38209
--keytab_file=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786336863388938-23197-0/krb5kdc/kudu_127.22.167.67.keytab
--principal=kudu/127.22.167.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.167.84:33205
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786336863388938-23197-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786336863388938-23197-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786336863388938-23197-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 04:44:02.919168 17465 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 04:44:02.919371 17465 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 04:44:02.919394 17465 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 04:44:02.920461 17465 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 04:44:02.920493 17465 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 04:44:02.920518 17465 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.167.67
I20260810 04:44:02.921900 17465 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.167.84:33205
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786336863388938-23197-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786336863388938-23197-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786336863388938-23197-0/krb5kdc/kudu_127.22.167.67.keytab
--principal=kudu/127.22.167.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.167.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786336863388938-23197-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.22.167.67
--webserver_port=0
--tserver_master_addrs=127.22.167.126:38209
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17465
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.167.67
--log_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786336863388938-23197-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 03:54:01 UTC on ccycloud.quasar-icijmt.root.comops.site
build id 4577009
I20260810 04:44:02.922083 17465 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 04:44:02.922305 17465 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 04:44:02.927062 17479 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 04:44:02.927079 17475 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 04:44:02.927078 17476 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 04:44:02.927371 17465 server_base.cc:1034] running on GCE node
I20260810 04:44:02.927562 17465 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 04:44:02.927757 17465 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 04:44:02.928911 17465 hybrid_clock.cc:648] HybridClock initialized: now 1786337042928899 us; error 30 us; skew 500 ppm
Aug 10 04:44:02 dist-test-slave-bqcl krb5kdc[16801](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786337042, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.167.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 04:44:02.931447 17465 init.cc:377] Logged in from keytab as kudu/127.22.167.67@KRBTEST.COM (short username kudu)
I20260810 04:44:02.931859 17465 webserver.cc:460] Webserver started at http://127.22.167.67:46093/ using document root <none> and password file <none>
I20260810 04:44:02.932039 17465 fs_manager.cc:359] Metadata directory not provided
I20260810 04:44:02.932075 17465 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 04:44:02.932169 17465 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 04:44:02.933161 17465 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786336863388938-23197-0/minicluster-data/ts-2/data/instance:
uuid: "671e8749ce234fd9a175013a5198a496"
format_stamp: "Formatted at 2026-08-10 04:44:02 on dist-test-slave-bqcl"
server_key: "162bd4b4c572401ddcc172ca46430063"
server_key_iv: "b4da5ca9ac9870928944a17c8ae016ea"
server_key_version: "encryptionkey@0"
I20260810 04:44:02.933454 17465 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786336863388938-23197-0/minicluster-data/ts-2/wal/instance:
uuid: "671e8749ce234fd9a175013a5198a496"
format_stamp: "Formatted at 2026-08-10 04:44:02 on dist-test-slave-bqcl"
server_key: "162bd4b4c572401ddcc172ca46430063"
server_key_iv: "b4da5ca9ac9870928944a17c8ae016ea"
server_key_version: "encryptionkey@0"
I20260810 04:44:02.934693 17465 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260810 04:44:02.935343 17488 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 04:44:02.935514 17465 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260810 04:44:02.935597 17465 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786336863388938-23197-0/minicluster-data/ts-2/data,/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786336863388938-23197-0/minicluster-data/ts-2/wal
uuid: "671e8749ce234fd9a175013a5198a496"
format_stamp: "Formatted at 2026-08-10 04:44:02 on dist-test-slave-bqcl"
server_key: "162bd4b4c572401ddcc172ca46430063"
server_key_iv: "b4da5ca9ac9870928944a17c8ae016ea"
server_key_version: "encryptionkey@0"
I20260810 04:44:02.935663 17465 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786336863388938-23197-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786336863388938-23197-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786336863388938-23197-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 04:44:02.950721 17465 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 04:44:02.952641 17465 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 04:44:02.952775 17465 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 04:44:02.952971 17465 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 04:44:02.953272 17465 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 04:44:02.953312 17465 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 04:44:02.953334 17465 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 04:44:02.953356 17465 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 04:44:02.958773 17465 rpc_server.cc:307] RPC server started. Bound to: 127.22.167.67:37313
I20260810 04:44:02.958840 17694 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.167.67:37313 every 8 connection(s)
I20260810 04:44:02.959111 17465 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786336863388938-23197-0/minicluster-data/ts-2/data/info.pb
I20260810 04:44:02.963479 23197 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskemv_AM/build/release/bin/kudu as pid 17465
I20260810 04:44:02.963549 23197 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786336863388938-23197-0/minicluster-data/ts-2/wal/instance
I20260810 04:44:02.963696 23197 external_mini_cluster.cc:1428] Setting key 3c01fe9eef586a37f6eb58e06c692a49
Aug 10 04:44:02 dist-test-slave-bqcl krb5kdc[16801](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786337042, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.167.67@KRBTEST.COM for kudu/127.22.167.126@KRBTEST.COM
I20260810 04:44:02.967947 16959 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:44:02.959797 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.167.67:50201 (local address 127.22.167.126:38209)
0810 04:44:02.959855 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0810 04:44:02.959857 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 04:44:02.960041 (+ 184us) server_negotiation.cc:407] Connection header received
0810 04:44:02.960535 (+ 494us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:44:02.960537 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:44:02.960550 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:44:02.960602 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0810 04:44:02.961697 (+ 1095us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:44:02.962232 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:44:02.963197 (+ 965us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:44:02.963411 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:44:02.963466 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:44:02.965442 (+ 1976us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 04:44:02.965443 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 04:44:02.965444 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 04:44:02.965451 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 04:44:02.966712 (+ 1261us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 04:44:02.967071 (+ 359us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 04:44:02.967073 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 04:44:02.967073 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 04:44:02.967128 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 04:44:02.967348 (+ 220us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 04:44:02.967349 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 04:44:02.967349 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 04:44:02.967456 (+ 107us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 04:44:02.967512 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0810 04:44:02.967777 (+ 265us) server_negotiation.cc:299] Negotiation successful
0810 04:44:02.967858 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260810 04:44:02.968107 17701 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:44:02.959767 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.126:38209 (local address 127.22.167.67:50201)
0810 04:44:02.959929 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0810 04:44:02.959965 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0810 04:44:02.960443 (+ 478us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 04:44:02.960690 (+ 247us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 04:44:02.960695 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 04:44:02.961383 (+ 688us) client_negotiation.cc:189] Negotiated authn=SASL
0810 04:44:02.961592 (+ 209us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:44:02.961598 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:44:02.962307 (+ 709us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:44:02.962308 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:44:02.963110 (+ 802us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:44:02.963118 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:44:02.963480 (+ 362us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:44:02.963481 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:44:02.963617 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:44:02.964070 (+ 453us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 04:44:02.964078 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0810 04:44:02.965357 (+ 1279us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 04:44:02.966784 (+ 1427us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 04:44:02.966791 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 04:44:02.966793 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 04:44:02.967005 (+ 212us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 04:44:02.967188 (+ 183us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 04:44:02.967189 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 04:44:02.967190 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 04:44:02.967293 (+ 103us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 04:44:02.967509 (+ 216us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 04:44:02.967516 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 04:44:02.967657 (+ 141us) client_negotiation.cc:769] Sending connection context
0810 04:44:02.967745 (+ 88us) client_negotiation.cc:240] Negotiation successful
0810 04:44:02.967898 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":82,"threads_started":1}
I20260810 04:44:02.968509 17695 heartbeater.cc:347] Connected to a master server at 127.22.167.126:38209
I20260810 04:44:02.968600 17695 heartbeater.cc:464] Registering TS with master...
I20260810 04:44:02.968783 17695 heartbeater.cc:511] Master 127.22.167.126:38209 requested a full tablet report, sending...
I20260810 04:44:02.969127 16884 ts_manager.cc:194] Registered new tserver with Master: 671e8749ce234fd9a175013a5198a496 (127.22.167.67:37313)
I20260810 04:44:02.969489 16884 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.167.67@KRBTEST.COM'} at 127.22.167.67:50201
I20260810 04:44:02.975126 23197 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260810 04:44:02.981798 16959 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:44:02.976349 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54384 (local address 127.22.167.126:38209)
0810 04:44:02.976410 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0810 04:44:02.976412 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 04:44:02.976487 (+ 75us) server_negotiation.cc:407] Connection header received
0810 04:44:02.976528 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:44:02.976529 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:44:02.976538 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:44:02.976579 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0810 04:44:02.977116 (+ 537us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:44:02.977584 (+ 468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:44:02.978280 (+ 696us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:44:02.978457 (+ 177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:44:02.978500 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:44:02.979480 (+ 980us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 04:44:02.979481 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 04:44:02.979481 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 04:44:02.979487 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0810 04:44:02.980668 (+ 1181us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 04:44:02.980954 (+ 286us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 04:44:02.980955 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 04:44:02.980955 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 04:44:02.981003 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 04:44:02.981216 (+ 213us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 04:44:02.981217 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 04:44:02.981218 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 04:44:02.981434 (+ 216us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 04:44:02.981518 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0810 04:44:02.981630 (+ 112us) server_negotiation.cc:299] Negotiation successful
0810 04:44:02.981715 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260810 04:44:02.983084 16884 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54384:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 04:44:02.984648 16884 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 04:44:02.990945 17719 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:44:02.986842 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.66:45913 (local address 127.0.0.1:43186)
0810 04:44:02.987365 (+ 523us) negotiation.cc:107] Waiting for socket to connect
0810 04:44:02.987374 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0810 04:44:02.987425 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 04:44:02.987644 (+ 219us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 04:44:02.987645 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 04:44:02.987647 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 04:44:02.987703 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:44:02.987705 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:44:02.988671 (+ 966us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:44:02.988672 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:44:02.989925 (+ 1253us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:44:02.989928 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:44:02.990656 (+ 728us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:44:02.990657 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:44:02.990733 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:44:02.990747 (+ 14us) client_negotiation.cc:769] Sending connection context
0810 04:44:02.990810 (+ 63us) client_negotiation.cc:240] Negotiation successful
0810 04:44:02.990867 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":508,"mutex_wait_us":10,"thread_start_us":90,"threads_started":1}
I20260810 04:44:02.991082 17722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:44:02.987004 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50794 (local address 127.22.167.65:37211)
0810 04:44:02.987161 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0810 04:44:02.987166 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 04:44:02.987340 (+ 174us) server_negotiation.cc:407] Connection header received
0810 04:44:02.987402 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:44:02.987405 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:44:02.987422 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:44:02.987497 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 04:44:02.987858 (+ 361us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:44:02.988930 (+ 1072us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:44:02.989985 (+ 1055us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:44:02.990746 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:44:02.990825 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:44:02.990883 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0810 04:44:02.990957 (+ 74us) server_negotiation.cc:299] Negotiation successful
0810 04:44:02.990996 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":122,"thread_start_us":85,"threads_started":1}
I20260810 04:44:02.991151 17718 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:44:02.986842 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.67:37313 (local address 127.0.0.1:38496)
0810 04:44:02.987186 (+ 344us) negotiation.cc:107] Waiting for socket to connect
0810 04:44:02.987196 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0810 04:44:02.987264 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 04:44:02.987547 (+ 283us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 04:44:02.987563 (+ 16us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 04:44:02.987565 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 04:44:02.987626 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:44:02.987628 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:44:02.988937 (+ 1309us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:44:02.988938 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:44:02.989991 (+ 1053us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:44:02.989993 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:44:02.990919 (+ 926us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:44:02.990920 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:44:02.990997 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:44:02.991000 (+ 3us) client_negotiation.cc:769] Sending connection context
0810 04:44:02.991050 (+ 50us) client_negotiation.cc:240] Negotiation successful
0810 04:44:02.991089 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":324,"thread_start_us":99,"threads_started":1}
I20260810 04:44:02.991204 17724 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:44:02.987051 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43186 (local address 127.22.167.66:45913)
0810 04:44:02.987249 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0810 04:44:02.987254 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 04:44:02.987425 (+ 171us) server_negotiation.cc:407] Connection header received
0810 04:44:02.987531 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:44:02.987533 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:44:02.987548 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:44:02.987640 (+ 92us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 04:44:02.987838 (+ 198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:44:02.988570 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:44:02.990023 (+ 1453us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:44:02.990580 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:44:02.990653 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:44:02.990717 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0810 04:44:02.991065 (+ 348us) server_negotiation.cc:299] Negotiation successful
0810 04:44:02.991116 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":167,"thread_start_us":86,"threads_started":1}
I20260810 04:44:02.991377 17726 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:44:02.987108 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38496 (local address 127.22.167.67:37313)
0810 04:44:02.987281 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0810 04:44:02.987285 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 04:44:02.987295 (+ 10us) server_negotiation.cc:407] Connection header received
0810 04:44:02.987339 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:44:02.987341 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:44:02.987356 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:44:02.987457 (+ 101us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 04:44:02.987709 (+ 252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:44:02.988838 (+ 1129us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:44:02.990053 (+ 1215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:44:02.990746 (+ 693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:44:02.990829 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:44:02.990883 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0810 04:44:02.991253 (+ 370us) server_negotiation.cc:299] Negotiation successful
0810 04:44:02.991300 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":140,"thread_start_us":100,"threads_started":1}
I20260810 04:44:02.991762 17717 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:44:02.986836 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.65:37211 (local address 127.0.0.1:50794)
0810 04:44:02.987006 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0810 04:44:02.987050 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0810 04:44:02.987326 (+ 276us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 04:44:02.987592 (+ 266us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 04:44:02.987593 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 04:44:02.987595 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 04:44:02.987730 (+ 135us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:44:02.987733 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:44:02.989006 (+ 1273us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:44:02.989007 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:44:02.989913 (+ 906us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:44:02.989915 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:44:02.990839 (+ 924us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:44:02.990840 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:44:02.990892 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:44:02.990895 (+ 3us) client_negotiation.cc:769] Sending connection context
0810 04:44:02.990945 (+ 50us) client_negotiation.cc:240] Negotiation successful
0810 04:44:02.990978 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":142,"mutex_wait_us":100,"thread_start_us":45,"threads_started":1}
I20260810 04:44:02.991966 17346 tablet_service.cc:1467] Processing CreateTablet for tablet 3819f550523a4744a19f7f353bb58809 (DEFAULT_TABLE table=test-table [id=1fb0ef4c9af041439b810cf07adcc04c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 04:44:02.992041 17573 tablet_service.cc:1467] Processing CreateTablet for tablet 3819f550523a4744a19f7f353bb58809 (DEFAULT_TABLE table=test-table [id=1fb0ef4c9af041439b810cf07adcc04c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 04:44:02.992376 17346 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3819f550523a4744a19f7f353bb58809. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 04:44:02.992403 17573 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3819f550523a4744a19f7f353bb58809. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 04:44:02.992415 17099 tablet_service.cc:1467] Processing CreateTablet for tablet 3819f550523a4744a19f7f353bb58809 (DEFAULT_TABLE table=test-table [id=1fb0ef4c9af041439b810cf07adcc04c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 04:44:02.992759 17099 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3819f550523a4744a19f7f353bb58809. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 04:44:02.995596 17728 tablet_bootstrap.cc:492] T 3819f550523a4744a19f7f353bb58809 P 671e8749ce234fd9a175013a5198a496: Bootstrap starting.
I20260810 04:44:02.995849 17729 tablet_bootstrap.cc:492] T 3819f550523a4744a19f7f353bb58809 P 7e7834fc8be34efcb685d3a20756d41b: Bootstrap starting.
I20260810 04:44:02.995990 17727 tablet_bootstrap.cc:492] T 3819f550523a4744a19f7f353bb58809 P 6ee21ed53fbc49d284bbecab757bb9db: Bootstrap starting.
I20260810 04:44:02.996279 17728 tablet_bootstrap.cc:654] T 3819f550523a4744a19f7f353bb58809 P 671e8749ce234fd9a175013a5198a496: Neither blocks nor log segments found. Creating new log.
I20260810 04:44:02.996515 17729 tablet_bootstrap.cc:654] T 3819f550523a4744a19f7f353bb58809 P 7e7834fc8be34efcb685d3a20756d41b: Neither blocks nor log segments found. Creating new log.
I20260810 04:44:02.996556 17727 tablet_bootstrap.cc:654] T 3819f550523a4744a19f7f353bb58809 P 6ee21ed53fbc49d284bbecab757bb9db: Neither blocks nor log segments found. Creating new log.
I20260810 04:44:02.996836 17728 log.cc:824] T 3819f550523a4744a19f7f353bb58809 P 671e8749ce234fd9a175013a5198a496: Log is configured to *not* fsync() on all Append() calls
I20260810 04:44:02.997083 17729 log.cc:824] T 3819f550523a4744a19f7f353bb58809 P 7e7834fc8be34efcb685d3a20756d41b: Log is configured to *not* fsync() on all Append() calls
I20260810 04:44:02.997083 17727 log.cc:824] T 3819f550523a4744a19f7f353bb58809 P 6ee21ed53fbc49d284bbecab757bb9db: Log is configured to *not* fsync() on all Append() calls
I20260810 04:44:02.997598 17728 tablet_bootstrap.cc:492] T 3819f550523a4744a19f7f353bb58809 P 671e8749ce234fd9a175013a5198a496: No bootstrap required, opened a new log
I20260810 04:44:02.997675 17728 ts_tablet_manager.cc:1397] T 3819f550523a4744a19f7f353bb58809 P 671e8749ce234fd9a175013a5198a496: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260810 04:44:02.997799 17727 tablet_bootstrap.cc:492] T 3819f550523a4744a19f7f353bb58809 P 6ee21ed53fbc49d284bbecab757bb9db: No bootstrap required, opened a new log
I20260810 04:44:02.997869 17727 ts_tablet_manager.cc:1397] T 3819f550523a4744a19f7f353bb58809 P 6ee21ed53fbc49d284bbecab757bb9db: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260810 04:44:02.998006 17729 tablet_bootstrap.cc:492] T 3819f550523a4744a19f7f353bb58809 P 7e7834fc8be34efcb685d3a20756d41b: No bootstrap required, opened a new log
I20260810 04:44:02.998068 17729 ts_tablet_manager.cc:1397] T 3819f550523a4744a19f7f353bb58809 P 7e7834fc8be34efcb685d3a20756d41b: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260810 04:44:02.999379 17728 raft_consensus.cc:348] T 3819f550523a4744a19f7f353bb58809 P 671e8749ce234fd9a175013a5198a496 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e7834fc8be34efcb685d3a20756d41b" member_type: VOTER last_known_addr { host: "127.22.167.65" port: 37211 } } peers { permanent_uuid: "671e8749ce234fd9a175013a5198a496" member_type: VOTER last_known_addr { host: "127.22.167.67" port: 37313 } } peers { permanent_uuid: "6ee21ed53fbc49d284bbecab757bb9db" member_type: VOTER last_known_addr { host: "127.22.167.66" port: 45913 } }
I20260810 04:44:02.999518 17728 raft_consensus.cc:374] T 3819f550523a4744a19f7f353bb58809 P 671e8749ce234fd9a175013a5198a496 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 04:44:02.999552 17728 raft_consensus.cc:729] T 3819f550523a4744a19f7f353bb58809 P 671e8749ce234fd9a175013a5198a496 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 671e8749ce234fd9a175013a5198a496, State: Initialized, Role: FOLLOWER
I20260810 04:44:02.999694 17728 consensus_queue.cc:260] T 3819f550523a4744a19f7f353bb58809 P 671e8749ce234fd9a175013a5198a496 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e7834fc8be34efcb685d3a20756d41b" member_type: VOTER last_known_addr { host: "127.22.167.65" port: 37211 } } peers { permanent_uuid: "671e8749ce234fd9a175013a5198a496" member_type: VOTER last_known_addr { host: "127.22.167.67" port: 37313 } } peers { permanent_uuid: "6ee21ed53fbc49d284bbecab757bb9db" member_type: VOTER last_known_addr { host: "127.22.167.66" port: 45913 } }
I20260810 04:44:02.999749 17727 raft_consensus.cc:348] T 3819f550523a4744a19f7f353bb58809 P 6ee21ed53fbc49d284bbecab757bb9db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e7834fc8be34efcb685d3a20756d41b" member_type: VOTER last_known_addr { host: "127.22.167.65" port: 37211 } } peers { permanent_uuid: "671e8749ce234fd9a175013a5198a496" member_type: VOTER last_known_addr { host: "127.22.167.67" port: 37313 } } peers { permanent_uuid: "6ee21ed53fbc49d284bbecab757bb9db" member_type: VOTER last_known_addr { host: "127.22.167.66" port: 45913 } }
I20260810 04:44:02.999923 17727 raft_consensus.cc:374] T 3819f550523a4744a19f7f353bb58809 P 6ee21ed53fbc49d284bbecab757bb9db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 04:44:02.999964 17727 raft_consensus.cc:729] T 3819f550523a4744a19f7f353bb58809 P 6ee21ed53fbc49d284bbecab757bb9db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6ee21ed53fbc49d284bbecab757bb9db, State: Initialized, Role: FOLLOWER
I20260810 04:44:02.999962 17728 ts_tablet_manager.cc:1428] T 3819f550523a4744a19f7f353bb58809 P 671e8749ce234fd9a175013a5198a496: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260810 04:44:02.999903 17729 raft_consensus.cc:348] T 3819f550523a4744a19f7f353bb58809 P 7e7834fc8be34efcb685d3a20756d41b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e7834fc8be34efcb685d3a20756d41b" member_type: VOTER last_known_addr { host: "127.22.167.65" port: 37211 } } peers { permanent_uuid: "671e8749ce234fd9a175013a5198a496" member_type: VOTER last_known_addr { host: "127.22.167.67" port: 37313 } } peers { permanent_uuid: "6ee21ed53fbc49d284bbecab757bb9db" member_type: VOTER last_known_addr { host: "127.22.167.66" port: 45913 } }
I20260810 04:44:03.000056 17729 raft_consensus.cc:374] T 3819f550523a4744a19f7f353bb58809 P 7e7834fc8be34efcb685d3a20756d41b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 04:44:03.000097 17729 raft_consensus.cc:729] T 3819f550523a4744a19f7f353bb58809 P 7e7834fc8be34efcb685d3a20756d41b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7e7834fc8be34efcb685d3a20756d41b, State: Initialized, Role: FOLLOWER
I20260810 04:44:03.000108 17695 heartbeater.cc:503] Master 127.22.167.126:38209 was elected leader, sending a full tablet report...
I20260810 04:44:03.000075 17727 consensus_queue.cc:260] T 3819f550523a4744a19f7f353bb58809 P 6ee21ed53fbc49d284bbecab757bb9db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e7834fc8be34efcb685d3a20756d41b" member_type: VOTER last_known_addr { host: "127.22.167.65" port: 37211 } } peers { permanent_uuid: "671e8749ce234fd9a175013a5198a496" member_type: VOTER last_known_addr { host: "127.22.167.67" port: 37313 } } peers { permanent_uuid: "6ee21ed53fbc49d284bbecab757bb9db" member_type: VOTER last_known_addr { host: "127.22.167.66" port: 45913 } }
I20260810 04:44:03.000240 17729 consensus_queue.cc:260] T 3819f550523a4744a19f7f353bb58809 P 7e7834fc8be34efcb685d3a20756d41b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e7834fc8be34efcb685d3a20756d41b" member_type: VOTER last_known_addr { host: "127.22.167.65" port: 37211 } } peers { permanent_uuid: "671e8749ce234fd9a175013a5198a496" member_type: VOTER last_known_addr { host: "127.22.167.67" port: 37313 } } peers { permanent_uuid: "6ee21ed53fbc49d284bbecab757bb9db" member_type: VOTER last_known_addr { host: "127.22.167.66" port: 45913 } }
I20260810 04:44:03.000344 17727 ts_tablet_manager.cc:1428] T 3819f550523a4744a19f7f353bb58809 P 6ee21ed53fbc49d284bbecab757bb9db: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260810 04:44:03.000471 17455 heartbeater.cc:503] Master 127.22.167.126:38209 was elected leader, sending a full tablet report...
I20260810 04:44:03.000752 17729 ts_tablet_manager.cc:1428] T 3819f550523a4744a19f7f353bb58809 P 7e7834fc8be34efcb685d3a20756d41b: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260810 04:44:03.000862 17228 heartbeater.cc:503] Master 127.22.167.126:38209 was elected leader, sending a full tablet report...
W20260810 04:44:03.061970 17458 tablet.cc:2369] T 3819f550523a4744a19f7f353bb58809 P 6ee21ed53fbc49d284bbecab757bb9db: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 04:44:03.151350 17738 raft_consensus.cc:482] T 3819f550523a4744a19f7f353bb58809 P 671e8749ce234fd9a175013a5198a496 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 04:44:03.151502 17738 raft_consensus.cc:504] T 3819f550523a4744a19f7f353bb58809 P 671e8749ce234fd9a175013a5198a496 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e7834fc8be34efcb685d3a20756d41b" member_type: VOTER last_known_addr { host: "127.22.167.65" port: 37211 } } peers { permanent_uuid: "671e8749ce234fd9a175013a5198a496" member_type: VOTER last_known_addr { host: "127.22.167.67" port: 37313 } } peers { permanent_uuid: "6ee21ed53fbc49d284bbecab757bb9db" member_type: VOTER last_known_addr { host: "127.22.167.66" port: 45913 } }
I20260810 04:44:03.151899 17738 leader_election.cc:288] T 3819f550523a4744a19f7f353bb58809 P 671e8749ce234fd9a175013a5198a496 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e7834fc8be34efcb685d3a20756d41b (127.22.167.65:37211), 6ee21ed53fbc49d284bbecab757bb9db (127.22.167.66:45913)
I20260810 04:44:03.155725 17724 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:44:03.152188 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.167.67:59317 (local address 127.22.167.66:45913)
0810 04:44:03.152231 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0810 04:44:03.152233 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 04:44:03.152402 (+ 169us) server_negotiation.cc:407] Connection header received
0810 04:44:03.152437 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:44:03.152438 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:44:03.152447 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:44:03.152489 (+ 42us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 04:44:03.152649 (+ 160us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:44:03.153329 (+ 680us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:44:03.154795 (+ 1466us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:44:03.155370 (+ 575us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:44:03.155443 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:44:03.155451 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0810 04:44:03.155625 (+ 174us) server_negotiation.cc:299] Negotiation successful
0810 04:44:03.155661 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260810 04:44:03.155725 17722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:44:03.152186 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.167.67:59331 (local address 127.22.167.65:37211)
0810 04:44:03.152240 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0810 04:44:03.152242 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 04:44:03.152253 (+ 11us) server_negotiation.cc:407] Connection header received
0810 04:44:03.152323 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:44:03.152324 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:44:03.152332 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:44:03.152376 (+ 44us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 04:44:03.152545 (+ 169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:44:03.153307 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:44:03.154787 (+ 1480us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:44:03.155370 (+ 583us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:44:03.155444 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:44:03.155451 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0810 04:44:03.155601 (+ 150us) server_negotiation.cc:299] Negotiation successful
0810 04:44:03.155647 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260810 04:44:03.155736 17701 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:44:03.152096 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.65:37211 (local address 127.22.167.67:59331)
0810 04:44:03.152182 (+ 86us) negotiation.cc:107] Waiting for socket to connect
0810 04:44:03.152195 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0810 04:44:03.152246 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 04:44:03.152395 (+ 149us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 04:44:03.152396 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 04:44:03.152398 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 04:44:03.152478 (+ 80us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:44:03.152480 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:44:03.153394 (+ 914us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:44:03.153394 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:44:03.154702 (+ 1308us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:44:03.154704 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:44:03.155448 (+ 744us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:44:03.155449 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:44:03.155538 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:44:03.155540 (+ 2us) client_negotiation.cc:769] Sending connection context
0810 04:44:03.155589 (+ 49us) client_negotiation.cc:240] Negotiation successful
0810 04:44:03.155634 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":64,"mutex_wait_us":111}
I20260810 04:44:03.155743 17746 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:44:03.152124 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.66:45913 (local address 127.22.167.67:59317)
0810 04:44:03.152317 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0810 04:44:03.152333 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0810 04:44:03.152387 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 04:44:03.152504 (+ 117us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 04:44:03.152505 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 04:44:03.152506 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 04:44:03.152565 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:44:03.152566 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:44:03.153394 (+ 828us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:44:03.153394 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:44:03.154701 (+ 1307us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:44:03.154704 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:44:03.155448 (+ 744us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:44:03.155449 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:44:03.155535 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:44:03.155540 (+ 5us) client_negotiation.cc:769] Sending connection context
0810 04:44:03.155590 (+ 50us) client_negotiation.cc:240] Negotiation successful
0810 04:44:03.155634 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":177,"mutex_wait_us":12,"thread_start_us":135,"threads_started":1}
I20260810 04:44:03.156116 17140 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3819f550523a4744a19f7f353bb58809" candidate_uuid: "671e8749ce234fd9a175013a5198a496" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7e7834fc8be34efcb685d3a20756d41b" is_pre_election: true
I20260810 04:44:03.156117 17384 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3819f550523a4744a19f7f353bb58809" candidate_uuid: "671e8749ce234fd9a175013a5198a496" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6ee21ed53fbc49d284bbecab757bb9db" is_pre_election: true
I20260810 04:44:03.156261 17384 raft_consensus.cc:2454] T 3819f550523a4744a19f7f353bb58809 P 6ee21ed53fbc49d284bbecab757bb9db [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 671e8749ce234fd9a175013a5198a496 in term 0.
I20260810 04:44:03.156273 17140 raft_consensus.cc:2454] T 3819f550523a4744a19f7f353bb58809 P 7e7834fc8be34efcb685d3a20756d41b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 671e8749ce234fd9a175013a5198a496 in term 0.
I20260810 04:44:03.156491 17493 leader_election.cc:302] T 3819f550523a4744a19f7f353bb58809 P 671e8749ce234fd9a175013a5198a496 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 671e8749ce234fd9a175013a5198a496, 6ee21ed53fbc49d284bbecab757bb9db; no voters:
I20260810 04:44:03.156626 17738 raft_consensus.cc:2781] T 3819f550523a4744a19f7f353bb58809 P 671e8749ce234fd9a175013a5198a496 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 04:44:03.156693 17738 raft_consensus.cc:482] T 3819f550523a4744a19f7f353bb58809 P 671e8749ce234fd9a175013a5198a496 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 04:44:03.156709 17738 raft_consensus.cc:3037] T 3819f550523a4744a19f7f353bb58809 P 671e8749ce234fd9a175013a5198a496 [term 0 FOLLOWER]: Advancing to term 1
I20260810 04:44:03.157382 17738 raft_consensus.cc:504] T 3819f550523a4744a19f7f353bb58809 P 671e8749ce234fd9a175013a5198a496 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e7834fc8be34efcb685d3a20756d41b" member_type: VOTER last_known_addr { host: "127.22.167.65" port: 37211 } } peers { permanent_uuid: "671e8749ce234fd9a175013a5198a496" member_type: VOTER last_known_addr { host: "127.22.167.67" port: 37313 } } peers { permanent_uuid: "6ee21ed53fbc49d284bbecab757bb9db" member_type: VOTER last_known_addr { host: "127.22.167.66" port: 45913 } }
I20260810 04:44:03.157495 17738 leader_election.cc:288] T 3819f550523a4744a19f7f353bb58809 P 671e8749ce234fd9a175013a5198a496 [CANDIDATE]: Term 1 election: Requested vote from peers 7e7834fc8be34efcb685d3a20756d41b (127.22.167.65:37211), 6ee21ed53fbc49d284bbecab757bb9db (127.22.167.66:45913)
I20260810 04:44:03.157665 17384 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3819f550523a4744a19f7f353bb58809" candidate_uuid: "671e8749ce234fd9a175013a5198a496" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6ee21ed53fbc49d284bbecab757bb9db"
I20260810 04:44:03.157682 17140 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3819f550523a4744a19f7f353bb58809" candidate_uuid: "671e8749ce234fd9a175013a5198a496" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7e7834fc8be34efcb685d3a20756d41b"
I20260810 04:44:03.157749 17140 raft_consensus.cc:3037] T 3819f550523a4744a19f7f353bb58809 P 7e7834fc8be34efcb685d3a20756d41b [term 0 FOLLOWER]: Advancing to term 1
I20260810 04:44:03.157753 17384 raft_consensus.cc:3037] T 3819f550523a4744a19f7f353bb58809 P 6ee21ed53fbc49d284bbecab757bb9db [term 0 FOLLOWER]: Advancing to term 1
I20260810 04:44:03.158489 17140 raft_consensus.cc:2454] T 3819f550523a4744a19f7f353bb58809 P 7e7834fc8be34efcb685d3a20756d41b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 671e8749ce234fd9a175013a5198a496 in term 1.
I20260810 04:44:03.158548 17384 raft_consensus.cc:2454] T 3819f550523a4744a19f7f353bb58809 P 6ee21ed53fbc49d284bbecab757bb9db [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 671e8749ce234fd9a175013a5198a496 in term 1.
W20260810 04:44:03.158646 17231 tablet.cc:2369] T 3819f550523a4744a19f7f353bb58809 P 7e7834fc8be34efcb685d3a20756d41b: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 04:44:03.158689 17489 leader_election.cc:302] T 3819f550523a4744a19f7f353bb58809 P 671e8749ce234fd9a175013a5198a496 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 671e8749ce234fd9a175013a5198a496, 7e7834fc8be34efcb685d3a20756d41b; no voters:
I20260810 04:44:03.158777 17738 raft_consensus.cc:2781] T 3819f550523a4744a19f7f353bb58809 P 671e8749ce234fd9a175013a5198a496 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 04:44:03.158927 17738 raft_consensus.cc:686] T 3819f550523a4744a19f7f353bb58809 P 671e8749ce234fd9a175013a5198a496 [term 1 LEADER]: Becoming Leader. State: Replica: 671e8749ce234fd9a175013a5198a496, State: Running, Role: LEADER
I20260810 04:44:03.159014 17738 consensus_queue.cc:237] T 3819f550523a4744a19f7f353bb58809 P 671e8749ce234fd9a175013a5198a496 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e7834fc8be34efcb685d3a20756d41b" member_type: VOTER last_known_addr { host: "127.22.167.65" port: 37211 } } peers { permanent_uuid: "671e8749ce234fd9a175013a5198a496" member_type: VOTER last_known_addr { host: "127.22.167.67" port: 37313 } } peers { permanent_uuid: "6ee21ed53fbc49d284bbecab757bb9db" member_type: VOTER last_known_addr { host: "127.22.167.66" port: 45913 } }
I20260810 04:44:03.159878 16884 catalog_manager.cc:5495] T 3819f550523a4744a19f7f353bb58809 P 671e8749ce234fd9a175013a5198a496 reported cstate change: term changed from 0 to 1, leader changed from <none> to 671e8749ce234fd9a175013a5198a496 (127.22.167.67). New cstate: current_term: 1 leader_uuid: "671e8749ce234fd9a175013a5198a496" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e7834fc8be34efcb685d3a20756d41b" member_type: VOTER last_known_addr { host: "127.22.167.65" port: 37211 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "671e8749ce234fd9a175013a5198a496" member_type: VOTER last_known_addr { host: "127.22.167.67" port: 37313 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6ee21ed53fbc49d284bbecab757bb9db" member_type: VOTER last_known_addr { host: "127.22.167.66" port: 45913 } health_report { overall_health: UNKNOWN } } }
I20260810 04:44:03.169598 17726 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:44:03.167583 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38498 (local address 127.22.167.67:37313)
0810 04:44:03.167634 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0810 04:44:03.167637 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 04:44:03.167645 (+ 8us) server_negotiation.cc:407] Connection header received
0810 04:44:03.167674 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:44:03.167674 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:44:03.167691 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:44:03.167753 (+ 62us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 04:44:03.167855 (+ 102us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:44:03.168310 (+ 455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:44:03.169010 (+ 700us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:44:03.169208 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:44:03.169273 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:44:03.169362 (+ 89us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 04:44:03.169454 (+ 92us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 04:44:03.169509 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0810 04:44:03.169522 (+ 13us) server_negotiation.cc:299] Negotiation successful
0810 04:44:03.169539 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260810 04:44:03.171291 17384 raft_consensus.cc:1261] T 3819f550523a4744a19f7f353bb58809 P 6ee21ed53fbc49d284bbecab757bb9db [term 1 FOLLOWER]: Refusing update from remote peer 671e8749ce234fd9a175013a5198a496: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 04:44:03.171293 17140 raft_consensus.cc:1261] T 3819f550523a4744a19f7f353bb58809 P 7e7834fc8be34efcb685d3a20756d41b [term 1 FOLLOWER]: Refusing update from remote peer 671e8749ce234fd9a175013a5198a496: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 04:44:03.171631 17747 consensus_queue.cc:1035] T 3819f550523a4744a19f7f353bb58809 P 671e8749ce234fd9a175013a5198a496 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e7834fc8be34efcb685d3a20756d41b" member_type: VOTER last_known_addr { host: "127.22.167.65" port: 37211 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 04:44:03.171748 17738 consensus_queue.cc:1035] T 3819f550523a4744a19f7f353bb58809 P 671e8749ce234fd9a175013a5198a496 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ee21ed53fbc49d284bbecab757bb9db" member_type: VOTER last_known_addr { host: "127.22.167.66" port: 45913 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 04:44:03.174659 17757 mvcc.cc:205] Tried to move back new op lower bound from 7316836528827359232 to 7316836528779956224. Current Snapshot: MvccSnapshot[applied={T|T < 7316836528827359232 or (T in {7316836528827359232})}]
I20260810 04:44:03.175155 17756 mvcc.cc:205] Tried to move back new op lower bound from 7316836528827359232 to 7316836528779956224. Current Snapshot: MvccSnapshot[applied={T|T < 7316836528827359232 or (T in {7316836528827359232})}]
I20260810 04:44:03.176275 16884 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54384:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 04:44:03.176355 16884 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54384:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 04:44:03.177003 16884 catalog_manager.cc:5782] T 00000000000000000000000000000000 P f49432ad9f9f4c31b0380574d0265634: Sending DeleteTablet for 3 replicas of tablet 3819f550523a4744a19f7f353bb58809
I20260810 04:44:03.177357 17099 tablet_service.cc:1514] Processing DeleteTablet for tablet 3819f550523a4744a19f7f353bb58809 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 04:44:03 UTC) from {username='kudu', principal='kudu/127.22.167.126@KRBTEST.COM'} at 127.0.0.1:50794
I20260810 04:44:03.177393 17346 tablet_service.cc:1514] Processing DeleteTablet for tablet 3819f550523a4744a19f7f353bb58809 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 04:44:03 UTC) from {username='kudu', principal='kudu/127.22.167.126@KRBTEST.COM'} at 127.0.0.1:43186
I20260810 04:44:03.177438 23197 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskemv_AM/build/release/bin/kudu with pid 16985
I20260810 04:44:03.182370 17573 tablet_service.cc:1514] Processing DeleteTablet for tablet 3819f550523a4744a19f7f353bb58809 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 04:44:03 UTC) from {username='kudu', principal='kudu/127.22.167.126@KRBTEST.COM'} at 127.0.0.1:38496
I20260810 04:44:03.185262 17778 tablet_replica.cc:330] T 3819f550523a4744a19f7f353bb58809 P 671e8749ce234fd9a175013a5198a496: stopping tablet replica
I20260810 04:44:03.185411 17778 raft_consensus.cc:2229] T 3819f550523a4744a19f7f353bb58809 P 671e8749ce234fd9a175013a5198a496 [term 1 LEADER]: Raft consensus shutting down.
I20260810 04:44:03.185583 17778 raft_consensus.cc:2258] T 3819f550523a4744a19f7f353bb58809 P 671e8749ce234fd9a175013a5198a496 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 04:44:03.187562 17778 ts_tablet_manager.cc:1905] T 3819f550523a4744a19f7f353bb58809 P 671e8749ce234fd9a175013a5198a496: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 04:44:03.189196 17778 ts_tablet_manager.cc:1918] T 3819f550523a4744a19f7f353bb58809 P 671e8749ce234fd9a175013a5198a496: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 04:44:03.189276 17778 log.cc:1196] T 3819f550523a4744a19f7f353bb58809 P 671e8749ce234fd9a175013a5198a496: Deleting WAL directory at /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786336863388938-23197-0/minicluster-data/ts-2/wal/wals/3819f550523a4744a19f7f353bb58809
I20260810 04:44:03.189540 17778 ts_tablet_manager.cc:1939] T 3819f550523a4744a19f7f353bb58809 P 671e8749ce234fd9a175013a5198a496: Deleting consensus metadata
I20260810 04:44:03.189991 16854 catalog_manager.cc:4841] TS 671e8749ce234fd9a175013a5198a496 (127.22.167.67:37313): tablet 3819f550523a4744a19f7f353bb58809 (table test-table [id=1fb0ef4c9af041439b810cf07adcc04c]) successfully deleted
W20260810 04:44:03.193037 16852 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.22.167.65:37211 (error 108)
I20260810 04:44:03.193212 17777 tablet_replica.cc:330] T 3819f550523a4744a19f7f353bb58809 P 6ee21ed53fbc49d284bbecab757bb9db: stopping tablet replica
I20260810 04:44:03.193383 23197 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskemv_AM/build/release/bin/kudu with pid 17239
I20260810 04:44:03.193392 17777 raft_consensus.cc:2229] T 3819f550523a4744a19f7f353bb58809 P 6ee21ed53fbc49d284bbecab757bb9db [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 04:44:03.193584 17777 raft_consensus.cc:2258] T 3819f550523a4744a19f7f353bb58809 P 6ee21ed53fbc49d284bbecab757bb9db [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 04:44:03.202574 17717 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:44:03.202445 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.65:37211 (local address 127.0.0.1:50804)
0810 04:44:03.202500 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0810 04:44:03.202539 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.167.65:37211: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
W20260810 04:44:03.202756 16852 catalog_manager.cc:4584] TS 7e7834fc8be34efcb685d3a20756d41b (127.22.167.65:37211): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 3819f550523a4744a19f7f353bb58809: Network error: Client connection negotiation failed: client connection to 127.22.167.65:37211: connect: Connection refused (error 111)
I20260810 04:44:03.203478 17717 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:44:03.203382 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.66:45913 (local address 127.0.0.1:43188)
0810 04:44:03.203424 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0810 04:44:03.203450 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.167.66:45913: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":24}
I20260810 04:44:03.203649 23197 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskemv_AM/build/release/bin/kudu with pid 17465
I20260810 04:44:03.212090 23197 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskemv_AM/build/release/bin/kudu with pid 16830
I20260810 04:44:23.222923 23197 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskemv_AM/build/release/bin/kudu
/tmp/dist-test-taskemv_AM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786336863388938-23197-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786336863388938-23197-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786336863388938-23197-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786336863388938-23197-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.167.126:38209
--webserver_interface=127.22.167.126
--webserver_port=35293
--keytab_file=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786336863388938-23197-0/krb5kdc/kudu_127.22.167.126.keytab
--principal=kudu/127.22.167.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.167.84:33205
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.167.126:38209
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--tsk_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786336863388938-23197-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786336863388938-23197-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786336863388938-23197-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 04:44:23.308606 17780 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 04:44:23.308827 17780 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 04:44:23.308859 17780 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 04:44:23.309998 17780 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260810 04:44:23.310045 17780 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260810 04:44:23.310066 17780 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260810 04:44:23.310084 17780 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 04:44:23.310103 17780 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 04:44:23.310159 17780 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 04:44:23.310184 17780 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 04:44:23.310211 17780 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 04:44:23.311683 17780 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.167.84:33205
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786336863388938-23197-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786336863388938-23197-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_private_key_password_cmd=echo foo
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.22.167.126:38209
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786336863388938-23197-0/krb5kdc/kudu_127.22.167.126.keytab
--principal=kudu/127.22.167.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.167.126:38209
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786336863388938-23197-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.167.126
--webserver_port=35293
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17780
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786336863388938-23197-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 03:54:01 UTC on ccycloud.quasar-icijmt.root.comops.site
build id 4577009
I20260810 04:44:23.311933 17780 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 04:44:23.312163 17780 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 04:44:23.321733 17792 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 04:44:23.321760 17794 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 04:44:23.321898 17801 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 04:44:23.321887 17780 server_base.cc:1034] running on GCE node
I20260810 04:44:23.322578 17780 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 04:44:23.322891 17780 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 04:44:23.324030 17780 hybrid_clock.cc:648] HybridClock initialized: now 1786337063324018 us; error 30 us; skew 500 ppm
Aug 10 04:44:23 dist-test-slave-bqcl krb5kdc[16801](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786337063, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.167.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 04:44:23.326570 17780 init.cc:377] Logged in from keytab as kudu/127.22.167.126@KRBTEST.COM (short username kudu)
I20260810 04:44:23.326972 17780 webserver.cc:460] Webserver started at http://127.22.167.126:35293/ using document root <none> and password file <none>
I20260810 04:44:23.327171 17780 fs_manager.cc:359] Metadata directory not provided
I20260810 04:44:23.327225 17780 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 04:44:23.328529 17780 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260810 04:44:23.329123 17809 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 04:44:23.329293 17780 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260810 04:44:23.329380 17780 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786336863388938-23197-0/minicluster-data/master-0/data,/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786336863388938-23197-0/minicluster-data/master-0/wal
uuid: "f49432ad9f9f4c31b0380574d0265634"
format_stamp: "Formatted at 2026-08-10 04:44:02 on dist-test-slave-bqcl"
server_key: "934a4a74932f95d0458d6d58d3085223"
server_key_iv: "995b6f0cd97aa1961bf7aa35fd595378"
server_key_version: "encryptionkey@0"
I20260810 04:44:23.329777 17780 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786336863388938-23197-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786336863388938-23197-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786336863388938-23197-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 04:44:23.340761 17780 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 04:44:23.342533 17780 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 04:44:23.342649 17780 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 04:44:23.346426 17780 rpc_server.cc:307] RPC server started. Bound to: 127.22.167.126:38209
I20260810 04:44:23.346464 17899 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.167.126:38209 every 8 connection(s)
I20260810 04:44:23.346825 17780 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786336863388938-23197-0/minicluster-data/master-0/data/info.pb
I20260810 04:44:23.348296 17900 sys_catalog.cc:263] Verifying existing consensus state
I20260810 04:44:23.348928 17900 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f49432ad9f9f4c31b0380574d0265634: Bootstrap starting.
I20260810 04:44:23.349200 23197 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskemv_AM/build/release/bin/kudu as pid 17780
I20260810 04:44:23.349499 23197 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskemv_AM/build/release/bin/kudu
/tmp/dist-test-taskemv_AM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786336863388938-23197-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786336863388938-23197-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786336863388938-23197-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786336863388938-23197-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.167.65:37211
--local_ip_for_outbound_sockets=127.22.167.65
--tserver_master_addrs=127.22.167.126:38209
--webserver_port=38707
--webserver_interface=127.22.167.65
--keytab_file=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786336863388938-23197-0/krb5kdc/kudu_127.22.167.65.keytab
--principal=kudu/127.22.167.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.167.84:33205
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786336863388938-23197-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786336863388938-23197-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786336863388938-23197-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 04:44:23.358783 17900 log.cc:824] T 00000000000000000000000000000000 P f49432ad9f9f4c31b0380574d0265634: Log is configured to *not* fsync() on all Append() calls
I20260810 04:44:23.360670 17900 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f49432ad9f9f4c31b0380574d0265634: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260810 04:44:23.360947 17900 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f49432ad9f9f4c31b0380574d0265634: Bootstrap complete.
I20260810 04:44:23.363165 17900 raft_consensus.cc:348] T 00000000000000000000000000000000 P f49432ad9f9f4c31b0380574d0265634 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f49432ad9f9f4c31b0380574d0265634" member_type: VOTER last_known_addr { host: "127.22.167.126" port: 38209 } }
I20260810 04:44:23.363396 17900 raft_consensus.cc:729] T 00000000000000000000000000000000 P f49432ad9f9f4c31b0380574d0265634 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: f49432ad9f9f4c31b0380574d0265634, State: Initialized, Role: FOLLOWER
I20260810 04:44:23.363554 17900 consensus_queue.cc:260] T 00000000000000000000000000000000 P f49432ad9f9f4c31b0380574d0265634 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f49432ad9f9f4c31b0380574d0265634" member_type: VOTER last_known_addr { host: "127.22.167.126" port: 38209 } }
I20260810 04:44:23.363632 17900 raft_consensus.cc:388] T 00000000000000000000000000000000 P f49432ad9f9f4c31b0380574d0265634 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 04:44:23.363688 17900 raft_consensus.cc:482] T 00000000000000000000000000000000 P f49432ad9f9f4c31b0380574d0265634 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 04:44:23.363735 17900 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f49432ad9f9f4c31b0380574d0265634 [term 1 FOLLOWER]: Advancing to term 2
I20260810 04:44:23.364729 17900 raft_consensus.cc:504] T 00000000000000000000000000000000 P f49432ad9f9f4c31b0380574d0265634 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f49432ad9f9f4c31b0380574d0265634" member_type: VOTER last_known_addr { host: "127.22.167.126" port: 38209 } }
I20260810 04:44:23.364820 17900 leader_election.cc:302] T 00000000000000000000000000000000 P f49432ad9f9f4c31b0380574d0265634 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f49432ad9f9f4c31b0380574d0265634; no voters:
I20260810 04:44:23.364981 17900 leader_election.cc:288] T 00000000000000000000000000000000 P f49432ad9f9f4c31b0380574d0265634 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260810 04:44:23.365031 17906 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f49432ad9f9f4c31b0380574d0265634 [term 2 FOLLOWER]: Leader election won for term 2
I20260810 04:44:23.365156 17906 raft_consensus.cc:686] T 00000000000000000000000000000000 P f49432ad9f9f4c31b0380574d0265634 [term 2 LEADER]: Becoming Leader. State: Replica: f49432ad9f9f4c31b0380574d0265634, State: Running, Role: LEADER
I20260810 04:44:23.365268 17900 sys_catalog.cc:564] T 00000000000000000000000000000000 P f49432ad9f9f4c31b0380574d0265634 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 04:44:23.365264 17906 consensus_queue.cc:237] T 00000000000000000000000000000000 P f49432ad9f9f4c31b0380574d0265634 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f49432ad9f9f4c31b0380574d0265634" member_type: VOTER last_known_addr { host: "127.22.167.126" port: 38209 } }
I20260810 04:44:23.365581 17907 sys_catalog.cc:455] T 00000000000000000000000000000000 P f49432ad9f9f4c31b0380574d0265634 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "f49432ad9f9f4c31b0380574d0265634" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f49432ad9f9f4c31b0380574d0265634" member_type: VOTER last_known_addr { host: "127.22.167.126" port: 38209 } } }
I20260810 04:44:23.365661 17907 sys_catalog.cc:458] T 00000000000000000000000000000000 P f49432ad9f9f4c31b0380574d0265634 [sys.catalog]: This master's current role is: LEADER
I20260810 04:44:23.365965 17908 sys_catalog.cc:455] T 00000000000000000000000000000000 P f49432ad9f9f4c31b0380574d0265634 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f49432ad9f9f4c31b0380574d0265634. Latest consensus state: current_term: 2 leader_uuid: "f49432ad9f9f4c31b0380574d0265634" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f49432ad9f9f4c31b0380574d0265634" member_type: VOTER last_known_addr { host: "127.22.167.126" port: 38209 } } }
I20260810 04:44:23.366057 17908 sys_catalog.cc:458] T 00000000000000000000000000000000 P f49432ad9f9f4c31b0380574d0265634 [sys.catalog]: This master's current role is: LEADER
I20260810 04:44:23.365979 17912 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 04:44:23.366868 17912 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 04:44:23.366952 17912 catalog_manager.cc:1259] Loaded cluster ID: bd36be342a8e4f998163ba9d6f1c530e
I20260810 04:44:23.366986 17912 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 04:44:23.367839 17912 catalog_manager.cc:1504] Loading token signing keys...
I20260810 04:44:23.367996 17912 catalog_manager.cc:5879] T 00000000000000000000000000000000 P f49432ad9f9f4c31b0380574d0265634: Loaded TSK: 0
I20260810 04:44:23.390234 17912 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f49432ad9f9f4c31b0380574d0265634: Generated new TSK 1
I20260810 04:44:23.390336 17912 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260810 04:44:23.432174 17902 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 04:44:23.432363 17902 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 04:44:23.432394 17902 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 04:44:23.433555 17902 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 04:44:23.433597 17902 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 04:44:23.433631 17902 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.167.65
I20260810 04:44:23.435096 17902 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.167.84:33205
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786336863388938-23197-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786336863388938-23197-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786336863388938-23197-0/krb5kdc/kudu_127.22.167.65.keytab
--principal=kudu/127.22.167.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.167.65:37211
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786336863388938-23197-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.22.167.65
--webserver_port=38707
--tserver_master_addrs=127.22.167.126:38209
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17902
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.167.65
--log_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786336863388938-23197-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 03:54:01 UTC on ccycloud.quasar-icijmt.root.comops.site
build id 4577009
I20260810 04:44:23.435321 17902 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 04:44:23.435523 17902 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 04:44:23.440251 17943 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 04:44:23.440317 17947 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 04:44:23.440251 17944 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 04:44:23.440632 17902 server_base.cc:1034] running on GCE node
I20260810 04:44:23.440816 17902 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 04:44:23.441054 17902 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 04:44:23.442183 17902 hybrid_clock.cc:648] HybridClock initialized: now 1786337063442170 us; error 31 us; skew 500 ppm
Aug 10 04:44:23 dist-test-slave-bqcl krb5kdc[16801](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786337063, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.167.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 04:44:23.444505 17902 init.cc:377] Logged in from keytab as kudu/127.22.167.65@KRBTEST.COM (short username kudu)
I20260810 04:44:23.444891 17902 webserver.cc:460] Webserver started at http://127.22.167.65:38707/ using document root <none> and password file <none>
I20260810 04:44:23.445099 17902 fs_manager.cc:359] Metadata directory not provided
I20260810 04:44:23.445165 17902 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 04:44:23.446483 17902 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260810 04:44:23.447149 17966 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 04:44:23.447301 17902 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260810 04:44:23.447382 17902 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786336863388938-23197-0/minicluster-data/ts-0/data,/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786336863388938-23197-0/minicluster-data/ts-0/wal
uuid: "7e7834fc8be34efcb685d3a20756d41b"
format_stamp: "Formatted at 2026-08-10 04:44:02 on dist-test-slave-bqcl"
server_key: "eb7825fd5d6485fca2190b72dbfdb5f6"
server_key_iv: "9228f8ec69eec8ecfc92f4d36d095375"
server_key_version: "encryptionkey@0"
I20260810 04:44:23.447685 17902 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786336863388938-23197-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786336863388938-23197-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786336863388938-23197-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 04:44:23.478348 17902 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 04:44:23.480479 17902 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 04:44:23.480618 17902 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 04:44:23.480855 17902 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 04:44:23.481307 17978 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260810 04:44:23.483210 17902 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260810 04:44:23.483253 17902 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260810 04:44:23.483287 17902 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260810 04:44:23.483901 17902 ts_tablet_manager.cc:610] Registered 1 tablets
I20260810 04:44:23.483934 17902 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260810 04:44:23.484014 17978 tablet_bootstrap.cc:492] T 3819f550523a4744a19f7f353bb58809 P 7e7834fc8be34efcb685d3a20756d41b: Bootstrap starting.
I20260810 04:44:23.490634 17902 rpc_server.cc:307] RPC server started. Bound to: 127.22.167.65:37211
I20260810 04:44:23.490999 17902 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786336863388938-23197-0/minicluster-data/ts-0/data/info.pb
I20260810 04:44:23.495518 17978 log.cc:824] T 3819f550523a4744a19f7f353bb58809 P 7e7834fc8be34efcb685d3a20756d41b: Log is configured to *not* fsync() on all Append() calls
I20260810 04:44:23.495586 23197 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskemv_AM/build/release/bin/kudu as pid 17902
I20260810 04:44:23.495893 23197 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskemv_AM/build/release/bin/kudu
/tmp/dist-test-taskemv_AM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786336863388938-23197-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786336863388938-23197-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786336863388938-23197-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786336863388938-23197-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.167.66:45913
--local_ip_for_outbound_sockets=127.22.167.66
--tserver_master_addrs=127.22.167.126:38209
--webserver_port=35009
--webserver_interface=127.22.167.66
--keytab_file=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786336863388938-23197-0/krb5kdc/kudu_127.22.167.66.keytab
--principal=kudu/127.22.167.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.167.84:33205
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786336863388938-23197-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786336863388938-23197-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786336863388938-23197-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 04:44:23.497965 17978 tablet_bootstrap.cc:492] T 3819f550523a4744a19f7f353bb58809 P 7e7834fc8be34efcb685d3a20756d41b: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
Aug 10 04:44:23 dist-test-slave-bqcl krb5kdc[16801](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786337063, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.167.65@KRBTEST.COM for kudu/127.22.167.126@KRBTEST.COM
I20260810 04:44:23.498257 18157 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.167.65:37211 every 8 connection(s)
I20260810 04:44:23.498332 17978 tablet_bootstrap.cc:492] T 3819f550523a4744a19f7f353bb58809 P 7e7834fc8be34efcb685d3a20756d41b: Bootstrap complete.
I20260810 04:44:23.499583 17978 ts_tablet_manager.cc:1397] T 3819f550523a4744a19f7f353bb58809 P 7e7834fc8be34efcb685d3a20756d41b: Time spent bootstrapping tablet: real 0.016s user 0.011s sys 0.002s
I20260810 04:44:23.500864 17978 raft_consensus.cc:348] T 3819f550523a4744a19f7f353bb58809 P 7e7834fc8be34efcb685d3a20756d41b [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e7834fc8be34efcb685d3a20756d41b" member_type: VOTER last_known_addr { host: "127.22.167.65" port: 37211 } } peers { permanent_uuid: "671e8749ce234fd9a175013a5198a496" member_type: VOTER last_known_addr { host: "127.22.167.67" port: 37313 } } peers { permanent_uuid: "6ee21ed53fbc49d284bbecab757bb9db" member_type: VOTER last_known_addr { host: "127.22.167.66" port: 45913 } }
I20260810 04:44:23.501187 17978 raft_consensus.cc:729] T 3819f550523a4744a19f7f353bb58809 P 7e7834fc8be34efcb685d3a20756d41b [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7e7834fc8be34efcb685d3a20756d41b, State: Initialized, Role: FOLLOWER
I20260810 04:44:23.501331 17978 consensus_queue.cc:260] T 3819f550523a4744a19f7f353bb58809 P 7e7834fc8be34efcb685d3a20756d41b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e7834fc8be34efcb685d3a20756d41b" member_type: VOTER last_known_addr { host: "127.22.167.65" port: 37211 } } peers { permanent_uuid: "671e8749ce234fd9a175013a5198a496" member_type: VOTER last_known_addr { host: "127.22.167.67" port: 37313 } } peers { permanent_uuid: "6ee21ed53fbc49d284bbecab757bb9db" member_type: VOTER last_known_addr { host: "127.22.167.66" port: 45913 } }
I20260810 04:44:23.501603 17978 ts_tablet_manager.cc:1428] T 3819f550523a4744a19f7f353bb58809 P 7e7834fc8be34efcb685d3a20756d41b: Time spent starting tablet: real 0.002s user 0.002s sys 0.002s
I20260810 04:44:23.501905 18165 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:44:23.491607 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.126:38209 (local address 127.22.167.65:33081)
0810 04:44:23.491791 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0810 04:44:23.491834 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0810 04:44:23.492309 (+ 475us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 04:44:23.492704 (+ 395us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 04:44:23.492707 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 04:44:23.492961 (+ 254us) client_negotiation.cc:189] Negotiated authn=SASL
0810 04:44:23.493137 (+ 176us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:44:23.493142 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:44:23.494248 (+ 1106us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:44:23.494249 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:44:23.495050 (+ 801us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:44:23.495052 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:44:23.496153 (+ 1101us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:44:23.496155 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:44:23.496291 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:44:23.496845 (+ 554us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 04:44:23.496855 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0810 04:44:23.498537 (+ 1682us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 04:44:23.500248 (+ 1711us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 04:44:23.500255 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 04:44:23.500258 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 04:44:23.500471 (+ 213us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 04:44:23.500700 (+ 229us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 04:44:23.500701 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 04:44:23.500702 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 04:44:23.500807 (+ 105us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 04:44:23.501138 (+ 331us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 04:44:23.501153 (+ 15us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 04:44:23.501306 (+ 153us) client_negotiation.cc:769] Sending connection context
0810 04:44:23.501409 (+ 103us) client_negotiation.cc:240] Negotiation successful
0810 04:44:23.501580 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":101,"threads_started":1}
I20260810 04:44:23.501905 18167 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:44:23.491747 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.167.65:33081 (local address 127.22.167.126:38209)
0810 04:44:23.491924 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0810 04:44:23.491933 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 04:44:23.491974 (+ 41us) server_negotiation.cc:407] Connection header received
0810 04:44:23.492502 (+ 528us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:44:23.492507 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:44:23.492548 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:44:23.492660 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0810 04:44:23.493243 (+ 583us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:44:23.494086 (+ 843us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:44:23.495127 (+ 1041us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:44:23.495734 (+ 607us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:44:23.495834 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:44:23.498632 (+ 2798us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 04:44:23.498636 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 04:44:23.498637 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 04:44:23.498654 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0810 04:44:23.500173 (+ 1519us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 04:44:23.500544 (+ 371us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 04:44:23.500547 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 04:44:23.500547 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 04:44:23.500634 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 04:44:23.500863 (+ 229us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 04:44:23.500863 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 04:44:23.500864 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 04:44:23.501075 (+ 211us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 04:44:23.501146 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0810 04:44:23.501541 (+ 395us) server_negotiation.cc:299] Negotiation successful
0810 04:44:23.501733 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":129,"thread_start_us":90,"threads_started":1}
I20260810 04:44:23.502875 18158 heartbeater.cc:347] Connected to a master server at 127.22.167.126:38209
I20260810 04:44:23.502966 18158 heartbeater.cc:464] Registering TS with master...
I20260810 04:44:23.503130 18158 heartbeater.cc:511] Master 127.22.167.126:38209 requested a full tablet report, sending...
I20260810 04:44:23.503790 17844 ts_manager.cc:194] Registered new tserver with Master: 7e7834fc8be34efcb685d3a20756d41b (127.22.167.65:37211)
I20260810 04:44:23.504750 17844 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.167.65@KRBTEST.COM'} at 127.22.167.65:33081
I20260810 04:44:23.505673 18158 heartbeater.cc:503] Master 127.22.167.126:38209 was elected leader, sending a full tablet report...
Aug 10 04:44:23 dist-test-slave-bqcl krb5kdc[16801](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786337063, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.167.126@KRBTEST.COM for kudu/127.22.167.65@KRBTEST.COM
I20260810 04:44:23.512154 18172 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:44:23.504323 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.65:37211 (local address 127.0.0.1:56816)
0810 04:44:23.504625 (+ 302us) negotiation.cc:107] Waiting for socket to connect
0810 04:44:23.504663 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0810 04:44:23.504735 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 04:44:23.505171 (+ 436us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 04:44:23.505175 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 04:44:23.505418 (+ 243us) client_negotiation.cc:189] Negotiated authn=SASL
0810 04:44:23.505460 (+ 42us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:44:23.505462 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:44:23.506481 (+ 1019us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:44:23.506482 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:44:23.507027 (+ 545us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:44:23.507029 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:44:23.507453 (+ 424us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:44:23.507454 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:44:23.507521 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:44:23.507904 (+ 383us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 04:44:23.507908 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0810 04:44:23.509262 (+ 1354us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 04:44:23.510964 (+ 1702us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 04:44:23.510973 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 04:44:23.510975 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 04:44:23.511269 (+ 294us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 04:44:23.511521 (+ 252us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 04:44:23.511522 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 04:44:23.511523 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 04:44:23.511603 (+ 80us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 04:44:23.511899 (+ 296us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 04:44:23.511905 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 04:44:23.511944 (+ 39us) client_negotiation.cc:769] Sending connection context
0810 04:44:23.511996 (+ 52us) client_negotiation.cc:240] Negotiation successful
0810 04:44:23.512080 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":276,"thread_start_us":74,"threads_started":1}
I20260810 04:44:23.512252 18174 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:44:23.504851 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56816 (local address 127.22.167.65:37211)
0810 04:44:23.505026 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0810 04:44:23.505030 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 04:44:23.505041 (+ 11us) server_negotiation.cc:407] Connection header received
0810 04:44:23.505073 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:44:23.505075 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:44:23.505090 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:44:23.505161 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0810 04:44:23.505532 (+ 371us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:44:23.506414 (+ 882us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:44:23.507099 (+ 685us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:44:23.507389 (+ 290us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:44:23.507446 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:44:23.509344 (+ 1898us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 04:44:23.509351 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 04:44:23.509352 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 04:44:23.509358 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0810 04:44:23.510864 (+ 1506us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 04:44:23.511374 (+ 510us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 04:44:23.511377 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 04:44:23.511378 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 04:44:23.511446 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 04:44:23.511652 (+ 206us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 04:44:23.511653 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 04:44:23.511653 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 04:44:23.511830 (+ 177us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 04:44:23.511916 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0810 04:44:23.512061 (+ 145us) server_negotiation.cc:299] Negotiation successful
0810 04:44:23.512159 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":5,"server-negotiator.queue_time_us":142,"thread_start_us":103,"threads_started":1}
I20260810 04:44:23.512547 18021 tablet_service.cc:1514] Processing DeleteTablet for tablet 3819f550523a4744a19f7f353bb58809 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 04:44:03 UTC) from {username='kudu', principal='kudu/127.22.167.126@KRBTEST.COM'} at 127.0.0.1:56816
I20260810 04:44:23.512964 18178 tablet_replica.cc:330] stopping tablet replica
I20260810 04:44:23.513043 18178 raft_consensus.cc:2229] T 3819f550523a4744a19f7f353bb58809 P 7e7834fc8be34efcb685d3a20756d41b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 04:44:23.513104 18178 raft_consensus.cc:2258] T 3819f550523a4744a19f7f353bb58809 P 7e7834fc8be34efcb685d3a20756d41b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 04:44:23.513356 18178 ts_tablet_manager.cc:1905] T 3819f550523a4744a19f7f353bb58809 P 7e7834fc8be34efcb685d3a20756d41b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 04:44:23.515309 18178 ts_tablet_manager.cc:1918] T 3819f550523a4744a19f7f353bb58809 P 7e7834fc8be34efcb685d3a20756d41b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 04:44:23.515372 18178 log.cc:1196] T 3819f550523a4744a19f7f353bb58809 P 7e7834fc8be34efcb685d3a20756d41b: Deleting WAL directory at /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786336863388938-23197-0/minicluster-data/ts-0/wal/wals/3819f550523a4744a19f7f353bb58809
I20260810 04:44:23.515561 18178 ts_tablet_manager.cc:1939] T 3819f550523a4744a19f7f353bb58809 P 7e7834fc8be34efcb685d3a20756d41b: Deleting consensus metadata
I20260810 04:44:23.516007 17812 catalog_manager.cc:4841] TS 7e7834fc8be34efcb685d3a20756d41b (127.22.167.65:37211): tablet 3819f550523a4744a19f7f353bb58809 (table test-table [id=1fb0ef4c9af041439b810cf07adcc04c]) successfully deleted
W20260810 04:44:23.583456 18169 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 04:44:23.583663 18169 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 04:44:23.583695 18169 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 04:44:23.584794 18169 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 04:44:23.584834 18169 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 04:44:23.584869 18169 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.167.66
I20260810 04:44:23.586297 18169 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.167.84:33205
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786336863388938-23197-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786336863388938-23197-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786336863388938-23197-0/krb5kdc/kudu_127.22.167.66.keytab
--principal=kudu/127.22.167.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.167.66:45913
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786336863388938-23197-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.22.167.66
--webserver_port=35009
--tserver_master_addrs=127.22.167.126:38209
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18169
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.167.66
--log_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786336863388938-23197-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 03:54:01 UTC on ccycloud.quasar-icijmt.root.comops.site
build id 4577009
I20260810 04:44:23.586516 18169 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 04:44:23.586725 18169 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 04:44:23.591362 18186 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 04:44:23.591367 18190 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 04:44:23.591491 18169 server_base.cc:1034] running on GCE node
W20260810 04:44:23.591583 18192 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 04:44:23.591789 18169 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 04:44:23.592013 18169 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 04:44:23.593134 18169 hybrid_clock.cc:648] HybridClock initialized: now 1786337063593113 us; error 28 us; skew 500 ppm
Aug 10 04:44:23 dist-test-slave-bqcl krb5kdc[16801](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786337063, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.167.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 04:44:23.595685 18169 init.cc:377] Logged in from keytab as kudu/127.22.167.66@KRBTEST.COM (short username kudu)
I20260810 04:44:23.596091 18169 webserver.cc:460] Webserver started at http://127.22.167.66:35009/ using document root <none> and password file <none>
I20260810 04:44:23.596287 18169 fs_manager.cc:359] Metadata directory not provided
I20260810 04:44:23.596350 18169 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 04:44:23.597641 18169 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260810 04:44:23.598271 18201 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 04:44:23.598407 18169 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260810 04:44:23.598518 18169 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786336863388938-23197-0/minicluster-data/ts-1/data,/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786336863388938-23197-0/minicluster-data/ts-1/wal
uuid: "6ee21ed53fbc49d284bbecab757bb9db"
format_stamp: "Formatted at 2026-08-10 04:44:02 on dist-test-slave-bqcl"
server_key: "b47c454c9f14a9d83f9fcc747b87c8e1"
server_key_iv: "454b1c4173200cffd387da3c134da7d4"
server_key_version: "encryptionkey@0"
I20260810 04:44:23.598819 18169 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786336863388938-23197-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786336863388938-23197-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786336863388938-23197-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 04:44:23.605299 18169 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 04:44:23.607036 18169 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 04:44:23.607170 18169 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 04:44:23.607367 18169 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 04:44:23.607761 18212 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260810 04:44:23.609043 18169 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260810 04:44:23.609094 18169 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.001s user 0.000s sys 0.000s
I20260810 04:44:23.609127 18169 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260810 04:44:23.609738 18169 ts_tablet_manager.cc:610] Registered 1 tablets
I20260810 04:44:23.609776 18169 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260810 04:44:23.609840 18212 tablet_bootstrap.cc:492] T 3819f550523a4744a19f7f353bb58809 P 6ee21ed53fbc49d284bbecab757bb9db: Bootstrap starting.
I20260810 04:44:23.616075 18169 rpc_server.cc:307] RPC server started. Bound to: 127.22.167.66:45913
I20260810 04:44:23.616187 18397 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.167.66:45913 every 8 connection(s)
I20260810 04:44:23.616460 18169 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786336863388938-23197-0/minicluster-data/ts-1/data/info.pb
I20260810 04:44:23.620589 18212 log.cc:824] T 3819f550523a4744a19f7f353bb58809 P 6ee21ed53fbc49d284bbecab757bb9db: Log is configured to *not* fsync() on all Append() calls
I20260810 04:44:23.622970 18212 tablet_bootstrap.cc:492] T 3819f550523a4744a19f7f353bb58809 P 6ee21ed53fbc49d284bbecab757bb9db: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
Aug 10 04:44:23 dist-test-slave-bqcl krb5kdc[16801](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786337063, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.167.66@KRBTEST.COM for kudu/127.22.167.126@KRBTEST.COM
I20260810 04:44:23.623258 18212 tablet_bootstrap.cc:492] T 3819f550523a4744a19f7f353bb58809 P 6ee21ed53fbc49d284bbecab757bb9db: Bootstrap complete.
I20260810 04:44:23.624123 18212 ts_tablet_manager.cc:1397] T 3819f550523a4744a19f7f353bb58809 P 6ee21ed53fbc49d284bbecab757bb9db: Time spent bootstrapping tablet: real 0.014s user 0.004s sys 0.008s
I20260810 04:44:23.625056 18212 raft_consensus.cc:348] T 3819f550523a4744a19f7f353bb58809 P 6ee21ed53fbc49d284bbecab757bb9db [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e7834fc8be34efcb685d3a20756d41b" member_type: VOTER last_known_addr { host: "127.22.167.65" port: 37211 } } peers { permanent_uuid: "671e8749ce234fd9a175013a5198a496" member_type: VOTER last_known_addr { host: "127.22.167.67" port: 37313 } } peers { permanent_uuid: "6ee21ed53fbc49d284bbecab757bb9db" member_type: VOTER last_known_addr { host: "127.22.167.66" port: 45913 } }
I20260810 04:44:23.625316 18212 raft_consensus.cc:729] T 3819f550523a4744a19f7f353bb58809 P 6ee21ed53fbc49d284bbecab757bb9db [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6ee21ed53fbc49d284bbecab757bb9db, State: Initialized, Role: FOLLOWER
I20260810 04:44:23.625442 18212 consensus_queue.cc:260] T 3819f550523a4744a19f7f353bb58809 P 6ee21ed53fbc49d284bbecab757bb9db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e7834fc8be34efcb685d3a20756d41b" member_type: VOTER last_known_addr { host: "127.22.167.65" port: 37211 } } peers { permanent_uuid: "671e8749ce234fd9a175013a5198a496" member_type: VOTER last_known_addr { host: "127.22.167.67" port: 37313 } } peers { permanent_uuid: "6ee21ed53fbc49d284bbecab757bb9db" member_type: VOTER last_known_addr { host: "127.22.167.66" port: 45913 } }
I20260810 04:44:23.625720 18212 ts_tablet_manager.cc:1428] T 3819f550523a4744a19f7f353bb58809 P 6ee21ed53fbc49d284bbecab757bb9db: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260810 04:44:23.626219 18167 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:44:23.617155 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.167.66:37549 (local address 127.22.167.126:38209)
0810 04:44:23.617199 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0810 04:44:23.617201 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 04:44:23.617398 (+ 197us) server_negotiation.cc:407] Connection header received
0810 04:44:23.618035 (+ 637us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:44:23.618036 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:44:23.618052 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:44:23.618113 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0810 04:44:23.619036 (+ 923us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:44:23.619523 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:44:23.620976 (+ 1453us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:44:23.621217 (+ 241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:44:23.621271 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:44:23.623590 (+ 2319us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 04:44:23.623591 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 04:44:23.623591 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 04:44:23.623600 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0810 04:44:23.624850 (+ 1250us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 04:44:23.625283 (+ 433us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 04:44:23.625285 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 04:44:23.625285 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 04:44:23.625339 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 04:44:23.625565 (+ 226us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 04:44:23.625566 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 04:44:23.625566 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 04:44:23.625688 (+ 122us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 04:44:23.625749 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0810 04:44:23.626013 (+ 264us) server_negotiation.cc:299] Negotiation successful
0810 04:44:23.626109 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260810 04:44:23.626330 18402 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:44:23.617098 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.126:38209 (local address 127.22.167.66:37549)
0810 04:44:23.617258 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0810 04:44:23.617302 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0810 04:44:23.617916 (+ 614us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 04:44:23.618213 (+ 297us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 04:44:23.618219 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 04:44:23.618614 (+ 395us) client_negotiation.cc:189] Negotiated authn=SASL
0810 04:44:23.618909 (+ 295us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:44:23.618917 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:44:23.619648 (+ 731us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:44:23.619649 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:44:23.620894 (+ 1245us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:44:23.620897 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:44:23.621314 (+ 417us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:44:23.621315 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:44:23.621449 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:44:23.621934 (+ 485us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 04:44:23.621944 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0810 04:44:23.623502 (+ 1558us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 04:44:23.624922 (+ 1420us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 04:44:23.624930 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 04:44:23.624932 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 04:44:23.625206 (+ 274us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 04:44:23.625403 (+ 197us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 04:44:23.625404 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 04:44:23.625404 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 04:44:23.625506 (+ 102us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 04:44:23.625747 (+ 241us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 04:44:23.625754 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 04:44:23.625895 (+ 141us) client_negotiation.cc:769] Sending connection context
0810 04:44:23.625980 (+ 85us) client_negotiation.cc:240] Negotiation successful
0810 04:44:23.626149 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":77,"threads_started":1}
I20260810 04:44:23.626673 23197 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskemv_AM/build/release/bin/kudu as pid 18169
I20260810 04:44:23.626721 18398 heartbeater.cc:347] Connected to a master server at 127.22.167.126:38209
I20260810 04:44:23.626799 18398 heartbeater.cc:464] Registering TS with master...
I20260810 04:44:23.626893 23197 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskemv_AM/build/release/bin/kudu
/tmp/dist-test-taskemv_AM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786336863388938-23197-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786336863388938-23197-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786336863388938-23197-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786336863388938-23197-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.167.67:37313
--local_ip_for_outbound_sockets=127.22.167.67
--tserver_master_addrs=127.22.167.126:38209
--webserver_port=46093
--webserver_interface=127.22.167.67
--keytab_file=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786336863388938-23197-0/krb5kdc/kudu_127.22.167.67.keytab
--principal=kudu/127.22.167.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.167.84:33205
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786336863388938-23197-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786336863388938-23197-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786336863388938-23197-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 04:44:23.626922 18398 heartbeater.cc:511] Master 127.22.167.126:38209 requested a full tablet report, sending...
I20260810 04:44:23.627384 17844 ts_manager.cc:194] Registered new tserver with Master: 6ee21ed53fbc49d284bbecab757bb9db (127.22.167.66:45913)
I20260810 04:44:23.627826 17844 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.167.66@KRBTEST.COM'} at 127.22.167.66:37549
I20260810 04:44:23.628449 18398 heartbeater.cc:503] Master 127.22.167.126:38209 was elected leader, sending a full tablet report...
I20260810 04:44:23.631999 18172 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:44:23.627668 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.66:45913 (local address 127.0.0.1:45626)
0810 04:44:23.627718 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0810 04:44:23.627727 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0810 04:44:23.627761 (+ 34us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 04:44:23.629115 (+ 1354us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 04:44:23.629116 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 04:44:23.629118 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 04:44:23.629167 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:44:23.629168 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:44:23.630205 (+ 1037us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:44:23.630205 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:44:23.630923 (+ 718us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:44:23.630925 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:44:23.631753 (+ 828us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:44:23.631754 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:44:23.631833 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:44:23.631876 (+ 43us) client_negotiation.cc:769] Sending connection context
0810 04:44:23.631916 (+ 40us) client_negotiation.cc:240] Negotiation successful
0810 04:44:23.631949 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":34}
I20260810 04:44:23.632040 18410 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:44:23.628506 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45626 (local address 127.22.167.66:45913)
0810 04:44:23.628955 (+ 449us) server_negotiation.cc:206] Beginning negotiation
0810 04:44:23.628960 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 04:44:23.628971 (+ 11us) server_negotiation.cc:407] Connection header received
0810 04:44:23.629004 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:44:23.629005 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:44:23.629019 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:44:23.629102 (+ 83us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 04:44:23.629214 (+ 112us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:44:23.630095 (+ 881us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:44:23.631006 (+ 911us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:44:23.631680 (+ 674us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:44:23.631755 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:44:23.631816 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0810 04:44:23.631935 (+ 119us) server_negotiation.cc:299] Negotiation successful
0810 04:44:23.631974 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":419,"thread_start_us":84,"threads_started":1}
I20260810 04:44:23.632395 18301 tablet_service.cc:1514] Processing DeleteTablet for tablet 3819f550523a4744a19f7f353bb58809 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 04:44:03 UTC) from {username='kudu', principal='kudu/127.22.167.126@KRBTEST.COM'} at 127.0.0.1:45626
I20260810 04:44:23.632905 18414 tablet_replica.cc:330] stopping tablet replica
I20260810 04:44:23.632997 18414 raft_consensus.cc:2229] T 3819f550523a4744a19f7f353bb58809 P 6ee21ed53fbc49d284bbecab757bb9db [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 04:44:23.633056 18414 raft_consensus.cc:2258] T 3819f550523a4744a19f7f353bb58809 P 6ee21ed53fbc49d284bbecab757bb9db [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 04:44:23.633324 18414 ts_tablet_manager.cc:1905] T 3819f550523a4744a19f7f353bb58809 P 6ee21ed53fbc49d284bbecab757bb9db: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 04:44:23.635224 18414 ts_tablet_manager.cc:1918] T 3819f550523a4744a19f7f353bb58809 P 6ee21ed53fbc49d284bbecab757bb9db: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 04:44:23.635289 18414 log.cc:1196] T 3819f550523a4744a19f7f353bb58809 P 6ee21ed53fbc49d284bbecab757bb9db: Deleting WAL directory at /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786336863388938-23197-0/minicluster-data/ts-1/wal/wals/3819f550523a4744a19f7f353bb58809
I20260810 04:44:23.635468 18414 ts_tablet_manager.cc:1939] T 3819f550523a4744a19f7f353bb58809 P 6ee21ed53fbc49d284bbecab757bb9db: Deleting consensus metadata
I20260810 04:44:23.635882 17823 catalog_manager.cc:4841] TS 6ee21ed53fbc49d284bbecab757bb9db (127.22.167.66:45913): tablet 3819f550523a4744a19f7f353bb58809 (table test-table [id=1fb0ef4c9af041439b810cf07adcc04c]) successfully deleted
W20260810 04:44:23.708371 18407 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 04:44:23.708575 18407 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 04:44:23.708597 18407 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 04:44:23.709714 18407 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 04:44:23.709757 18407 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 04:44:23.709785 18407 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.167.67
I20260810 04:44:23.711324 18407 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.167.84:33205
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786336863388938-23197-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786336863388938-23197-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786336863388938-23197-0/krb5kdc/kudu_127.22.167.67.keytab
--principal=kudu/127.22.167.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.167.67:37313
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786336863388938-23197-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.22.167.67
--webserver_port=46093
--tserver_master_addrs=127.22.167.126:38209
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18407
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.167.67
--log_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786336863388938-23197-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 03:54:01 UTC on ccycloud.quasar-icijmt.root.comops.site
build id 4577009
I20260810 04:44:23.711544 18407 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 04:44:23.711766 18407 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 04:44:23.716558 18427 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 04:44:23.716558 18423 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 04:44:23.716746 18407 server_base.cc:1034] running on GCE node
W20260810 04:44:23.716578 18425 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 04:44:23.717016 18407 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 04:44:23.717269 18407 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 04:44:23.718410 18407 hybrid_clock.cc:648] HybridClock initialized: now 1786337063718393 us; error 28 us; skew 500 ppm
Aug 10 04:44:23 dist-test-slave-bqcl krb5kdc[16801](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786337063, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.167.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 04:44:23.720781 18407 init.cc:377] Logged in from keytab as kudu/127.22.167.67@KRBTEST.COM (short username kudu)
I20260810 04:44:23.721223 18407 webserver.cc:460] Webserver started at http://127.22.167.67:46093/ using document root <none> and password file <none>
I20260810 04:44:23.721434 18407 fs_manager.cc:359] Metadata directory not provided
I20260810 04:44:23.721498 18407 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 04:44:23.722810 18407 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.000s
I20260810 04:44:23.723414 18436 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 04:44:23.723590 18407 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260810 04:44:23.723673 18407 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786336863388938-23197-0/minicluster-data/ts-2/data,/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786336863388938-23197-0/minicluster-data/ts-2/wal
uuid: "671e8749ce234fd9a175013a5198a496"
format_stamp: "Formatted at 2026-08-10 04:44:02 on dist-test-slave-bqcl"
server_key: "162bd4b4c572401ddcc172ca46430063"
server_key_iv: "b4da5ca9ac9870928944a17c8ae016ea"
server_key_version: "encryptionkey@0"
I20260810 04:44:23.723977 18407 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786336863388938-23197-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786336863388938-23197-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786336863388938-23197-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 04:44:23.743283 18407 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 04:44:23.745312 18407 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 04:44:23.745448 18407 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 04:44:23.745648 18407 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 04:44:23.745921 18407 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 04:44:23.745954 18407 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 04:44:23.745980 18407 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 04:44:23.745998 18407 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 04:44:23.751643 18407 rpc_server.cc:307] RPC server started. Bound to: 127.22.167.67:37313
I20260810 04:44:23.751698 18622 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.167.67:37313 every 8 connection(s)
I20260810 04:44:23.752013 18407 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786336863388938-23197-0/minicluster-data/ts-2/data/info.pb
I20260810 04:44:23.752548 23197 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskemv_AM/build/release/bin/kudu as pid 18407
Aug 10 04:44:23 dist-test-slave-bqcl krb5kdc[16801](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786337063, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.167.67@KRBTEST.COM for kudu/127.22.167.126@KRBTEST.COM
I20260810 04:44:23.759410 18630 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:44:23.753062 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40782 (local address 127.22.167.126:38209)
0810 04:44:23.753154 (+ 92us) server_negotiation.cc:206] Beginning negotiation
0810 04:44:23.753155 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0810 04:44:23.753163 (+ 8us) server_negotiation.cc:407] Connection header received
0810 04:44:23.753209 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:44:23.753210 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:44:23.753219 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:44:23.753274 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0810 04:44:23.753975 (+ 701us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:44:23.754507 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:44:23.755326 (+ 819us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:44:23.755540 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:44:23.755586 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:44:23.756758 (+ 1172us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 04:44:23.756759 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 04:44:23.756759 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 04:44:23.756763 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0810 04:44:23.758329 (+ 1566us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 04:44:23.758654 (+ 325us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 04:44:23.758655 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 04:44:23.758656 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 04:44:23.758722 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 04:44:23.758897 (+ 175us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 04:44:23.758898 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 04:44:23.758899 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 04:44:23.759014 (+ 115us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 04:44:23.759187 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0810 04:44:23.759227 (+ 40us) server_negotiation.cc:299] Negotiation successful
0810 04:44:23.759340 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75,"thread_start_us":41,"threads_started":1}
I20260810 04:44:23.761335 18167 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:44:23.752703 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.167.67:43517 (local address 127.22.167.126:38209)
0810 04:44:23.752766 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0810 04:44:23.752768 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 04:44:23.752989 (+ 221us) server_negotiation.cc:407] Connection header received
0810 04:44:23.753619 (+ 630us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:44:23.753619 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:44:23.753625 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:44:23.753651 (+ 26us) server_negotiation.cc:226] Negotiated authn=SASL
0810 04:44:23.754666 (+ 1015us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:44:23.755081 (+ 415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:44:23.756332 (+ 1251us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:44:23.756524 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:44:23.756575 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:44:23.758862 (+ 2287us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 04:44:23.758863 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 04:44:23.758863 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 04:44:23.758868 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0810 04:44:23.760164 (+ 1296us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 04:44:23.760512 (+ 348us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 04:44:23.760513 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 04:44:23.760513 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 04:44:23.760566 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 04:44:23.760776 (+ 210us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 04:44:23.760777 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 04:44:23.760777 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 04:44:23.760877 (+ 100us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 04:44:23.760930 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0810 04:44:23.761178 (+ 248us) server_negotiation.cc:299] Negotiation successful
0810 04:44:23.761253 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260810 04:44:23.761494 18626 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:44:23.752691 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.126:38209 (local address 127.22.167.67:43517)
0810 04:44:23.752863 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0810 04:44:23.752909 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0810 04:44:23.753515 (+ 606us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 04:44:23.753749 (+ 234us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 04:44:23.753753 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 04:44:23.754388 (+ 635us) client_negotiation.cc:189] Negotiated authn=SASL
0810 04:44:23.754582 (+ 194us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:44:23.754589 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:44:23.755149 (+ 560us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:44:23.755150 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:44:23.756247 (+ 1097us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:44:23.756250 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:44:23.756603 (+ 353us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:44:23.756604 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:44:23.756753 (+ 149us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:44:23.757187 (+ 434us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 04:44:23.757196 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0810 04:44:23.758800 (+ 1604us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 04:44:23.760239 (+ 1439us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 04:44:23.760245 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 04:44:23.760247 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 04:44:23.760448 (+ 201us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 04:44:23.760625 (+ 177us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 04:44:23.760626 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 04:44:23.760626 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 04:44:23.760721 (+ 95us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 04:44:23.760930 (+ 209us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 04:44:23.760937 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 04:44:23.761055 (+ 118us) client_negotiation.cc:769] Sending connection context
0810 04:44:23.761147 (+ 92us) client_negotiation.cc:240] Negotiation successful
0810 04:44:23.761292 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":96,"threads_started":1}
I20260810 04:44:23.761878 18623 heartbeater.cc:347] Connected to a master server at 127.22.167.126:38209
I20260810 04:44:23.761970 18623 heartbeater.cc:464] Registering TS with master...
I20260810 04:44:23.762179 18623 heartbeater.cc:511] Master 127.22.167.126:38209 requested a full tablet report, sending...
I20260810 04:44:23.762501 17844 ts_manager.cc:194] Registered new tserver with Master: 671e8749ce234fd9a175013a5198a496 (127.22.167.67:37313)
I20260810 04:44:23.762853 17844 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.167.67@KRBTEST.COM'} at 127.22.167.67:43517
I20260810 04:44:23.770833 23197 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260810 04:44:23.773248 18167 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:44:23.771134 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40792 (local address 127.22.167.126:38209)
0810 04:44:23.771187 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0810 04:44:23.771190 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 04:44:23.771285 (+ 95us) server_negotiation.cc:407] Connection header received
0810 04:44:23.771343 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:44:23.771344 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:44:23.771373 (+ 29us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:44:23.771429 (+ 56us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 04:44:23.771509 (+ 80us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:44:23.771986 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:44:23.772629 (+ 643us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:44:23.772801 (+ 172us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:44:23.772841 (+ 40us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:44:23.772937 (+ 96us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 04:44:23.772975 (+ 38us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_AUTHENTICATION_TOKEN: Not authorized: token expired
0810 04:44:23.773191 (+ 216us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:40792: token expired
Metrics: {"server-negotiator.queue_time_us":31}
W20260810 04:44:23.773310 18167 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:40792: token expired
W20260810 04:44:23.773308 17702 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.22.167.126:38209)
I20260810 04:44:23.778527 18167 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:44:23.773538 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40802 (local address 127.22.167.126:38209)
0810 04:44:23.773583 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0810 04:44:23.773585 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 04:44:23.773593 (+ 8us) server_negotiation.cc:407] Connection header received
0810 04:44:23.773660 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:44:23.773661 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:44:23.773669 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:44:23.773708 (+ 39us) server_negotiation.cc:226] Negotiated authn=SASL
0810 04:44:23.774117 (+ 409us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:44:23.774581 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:44:23.775241 (+ 660us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:44:23.775416 (+ 175us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:44:23.775466 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:44:23.776456 (+ 990us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 04:44:23.776457 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 04:44:23.776458 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 04:44:23.776464 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0810 04:44:23.777543 (+ 1079us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 04:44:23.777835 (+ 292us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 04:44:23.777836 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 04:44:23.777837 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 04:44:23.777888 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 04:44:23.778046 (+ 158us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 04:44:23.778046 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 04:44:23.778047 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 04:44:23.778189 (+ 142us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 04:44:23.778276 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0810 04:44:23.778378 (+ 102us) server_negotiation.cc:299] Negotiation successful
0810 04:44:23.778446 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260810 04:44:23.779510 17844 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40802:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 04:44:23.780992 17844 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 04:44:23.782866 18021 tablet_service.cc:1467] Processing CreateTablet for tablet 5ddb3715fece4198b35167574f1fc53c (DEFAULT_TABLE table=test-table [id=0f6c0c4b634e40d882d617c7229fba85]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 04:44:23.783043 18021 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5ddb3715fece4198b35167574f1fc53c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 04:44:23.784547 17978 tablet_bootstrap.cc:492] T 5ddb3715fece4198b35167574f1fc53c P 7e7834fc8be34efcb685d3a20756d41b: Bootstrap starting.
I20260810 04:44:23.784693 18301 tablet_service.cc:1467] Processing CreateTablet for tablet 5ddb3715fece4198b35167574f1fc53c (DEFAULT_TABLE table=test-table [id=0f6c0c4b634e40d882d617c7229fba85]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 04:44:23.784848 17978 tablet_bootstrap.cc:654] T 5ddb3715fece4198b35167574f1fc53c P 7e7834fc8be34efcb685d3a20756d41b: Neither blocks nor log segments found. Creating new log.
I20260810 04:44:23.784843 18301 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5ddb3715fece4198b35167574f1fc53c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 04:44:23.785928 18172 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:44:23.782633 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.67:37313 (local address 127.0.0.1:48072)
0810 04:44:23.782743 (+ 110us) negotiation.cc:107] Waiting for socket to connect
0810 04:44:23.782753 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0810 04:44:23.782789 (+ 36us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 04:44:23.783043 (+ 254us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 04:44:23.783044 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 04:44:23.783046 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 04:44:23.783082 (+ 36us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:44:23.783084 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:44:23.783968 (+ 884us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:44:23.783968 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:44:23.784932 (+ 964us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:44:23.784934 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:44:23.785734 (+ 800us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:44:23.785735 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:44:23.785797 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:44:23.785800 (+ 3us) client_negotiation.cc:769] Sending connection context
0810 04:44:23.785850 (+ 50us) client_negotiation.cc:240] Negotiation successful
0810 04:44:23.785884 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":95}
I20260810 04:44:23.786185 17978 tablet_bootstrap.cc:492] T 5ddb3715fece4198b35167574f1fc53c P 7e7834fc8be34efcb685d3a20756d41b: No bootstrap required, opened a new log
I20260810 04:44:23.786233 17978 ts_tablet_manager.cc:1397] T 5ddb3715fece4198b35167574f1fc53c P 7e7834fc8be34efcb685d3a20756d41b: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260810 04:44:23.786311 18212 tablet_bootstrap.cc:492] T 5ddb3715fece4198b35167574f1fc53c P 6ee21ed53fbc49d284bbecab757bb9db: Bootstrap starting.
I20260810 04:44:23.786376 17978 raft_consensus.cc:348] T 5ddb3715fece4198b35167574f1fc53c P 7e7834fc8be34efcb685d3a20756d41b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "671e8749ce234fd9a175013a5198a496" member_type: VOTER last_known_addr { host: "127.22.167.67" port: 37313 } } peers { permanent_uuid: "7e7834fc8be34efcb685d3a20756d41b" member_type: VOTER last_known_addr { host: "127.22.167.65" port: 37211 } } peers { permanent_uuid: "6ee21ed53fbc49d284bbecab757bb9db" member_type: VOTER last_known_addr { host: "127.22.167.66" port: 45913 } }
I20260810 04:44:23.786453 17978 raft_consensus.cc:374] T 5ddb3715fece4198b35167574f1fc53c P 7e7834fc8be34efcb685d3a20756d41b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 04:44:23.786502 17978 raft_consensus.cc:729] T 5ddb3715fece4198b35167574f1fc53c P 7e7834fc8be34efcb685d3a20756d41b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7e7834fc8be34efcb685d3a20756d41b, State: Initialized, Role: FOLLOWER
I20260810 04:44:23.786554 17978 consensus_queue.cc:260] T 5ddb3715fece4198b35167574f1fc53c P 7e7834fc8be34efcb685d3a20756d41b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "671e8749ce234fd9a175013a5198a496" member_type: VOTER last_known_addr { host: "127.22.167.67" port: 37313 } } peers { permanent_uuid: "7e7834fc8be34efcb685d3a20756d41b" member_type: VOTER last_known_addr { host: "127.22.167.65" port: 37211 } } peers { permanent_uuid: "6ee21ed53fbc49d284bbecab757bb9db" member_type: VOTER last_known_addr { host: "127.22.167.66" port: 45913 } }
I20260810 04:44:23.786614 18212 tablet_bootstrap.cc:654] T 5ddb3715fece4198b35167574f1fc53c P 6ee21ed53fbc49d284bbecab757bb9db: Neither blocks nor log segments found. Creating new log.
I20260810 04:44:23.786653 17978 ts_tablet_manager.cc:1428] T 5ddb3715fece4198b35167574f1fc53c P 7e7834fc8be34efcb685d3a20756d41b: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260810 04:44:23.786870 18634 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:44:23.782717 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48072 (local address 127.22.167.67:37313)
0810 04:44:23.782907 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0810 04:44:23.782912 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 04:44:23.782920 (+ 8us) server_negotiation.cc:407] Connection header received
0810 04:44:23.782955 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:44:23.782956 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:44:23.782972 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:44:23.783046 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 04:44:23.783125 (+ 79us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:44:23.783868 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:44:23.785029 (+ 1161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:44:23.785661 (+ 632us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:44:23.785725 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:44:23.785783 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0810 04:44:23.786750 (+ 967us) server_negotiation.cc:299] Negotiation successful
0810 04:44:23.786794 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":161,"thread_start_us":119,"threads_started":1}
I20260810 04:44:23.787140 18212 tablet_bootstrap.cc:492] T 5ddb3715fece4198b35167574f1fc53c P 6ee21ed53fbc49d284bbecab757bb9db: No bootstrap required, opened a new log
I20260810 04:44:23.787184 18212 ts_tablet_manager.cc:1397] T 5ddb3715fece4198b35167574f1fc53c P 6ee21ed53fbc49d284bbecab757bb9db: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.000s
I20260810 04:44:23.787278 18212 raft_consensus.cc:348] T 5ddb3715fece4198b35167574f1fc53c P 6ee21ed53fbc49d284bbecab757bb9db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "671e8749ce234fd9a175013a5198a496" member_type: VOTER last_known_addr { host: "127.22.167.67" port: 37313 } } peers { permanent_uuid: "7e7834fc8be34efcb685d3a20756d41b" member_type: VOTER last_known_addr { host: "127.22.167.65" port: 37211 } } peers { permanent_uuid: "6ee21ed53fbc49d284bbecab757bb9db" member_type: VOTER last_known_addr { host: "127.22.167.66" port: 45913 } }
I20260810 04:44:23.787326 18212 raft_consensus.cc:374] T 5ddb3715fece4198b35167574f1fc53c P 6ee21ed53fbc49d284bbecab757bb9db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 04:44:23.787353 18212 raft_consensus.cc:729] T 5ddb3715fece4198b35167574f1fc53c P 6ee21ed53fbc49d284bbecab757bb9db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6ee21ed53fbc49d284bbecab757bb9db, State: Initialized, Role: FOLLOWER
I20260810 04:44:23.787385 18212 consensus_queue.cc:260] T 5ddb3715fece4198b35167574f1fc53c P 6ee21ed53fbc49d284bbecab757bb9db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "671e8749ce234fd9a175013a5198a496" member_type: VOTER last_known_addr { host: "127.22.167.67" port: 37313 } } peers { permanent_uuid: "7e7834fc8be34efcb685d3a20756d41b" member_type: VOTER last_known_addr { host: "127.22.167.65" port: 37211 } } peers { permanent_uuid: "6ee21ed53fbc49d284bbecab757bb9db" member_type: VOTER last_known_addr { host: "127.22.167.66" port: 45913 } }
I20260810 04:44:23.787460 18212 ts_tablet_manager.cc:1428] T 5ddb3715fece4198b35167574f1fc53c P 6ee21ed53fbc49d284bbecab757bb9db: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260810 04:44:23.787523 18519 tablet_service.cc:1467] Processing CreateTablet for tablet 5ddb3715fece4198b35167574f1fc53c (DEFAULT_TABLE table=test-table [id=0f6c0c4b634e40d882d617c7229fba85]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 04:44:23.787861 18519 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5ddb3715fece4198b35167574f1fc53c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 04:44:23.791056 18639 tablet_bootstrap.cc:492] T 5ddb3715fece4198b35167574f1fc53c P 671e8749ce234fd9a175013a5198a496: Bootstrap starting.
I20260810 04:44:23.791666 18639 tablet_bootstrap.cc:654] T 5ddb3715fece4198b35167574f1fc53c P 671e8749ce234fd9a175013a5198a496: Neither blocks nor log segments found. Creating new log.
I20260810 04:44:23.792131 18639 log.cc:824] T 5ddb3715fece4198b35167574f1fc53c P 671e8749ce234fd9a175013a5198a496: Log is configured to *not* fsync() on all Append() calls
I20260810 04:44:23.792757 18639 tablet_bootstrap.cc:492] T 5ddb3715fece4198b35167574f1fc53c P 671e8749ce234fd9a175013a5198a496: No bootstrap required, opened a new log
I20260810 04:44:23.792814 18639 ts_tablet_manager.cc:1397] T 5ddb3715fece4198b35167574f1fc53c P 671e8749ce234fd9a175013a5198a496: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260810 04:44:23.794251 18639 raft_consensus.cc:348] T 5ddb3715fece4198b35167574f1fc53c P 671e8749ce234fd9a175013a5198a496 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "671e8749ce234fd9a175013a5198a496" member_type: VOTER last_known_addr { host: "127.22.167.67" port: 37313 } } peers { permanent_uuid: "7e7834fc8be34efcb685d3a20756d41b" member_type: VOTER last_known_addr { host: "127.22.167.65" port: 37211 } } peers { permanent_uuid: "6ee21ed53fbc49d284bbecab757bb9db" member_type: VOTER last_known_addr { host: "127.22.167.66" port: 45913 } }
I20260810 04:44:23.794384 18639 raft_consensus.cc:374] T 5ddb3715fece4198b35167574f1fc53c P 671e8749ce234fd9a175013a5198a496 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 04:44:23.794418 18639 raft_consensus.cc:729] T 5ddb3715fece4198b35167574f1fc53c P 671e8749ce234fd9a175013a5198a496 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 671e8749ce234fd9a175013a5198a496, State: Initialized, Role: FOLLOWER
I20260810 04:44:23.794529 18639 consensus_queue.cc:260] T 5ddb3715fece4198b35167574f1fc53c P 671e8749ce234fd9a175013a5198a496 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "671e8749ce234fd9a175013a5198a496" member_type: VOTER last_known_addr { host: "127.22.167.67" port: 37313 } } peers { permanent_uuid: "7e7834fc8be34efcb685d3a20756d41b" member_type: VOTER last_known_addr { host: "127.22.167.65" port: 37211 } } peers { permanent_uuid: "6ee21ed53fbc49d284bbecab757bb9db" member_type: VOTER last_known_addr { host: "127.22.167.66" port: 45913 } }
I20260810 04:44:23.794763 18639 ts_tablet_manager.cc:1428] T 5ddb3715fece4198b35167574f1fc53c P 671e8749ce234fd9a175013a5198a496: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260810 04:44:23.794806 18623 heartbeater.cc:503] Master 127.22.167.126:38209 was elected leader, sending a full tablet report...
I20260810 04:44:23.848848 18171 raft_consensus.cc:482] T 5ddb3715fece4198b35167574f1fc53c P 7e7834fc8be34efcb685d3a20756d41b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 04:44:23.848971 18171 raft_consensus.cc:504] T 5ddb3715fece4198b35167574f1fc53c P 7e7834fc8be34efcb685d3a20756d41b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "671e8749ce234fd9a175013a5198a496" member_type: VOTER last_known_addr { host: "127.22.167.67" port: 37313 } } peers { permanent_uuid: "7e7834fc8be34efcb685d3a20756d41b" member_type: VOTER last_known_addr { host: "127.22.167.65" port: 37211 } } peers { permanent_uuid: "6ee21ed53fbc49d284bbecab757bb9db" member_type: VOTER last_known_addr { host: "127.22.167.66" port: 45913 } }
I20260810 04:44:23.849305 18171 leader_election.cc:288] T 5ddb3715fece4198b35167574f1fc53c P 7e7834fc8be34efcb685d3a20756d41b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 671e8749ce234fd9a175013a5198a496 (127.22.167.67:37313), 6ee21ed53fbc49d284bbecab757bb9db (127.22.167.66:45913)
I20260810 04:44:23.852131 18634 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:44:23.849606 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.167.65:43691 (local address 127.22.167.67:37313)
0810 04:44:23.849689 (+ 83us) server_negotiation.cc:206] Beginning negotiation
0810 04:44:23.849691 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 04:44:23.849699 (+ 8us) server_negotiation.cc:407] Connection header received
0810 04:44:23.849722 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:44:23.849723 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:44:23.849731 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:44:23.849787 (+ 56us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 04:44:23.849935 (+ 148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:44:23.850453 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:44:23.851436 (+ 983us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:44:23.851793 (+ 357us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:44:23.851854 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:44:23.851863 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0810 04:44:23.852035 (+ 172us) server_negotiation.cc:299] Negotiation successful
0810 04:44:23.852073 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260810 04:44:23.852125 18165 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:44:23.849488 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.67:37313 (local address 127.22.167.65:43691)
0810 04:44:23.849541 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0810 04:44:23.849553 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0810 04:44:23.849617 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 04:44:23.849801 (+ 184us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 04:44:23.849802 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 04:44:23.849804 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 04:44:23.849872 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:44:23.849874 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:44:23.850527 (+ 653us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:44:23.850528 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:44:23.851345 (+ 817us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:44:23.851347 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:44:23.851862 (+ 515us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:44:23.851863 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:44:23.851944 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:44:23.851967 (+ 23us) client_negotiation.cc:769] Sending connection context
0810 04:44:23.852012 (+ 45us) client_negotiation.cc:240] Negotiation successful
0810 04:44:23.852050 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":27}
I20260810 04:44:23.852221 18410 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:44:23.849594 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.167.65:41669 (local address 127.22.167.66:45913)
0810 04:44:23.849649 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0810 04:44:23.849651 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 04:44:23.849783 (+ 132us) server_negotiation.cc:407] Connection header received
0810 04:44:23.849820 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:44:23.849821 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:44:23.849848 (+ 27us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:44:23.849899 (+ 51us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 04:44:23.850042 (+ 143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:44:23.850542 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:44:23.851527 (+ 985us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:44:23.851887 (+ 360us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:44:23.851946 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:44:23.851955 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0810 04:44:23.852119 (+ 164us) server_negotiation.cc:299] Negotiation successful
0810 04:44:23.852159 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260810 04:44:23.852236 18644 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:44:23.849533 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.66:45913 (local address 127.22.167.65:41669)
0810 04:44:23.849720 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0810 04:44:23.849729 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0810 04:44:23.849777 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 04:44:23.849904 (+ 127us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 04:44:23.849905 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 04:44:23.849907 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 04:44:23.849983 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:44:23.849985 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:44:23.850613 (+ 628us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:44:23.850614 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:44:23.851444 (+ 830us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:44:23.851446 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:44:23.851977 (+ 531us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:44:23.851978 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:44:23.852056 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:44:23.852059 (+ 3us) client_negotiation.cc:769] Sending connection context
0810 04:44:23.852104 (+ 45us) client_negotiation.cc:240] Negotiation successful
0810 04:44:23.852142 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":166,"thread_start_us":99,"threads_started":1}
I20260810 04:44:23.852490 18547 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5ddb3715fece4198b35167574f1fc53c" candidate_uuid: "7e7834fc8be34efcb685d3a20756d41b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "671e8749ce234fd9a175013a5198a496" is_pre_election: true
I20260810 04:44:23.852564 18330 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5ddb3715fece4198b35167574f1fc53c" candidate_uuid: "7e7834fc8be34efcb685d3a20756d41b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6ee21ed53fbc49d284bbecab757bb9db" is_pre_election: true
I20260810 04:44:23.852648 18547 raft_consensus.cc:2454] T 5ddb3715fece4198b35167574f1fc53c P 671e8749ce234fd9a175013a5198a496 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e7834fc8be34efcb685d3a20756d41b in term 0.
I20260810 04:44:23.852721 18330 raft_consensus.cc:2454] T 5ddb3715fece4198b35167574f1fc53c P 6ee21ed53fbc49d284bbecab757bb9db [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e7834fc8be34efcb685d3a20756d41b in term 0.
I20260810 04:44:23.852900 17968 leader_election.cc:302] T 5ddb3715fece4198b35167574f1fc53c P 7e7834fc8be34efcb685d3a20756d41b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 671e8749ce234fd9a175013a5198a496, 7e7834fc8be34efcb685d3a20756d41b; no voters:
I20260810 04:44:23.853055 18171 raft_consensus.cc:2781] T 5ddb3715fece4198b35167574f1fc53c P 7e7834fc8be34efcb685d3a20756d41b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 04:44:23.853124 18171 raft_consensus.cc:482] T 5ddb3715fece4198b35167574f1fc53c P 7e7834fc8be34efcb685d3a20756d41b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 04:44:23.853142 18171 raft_consensus.cc:3037] T 5ddb3715fece4198b35167574f1fc53c P 7e7834fc8be34efcb685d3a20756d41b [term 0 FOLLOWER]: Advancing to term 1
I20260810 04:44:23.853797 18171 raft_consensus.cc:504] T 5ddb3715fece4198b35167574f1fc53c P 7e7834fc8be34efcb685d3a20756d41b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "671e8749ce234fd9a175013a5198a496" member_type: VOTER last_known_addr { host: "127.22.167.67" port: 37313 } } peers { permanent_uuid: "7e7834fc8be34efcb685d3a20756d41b" member_type: VOTER last_known_addr { host: "127.22.167.65" port: 37211 } } peers { permanent_uuid: "6ee21ed53fbc49d284bbecab757bb9db" member_type: VOTER last_known_addr { host: "127.22.167.66" port: 45913 } }
I20260810 04:44:23.853909 18171 leader_election.cc:288] T 5ddb3715fece4198b35167574f1fc53c P 7e7834fc8be34efcb685d3a20756d41b [CANDIDATE]: Term 1 election: Requested vote from peers 671e8749ce234fd9a175013a5198a496 (127.22.167.67:37313), 6ee21ed53fbc49d284bbecab757bb9db (127.22.167.66:45913)
I20260810 04:44:23.854045 18547 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5ddb3715fece4198b35167574f1fc53c" candidate_uuid: "7e7834fc8be34efcb685d3a20756d41b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "671e8749ce234fd9a175013a5198a496"
I20260810 04:44:23.854086 18330 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5ddb3715fece4198b35167574f1fc53c" candidate_uuid: "7e7834fc8be34efcb685d3a20756d41b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6ee21ed53fbc49d284bbecab757bb9db"
I20260810 04:44:23.854135 18547 raft_consensus.cc:3037] T 5ddb3715fece4198b35167574f1fc53c P 671e8749ce234fd9a175013a5198a496 [term 0 FOLLOWER]: Advancing to term 1
I20260810 04:44:23.854182 18330 raft_consensus.cc:3037] T 5ddb3715fece4198b35167574f1fc53c P 6ee21ed53fbc49d284bbecab757bb9db [term 0 FOLLOWER]: Advancing to term 1
I20260810 04:44:23.855015 18547 raft_consensus.cc:2454] T 5ddb3715fece4198b35167574f1fc53c P 671e8749ce234fd9a175013a5198a496 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e7834fc8be34efcb685d3a20756d41b in term 1.
I20260810 04:44:23.855172 17968 leader_election.cc:302] T 5ddb3715fece4198b35167574f1fc53c P 7e7834fc8be34efcb685d3a20756d41b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 671e8749ce234fd9a175013a5198a496, 7e7834fc8be34efcb685d3a20756d41b; no voters:
I20260810 04:44:23.855273 18171 raft_consensus.cc:2781] T 5ddb3715fece4198b35167574f1fc53c P 7e7834fc8be34efcb685d3a20756d41b [term 1 FOLLOWER]: Leader election won for term 1
I20260810 04:44:23.855429 18171 raft_consensus.cc:686] T 5ddb3715fece4198b35167574f1fc53c P 7e7834fc8be34efcb685d3a20756d41b [term 1 LEADER]: Becoming Leader. State: Replica: 7e7834fc8be34efcb685d3a20756d41b, State: Running, Role: LEADER
I20260810 04:44:23.855530 18330 raft_consensus.cc:2454] T 5ddb3715fece4198b35167574f1fc53c P 6ee21ed53fbc49d284bbecab757bb9db [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e7834fc8be34efcb685d3a20756d41b in term 1.
I20260810 04:44:23.855516 18171 consensus_queue.cc:237] T 5ddb3715fece4198b35167574f1fc53c P 7e7834fc8be34efcb685d3a20756d41b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "671e8749ce234fd9a175013a5198a496" member_type: VOTER last_known_addr { host: "127.22.167.67" port: 37313 } } peers { permanent_uuid: "7e7834fc8be34efcb685d3a20756d41b" member_type: VOTER last_known_addr { host: "127.22.167.65" port: 37211 } } peers { permanent_uuid: "6ee21ed53fbc49d284bbecab757bb9db" member_type: VOTER last_known_addr { host: "127.22.167.66" port: 45913 } }
I20260810 04:44:23.856213 17844 catalog_manager.cc:5495] T 5ddb3715fece4198b35167574f1fc53c P 7e7834fc8be34efcb685d3a20756d41b reported cstate change: term changed from 0 to 1, leader changed from <none> to 7e7834fc8be34efcb685d3a20756d41b (127.22.167.65). New cstate: current_term: 1 leader_uuid: "7e7834fc8be34efcb685d3a20756d41b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "671e8749ce234fd9a175013a5198a496" member_type: VOTER last_known_addr { host: "127.22.167.67" port: 37313 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7e7834fc8be34efcb685d3a20756d41b" member_type: VOTER last_known_addr { host: "127.22.167.65" port: 37211 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6ee21ed53fbc49d284bbecab757bb9db" member_type: VOTER last_known_addr { host: "127.22.167.66" port: 45913 } health_report { overall_health: UNKNOWN } } }
I20260810 04:44:23.861097 18174 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:44:23.859008 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56824 (local address 127.22.167.65:37211)
0810 04:44:23.859048 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0810 04:44:23.859050 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 04:44:23.859058 (+ 8us) server_negotiation.cc:407] Connection header received
0810 04:44:23.859131 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:44:23.859132 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:44:23.859148 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:44:23.859200 (+ 52us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 04:44:23.859323 (+ 123us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:44:23.859771 (+ 448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:44:23.860436 (+ 665us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:44:23.860635 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:44:23.860687 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:44:23.860824 (+ 137us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 04:44:23.860915 (+ 91us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 04:44:23.860973 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0810 04:44:23.861019 (+ 46us) server_negotiation.cc:299] Negotiation successful
0810 04:44:23.861037 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260810 04:44:23.862386 18547 raft_consensus.cc:1261] T 5ddb3715fece4198b35167574f1fc53c P 671e8749ce234fd9a175013a5198a496 [term 1 FOLLOWER]: Refusing update from remote peer 7e7834fc8be34efcb685d3a20756d41b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 04:44:23.862398 18330 raft_consensus.cc:1261] T 5ddb3715fece4198b35167574f1fc53c P 6ee21ed53fbc49d284bbecab757bb9db [term 1 FOLLOWER]: Refusing update from remote peer 7e7834fc8be34efcb685d3a20756d41b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 04:44:23.862695 18645 consensus_queue.cc:1035] T 5ddb3715fece4198b35167574f1fc53c P 7e7834fc8be34efcb685d3a20756d41b [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ee21ed53fbc49d284bbecab757bb9db" member_type: VOTER last_known_addr { host: "127.22.167.66" port: 45913 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 04:44:23.862807 18171 consensus_queue.cc:1035] T 5ddb3715fece4198b35167574f1fc53c P 7e7834fc8be34efcb685d3a20756d41b [LEADER]: Connected to new peer: Peer: permanent_uuid: "671e8749ce234fd9a175013a5198a496" member_type: VOTER last_known_addr { host: "127.22.167.67" port: 37313 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 04:44:23.864737 18651 mvcc.cc:205] Tried to move back new op lower bound from 7316836613578088448 to 7316836613552672768. Current Snapshot: MvccSnapshot[applied={T|T < 7316836613578088448}]
I20260810 04:44:23.866734 17843 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40802:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 04:44:23.866811 17843 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40802:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 04:44:23.867473 17843 catalog_manager.cc:5782] T 00000000000000000000000000000000 P f49432ad9f9f4c31b0380574d0265634: Sending DeleteTablet for 3 replicas of tablet 5ddb3715fece4198b35167574f1fc53c
I20260810 04:44:23.867790 18021 tablet_service.cc:1514] Processing DeleteTablet for tablet 5ddb3715fece4198b35167574f1fc53c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 04:44:23 UTC) from {username='kudu', principal='kudu/127.22.167.126@KRBTEST.COM'} at 127.0.0.1:56816
I20260810 04:44:23.867781 18519 tablet_service.cc:1514] Processing DeleteTablet for tablet 5ddb3715fece4198b35167574f1fc53c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 04:44:23 UTC) from {username='kudu', principal='kudu/127.22.167.126@KRBTEST.COM'} at 127.0.0.1:48072
I20260810 04:44:23.867800 18301 tablet_service.cc:1514] Processing DeleteTablet for tablet 5ddb3715fece4198b35167574f1fc53c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 04:44:23 UTC) from {username='kudu', principal='kudu/127.22.167.126@KRBTEST.COM'} at 127.0.0.1:45626
I20260810 04:44:23.867872 18178 tablet_replica.cc:330] T 5ddb3715fece4198b35167574f1fc53c P 7e7834fc8be34efcb685d3a20756d41b: stopping tablet replica
I20260810 04:44:23.867913 18178 raft_consensus.cc:2229] T 5ddb3715fece4198b35167574f1fc53c P 7e7834fc8be34efcb685d3a20756d41b [term 1 LEADER]: Raft consensus shutting down.
I20260810 04:44:23.867940 18414 tablet_replica.cc:330] T 5ddb3715fece4198b35167574f1fc53c P 6ee21ed53fbc49d284bbecab757bb9db: stopping tablet replica
I20260810 04:44:23.867981 18178 raft_consensus.cc:2258] T 5ddb3715fece4198b35167574f1fc53c P 7e7834fc8be34efcb685d3a20756d41b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 04:44:23.867986 18414 raft_consensus.cc:2229] T 5ddb3715fece4198b35167574f1fc53c P 6ee21ed53fbc49d284bbecab757bb9db [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 04:44:23.868000 23197 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskemv_AM/build/release/bin/kudu with pid 17902
I20260810 04:44:23.868049 18414 raft_consensus.cc:2258] T 5ddb3715fece4198b35167574f1fc53c P 6ee21ed53fbc49d284bbecab757bb9db [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 04:44:23.887760 18664 tablet_replica.cc:330] T 5ddb3715fece4198b35167574f1fc53c P 671e8749ce234fd9a175013a5198a496: stopping tablet replica
I20260810 04:44:23.887809 18414 ts_tablet_manager.cc:1905] T 5ddb3715fece4198b35167574f1fc53c P 6ee21ed53fbc49d284bbecab757bb9db: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 04:44:23.887965 18664 raft_consensus.cc:2229] T 5ddb3715fece4198b35167574f1fc53c P 671e8749ce234fd9a175013a5198a496 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 04:44:23.888094 18664 raft_consensus.cc:2258] T 5ddb3715fece4198b35167574f1fc53c P 671e8749ce234fd9a175013a5198a496 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 04:44:23.888485 18664 ts_tablet_manager.cc:1905] T 5ddb3715fece4198b35167574f1fc53c P 671e8749ce234fd9a175013a5198a496: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 04:44:23.889389 18414 ts_tablet_manager.cc:1918] T 5ddb3715fece4198b35167574f1fc53c P 6ee21ed53fbc49d284bbecab757bb9db: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 04:44:23.889447 18414 log.cc:1196] T 5ddb3715fece4198b35167574f1fc53c P 6ee21ed53fbc49d284bbecab757bb9db: Deleting WAL directory at /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786336863388938-23197-0/minicluster-data/ts-1/wal/wals/5ddb3715fece4198b35167574f1fc53c
I20260810 04:44:23.889650 18414 ts_tablet_manager.cc:1939] T 5ddb3715fece4198b35167574f1fc53c P 6ee21ed53fbc49d284bbecab757bb9db: Deleting consensus metadata
I20260810 04:44:23.889686 18664 ts_tablet_manager.cc:1918] T 5ddb3715fece4198b35167574f1fc53c P 671e8749ce234fd9a175013a5198a496: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 04:44:23.889756 18664 log.cc:1196] T 5ddb3715fece4198b35167574f1fc53c P 671e8749ce234fd9a175013a5198a496: Deleting WAL directory at /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786336863388938-23197-0/minicluster-data/ts-2/wal/wals/5ddb3715fece4198b35167574f1fc53c
I20260810 04:44:23.889976 17823 catalog_manager.cc:4841] TS 6ee21ed53fbc49d284bbecab757bb9db (127.22.167.66:45913): tablet 5ddb3715fece4198b35167574f1fc53c (table test-table [id=0f6c0c4b634e40d882d617c7229fba85]) successfully deleted
I20260810 04:44:23.890029 18664 ts_tablet_manager.cc:1939] T 5ddb3715fece4198b35167574f1fc53c P 671e8749ce234fd9a175013a5198a496: Deleting consensus metadata
I20260810 04:44:23.890434 17813 catalog_manager.cc:4841] TS 671e8749ce234fd9a175013a5198a496 (127.22.167.67:37313): tablet 5ddb3715fece4198b35167574f1fc53c (table test-table [id=0f6c0c4b634e40d882d617c7229fba85]) successfully deleted
W20260810 04:44:23.892642 17812 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.22.167.65:37211 (error 108)
I20260810 04:44:23.892908 23197 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskemv_AM/build/release/bin/kudu with pid 18169
I20260810 04:44:23.893064 18172 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:44:23.892943 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.65:37211 (local address 127.0.0.1:56840)
0810 04:44:23.892989 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0810 04:44:23.893033 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.167.65:37211: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
W20260810 04:44:23.897964 17812 catalog_manager.cc:4584] TS 7e7834fc8be34efcb685d3a20756d41b (127.22.167.65:37211): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 5ddb3715fece4198b35167574f1fc53c: Network error: Client connection negotiation failed: client connection to 127.22.167.65:37211: connect: Connection refused (error 111)
I20260810 04:44:23.910056 23197 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskemv_AM/build/release/bin/kudu with pid 18407
I20260810 04:44:23.928577 23197 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskemv_AM/build/release/bin/kudu with pid 17780
I20260810 04:44:43.940912 23197 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskemv_AM/build/release/bin/kudu
/tmp/dist-test-taskemv_AM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786336863388938-23197-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786336863388938-23197-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786336863388938-23197-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786336863388938-23197-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.167.126:38209
--webserver_interface=127.22.167.126
--webserver_port=35293
--keytab_file=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786336863388938-23197-0/krb5kdc/kudu_127.22.167.126.keytab
--principal=kudu/127.22.167.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.167.84:33205
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.167.126:38209
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786336863388938-23197-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786336863388938-23197-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786336863388938-23197-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 04:44:44.022356 18667 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 04:44:44.022543 18667 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 04:44:44.022573 18667 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 04:44:44.023659 18667 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260810 04:44:44.023696 18667 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260810 04:44:44.023713 18667 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260810 04:44:44.023727 18667 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 04:44:44.023742 18667 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 04:44:44.023758 18667 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 04:44:44.023782 18667 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 04:44:44.023805 18667 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 04:44:44.025228 18667 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.167.84:33205
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786336863388938-23197-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786336863388938-23197-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.22.167.126:38209
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786336863388938-23197-0/krb5kdc/kudu_127.22.167.126.keytab
--principal=kudu/127.22.167.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.167.126:38209
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786336863388938-23197-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.167.126
--webserver_port=35293
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18667
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786336863388938-23197-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 03:54:01 UTC on ccycloud.quasar-icijmt.root.comops.site
build id 4577009
I20260810 04:44:44.025473 18667 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 04:44:44.025673 18667 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 04:44:44.030180 18676 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 04:44:44.030244 18682 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 04:44:44.030260 18680 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 04:44:44.030284 18667 server_base.cc:1034] running on GCE node
I20260810 04:44:44.030598 18667 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 04:44:44.030869 18667 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 04:44:44.032001 18667 hybrid_clock.cc:648] HybridClock initialized: now 1786337084031991 us; error 28 us; skew 500 ppm
Aug 10 04:44:44 dist-test-slave-bqcl krb5kdc[16801](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786337084, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.167.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 04:44:44.034451 18667 init.cc:377] Logged in from keytab as kudu/127.22.167.126@KRBTEST.COM (short username kudu)
I20260810 04:44:44.034868 18667 webserver.cc:460] Webserver started at http://127.22.167.126:35293/ using document root <none> and password file <none>
I20260810 04:44:44.035059 18667 fs_manager.cc:359] Metadata directory not provided
I20260810 04:44:44.035099 18667 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 04:44:44.036266 18667 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260810 04:44:44.036921 18694 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 04:44:44.037071 18667 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260810 04:44:44.037144 18667 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786336863388938-23197-0/minicluster-data/master-0/data,/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786336863388938-23197-0/minicluster-data/master-0/wal
uuid: "f49432ad9f9f4c31b0380574d0265634"
format_stamp: "Formatted at 2026-08-10 04:44:02 on dist-test-slave-bqcl"
server_key: "934a4a74932f95d0458d6d58d3085223"
server_key_iv: "995b6f0cd97aa1961bf7aa35fd595378"
server_key_version: "encryptionkey@0"
I20260810 04:44:44.037436 18667 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786336863388938-23197-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786336863388938-23197-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786336863388938-23197-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 04:44:44.057005 18667 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 04:44:44.059182 18667 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 04:44:44.059315 18667 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 04:44:44.063326 18667 rpc_server.cc:307] RPC server started. Bound to: 127.22.167.126:38209
I20260810 04:44:44.063360 18776 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.167.126:38209 every 8 connection(s)
I20260810 04:44:44.063709 18667 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786336863388938-23197-0/minicluster-data/master-0/data/info.pb
I20260810 04:44:44.065081 18777 sys_catalog.cc:263] Verifying existing consensus state
I20260810 04:44:44.065699 18777 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f49432ad9f9f4c31b0380574d0265634: Bootstrap starting.
I20260810 04:44:44.066831 23197 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskemv_AM/build/release/bin/kudu as pid 18667
I20260810 04:44:44.067075 23197 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskemv_AM/build/release/bin/kudu
/tmp/dist-test-taskemv_AM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786336863388938-23197-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786336863388938-23197-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786336863388938-23197-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786336863388938-23197-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.167.65:37211
--local_ip_for_outbound_sockets=127.22.167.65
--tserver_master_addrs=127.22.167.126:38209
--webserver_port=38707
--webserver_interface=127.22.167.65
--keytab_file=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786336863388938-23197-0/krb5kdc/kudu_127.22.167.65.keytab
--principal=kudu/127.22.167.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.167.84:33205
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786336863388938-23197-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786336863388938-23197-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786336863388938-23197-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 04:44:44.074740 18777 log.cc:824] T 00000000000000000000000000000000 P f49432ad9f9f4c31b0380574d0265634: Log is configured to *not* fsync() on all Append() calls
I20260810 04:44:44.076853 18777 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f49432ad9f9f4c31b0380574d0265634: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=8 ignored=0} mutations{seen=8 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260810 04:44:44.077100 18777 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f49432ad9f9f4c31b0380574d0265634: Bootstrap complete.
I20260810 04:44:44.079162 18777 raft_consensus.cc:348] T 00000000000000000000000000000000 P f49432ad9f9f4c31b0380574d0265634 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f49432ad9f9f4c31b0380574d0265634" member_type: VOTER last_known_addr { host: "127.22.167.126" port: 38209 } }
I20260810 04:44:44.079376 18777 raft_consensus.cc:729] T 00000000000000000000000000000000 P f49432ad9f9f4c31b0380574d0265634 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: f49432ad9f9f4c31b0380574d0265634, State: Initialized, Role: FOLLOWER
I20260810 04:44:44.079526 18777 consensus_queue.cc:260] T 00000000000000000000000000000000 P f49432ad9f9f4c31b0380574d0265634 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f49432ad9f9f4c31b0380574d0265634" member_type: VOTER last_known_addr { host: "127.22.167.126" port: 38209 } }
I20260810 04:44:44.079609 18777 raft_consensus.cc:388] T 00000000000000000000000000000000 P f49432ad9f9f4c31b0380574d0265634 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 04:44:44.079658 18777 raft_consensus.cc:482] T 00000000000000000000000000000000 P f49432ad9f9f4c31b0380574d0265634 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 04:44:44.079703 18777 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f49432ad9f9f4c31b0380574d0265634 [term 2 FOLLOWER]: Advancing to term 3
I20260810 04:44:44.080610 18777 raft_consensus.cc:504] T 00000000000000000000000000000000 P f49432ad9f9f4c31b0380574d0265634 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f49432ad9f9f4c31b0380574d0265634" member_type: VOTER last_known_addr { host: "127.22.167.126" port: 38209 } }
I20260810 04:44:44.080703 18777 leader_election.cc:302] T 00000000000000000000000000000000 P f49432ad9f9f4c31b0380574d0265634 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f49432ad9f9f4c31b0380574d0265634; no voters:
I20260810 04:44:44.080852 18777 leader_election.cc:288] T 00000000000000000000000000000000 P f49432ad9f9f4c31b0380574d0265634 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260810 04:44:44.080901 18784 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f49432ad9f9f4c31b0380574d0265634 [term 3 FOLLOWER]: Leader election won for term 3
I20260810 04:44:44.081022 18784 raft_consensus.cc:686] T 00000000000000000000000000000000 P f49432ad9f9f4c31b0380574d0265634 [term 3 LEADER]: Becoming Leader. State: Replica: f49432ad9f9f4c31b0380574d0265634, State: Running, Role: LEADER
I20260810 04:44:44.081103 18777 sys_catalog.cc:564] T 00000000000000000000000000000000 P f49432ad9f9f4c31b0380574d0265634 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 04:44:44.081115 18784 consensus_queue.cc:237] T 00000000000000000000000000000000 P f49432ad9f9f4c31b0380574d0265634 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f49432ad9f9f4c31b0380574d0265634" member_type: VOTER last_known_addr { host: "127.22.167.126" port: 38209 } }
I20260810 04:44:44.081378 18784 sys_catalog.cc:455] T 00000000000000000000000000000000 P f49432ad9f9f4c31b0380574d0265634 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "f49432ad9f9f4c31b0380574d0265634" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f49432ad9f9f4c31b0380574d0265634" member_type: VOTER last_known_addr { host: "127.22.167.126" port: 38209 } } }
I20260810 04:44:44.081449 18784 sys_catalog.cc:458] T 00000000000000000000000000000000 P f49432ad9f9f4c31b0380574d0265634 [sys.catalog]: This master's current role is: LEADER
I20260810 04:44:44.081564 18784 sys_catalog.cc:455] T 00000000000000000000000000000000 P f49432ad9f9f4c31b0380574d0265634 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f49432ad9f9f4c31b0380574d0265634. Latest consensus state: current_term: 3 leader_uuid: "f49432ad9f9f4c31b0380574d0265634" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f49432ad9f9f4c31b0380574d0265634" member_type: VOTER last_known_addr { host: "127.22.167.126" port: 38209 } } }
I20260810 04:44:44.081624 18784 sys_catalog.cc:458] T 00000000000000000000000000000000 P f49432ad9f9f4c31b0380574d0265634 [sys.catalog]: This master's current role is: LEADER
I20260810 04:44:44.081914 18792 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 04:44:44.082695 18792 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 04:44:44.082791 18792 catalog_manager.cc:1259] Loaded cluster ID: bd36be342a8e4f998163ba9d6f1c530e
I20260810 04:44:44.082815 18792 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 04:44:44.084123 18792 catalog_manager.cc:1504] Loading token signing keys...
I20260810 04:44:44.084295 18792 catalog_manager.cc:5879] T 00000000000000000000000000000000 P f49432ad9f9f4c31b0380574d0265634: Loaded TSK: 0
I20260810 04:44:44.084342 18792 catalog_manager.cc:5879] T 00000000000000000000000000000000 P f49432ad9f9f4c31b0380574d0265634: Loaded TSK: 1
F20260810 04:44:44.084556 18792 token_signing_key.cc:66] Check failed: _s.ok() Bad status: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f35a3a439ad google::LogMessage::Fail() at ??:0
@ 0x7f35a3a4688a google::LogMessage::SendToLog() at ??:0
@ 0x7f35a3a434de google::LogMessage::Flush() at ??:0
@ 0x7f35a3a44a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f35a61b0104 kudu::security::TokenSigningPrivateKey::TokenSigningPrivateKey() at ??:0
@ 0x7f35a61add41 kudu::security::TokenSigner::ImportKeys() at ??:0
@ 0x7f35ab99d1cd kudu::master::CatalogManager::LoadTskEntries() at ??:0
@ 0x7f35ab99d48e kudu::master::CatalogManager::InitTokenSigner() at ??:0
@ 0x7f35ab99d594 _ZNSt17_Function_handlerIFN4kudu6StatusEvEZNS0_6master14CatalogManager24PrepareForLeadershipTaskEvEUlvE3_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x7f35ab963c48 _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_.isra.591 at ??:0
@ 0x7f35ab96dad0 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7f35a59110d3 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7f35a5906954 kudu::Thread::SuperviseThread() at ??:0
@ 0x7f35a676a6db start_thread at ??:0
@ 0x7f35a254161f clone at ??:0
W20260810 04:44:44.165364 18779 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 04:44:44.165553 18779 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 04:44:44.165576 18779 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 04:44:44.166709 18779 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 04:44:44.166749 18779 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 04:44:44.166774 18779 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.167.65
I20260810 04:44:44.168174 18779 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.167.84:33205
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786336863388938-23197-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786336863388938-23197-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786336863388938-23197-0/krb5kdc/kudu_127.22.167.65.keytab
--principal=kudu/127.22.167.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.167.65:37211
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786336863388938-23197-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.22.167.65
--webserver_port=38707
--tserver_master_addrs=127.22.167.126:38209
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18779
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.167.65
--log_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786336863388938-23197-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 03:54:01 UTC on ccycloud.quasar-icijmt.root.comops.site
build id 4577009
I20260810 04:44:44.168398 18779 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 04:44:44.168607 18779 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 04:44:44.173327 18842 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 04:44:44.173405 18846 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 04:44:44.173363 18841 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 04:44:44.173601 18779 server_base.cc:1034] running on GCE node
I20260810 04:44:44.173810 18779 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 04:44:44.174036 18779 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 04:44:44.175184 18779 hybrid_clock.cc:648] HybridClock initialized: now 1786337084175169 us; error 33 us; skew 500 ppm
Aug 10 04:44:44 dist-test-slave-bqcl krb5kdc[16801](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786337084, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.167.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 04:44:44.177762 18779 init.cc:377] Logged in from keytab as kudu/127.22.167.65@KRBTEST.COM (short username kudu)
I20260810 04:44:44.178184 18779 webserver.cc:460] Webserver started at http://127.22.167.65:38707/ using document root <none> and password file <none>
I20260810 04:44:44.178390 18779 fs_manager.cc:359] Metadata directory not provided
I20260810 04:44:44.178437 18779 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 04:44:44.179710 18779 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260810 04:44:44.180351 18861 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 04:44:44.180505 18779 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260810 04:44:44.180591 18779 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786336863388938-23197-0/minicluster-data/ts-0/data,/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786336863388938-23197-0/minicluster-data/ts-0/wal
uuid: "7e7834fc8be34efcb685d3a20756d41b"
format_stamp: "Formatted at 2026-08-10 04:44:02 on dist-test-slave-bqcl"
server_key: "eb7825fd5d6485fca2190b72dbfdb5f6"
server_key_iv: "9228f8ec69eec8ecfc92f4d36d095375"
server_key_version: "encryptionkey@0"
I20260810 04:44:44.180889 18779 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786336863388938-23197-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786336863388938-23197-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786336863388938-23197-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 04:44:44.212158 18779 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 04:44:44.214287 18779 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 04:44:44.214423 18779 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 04:44:44.214639 18779 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 04:44:44.215083 18869 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260810 04:44:44.217097 18779 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260810 04:44:44.217155 18779 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260810 04:44:44.217190 18779 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260810 04:44:44.217820 18779 ts_tablet_manager.cc:610] Registered 1 tablets
I20260810 04:44:44.217854 18779 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260810 04:44:44.217937 18869 tablet_bootstrap.cc:492] T 5ddb3715fece4198b35167574f1fc53c P 7e7834fc8be34efcb685d3a20756d41b: Bootstrap starting.
I20260810 04:44:44.224643 18779 rpc_server.cc:307] RPC server started. Bound to: 127.22.167.65:37211
I20260810 04:44:44.224768 19052 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.167.65:37211 every 8 connection(s)
I20260810 04:44:44.225034 18779 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786336863388938-23197-0/minicluster-data/ts-0/data/info.pb
I20260810 04:44:44.229645 18869 log.cc:824] T 5ddb3715fece4198b35167574f1fc53c P 7e7834fc8be34efcb685d3a20756d41b: Log is configured to *not* fsync() on all Append() calls
I20260810 04:44:44.232307 18869 tablet_bootstrap.cc:492] T 5ddb3715fece4198b35167574f1fc53c P 7e7834fc8be34efcb685d3a20756d41b: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260810 04:44:44.232620 18869 tablet_bootstrap.cc:492] T 5ddb3715fece4198b35167574f1fc53c P 7e7834fc8be34efcb685d3a20756d41b: Bootstrap complete.
I20260810 04:44:44.232944 23197 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskemv_AM/build/release/bin/kudu as pid 18779
I20260810 04:44:44.233173 23197 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskemv_AM/build/release/bin/kudu
/tmp/dist-test-taskemv_AM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786336863388938-23197-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786336863388938-23197-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786336863388938-23197-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786336863388938-23197-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.167.66:45913
--local_ip_for_outbound_sockets=127.22.167.66
--tserver_master_addrs=127.22.167.126:38209
--webserver_port=35009
--webserver_interface=127.22.167.66
--keytab_file=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786336863388938-23197-0/krb5kdc/kudu_127.22.167.66.keytab
--principal=kudu/127.22.167.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.167.84:33205
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786336863388938-23197-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786336863388938-23197-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786336863388938-23197-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 04:44:44.233842 18869 ts_tablet_manager.cc:1397] T 5ddb3715fece4198b35167574f1fc53c P 7e7834fc8be34efcb685d3a20756d41b: Time spent bootstrapping tablet: real 0.016s user 0.007s sys 0.005s
I20260810 04:44:44.235293 18869 raft_consensus.cc:348] T 5ddb3715fece4198b35167574f1fc53c P 7e7834fc8be34efcb685d3a20756d41b [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "671e8749ce234fd9a175013a5198a496" member_type: VOTER last_known_addr { host: "127.22.167.67" port: 37313 } } peers { permanent_uuid: "7e7834fc8be34efcb685d3a20756d41b" member_type: VOTER last_known_addr { host: "127.22.167.65" port: 37211 } } peers { permanent_uuid: "6ee21ed53fbc49d284bbecab757bb9db" member_type: VOTER last_known_addr { host: "127.22.167.66" port: 45913 } }
I20260810 04:44:44.235628 18869 raft_consensus.cc:729] T 5ddb3715fece4198b35167574f1fc53c P 7e7834fc8be34efcb685d3a20756d41b [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7e7834fc8be34efcb685d3a20756d41b, State: Initialized, Role: FOLLOWER
I20260810 04:44:44.235801 18869 consensus_queue.cc:260] T 5ddb3715fece4198b35167574f1fc53c P 7e7834fc8be34efcb685d3a20756d41b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "671e8749ce234fd9a175013a5198a496" member_type: VOTER last_known_addr { host: "127.22.167.67" port: 37313 } } peers { permanent_uuid: "7e7834fc8be34efcb685d3a20756d41b" member_type: VOTER last_known_addr { host: "127.22.167.65" port: 37211 } } peers { permanent_uuid: "6ee21ed53fbc49d284bbecab757bb9db" member_type: VOTER last_known_addr { host: "127.22.167.66" port: 45913 } }
I20260810 04:44:44.236111 18869 ts_tablet_manager.cc:1428] T 5ddb3715fece4198b35167574f1fc53c P 7e7834fc8be34efcb685d3a20756d41b: Time spent starting tablet: real 0.002s user 0.006s sys 0.000s
W20260810 04:44:44.276839 19056 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 04:44:44.225754 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.126:38209 (local address 127.22.167.65:56173)
0810 04:44:44.225928 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0810 04:44:44.225963 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0810 04:44:44.226437 (+ 474us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 04:44:44.276623 (+ 50186us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.167.126:38209: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":103,"threads_started":1}
W20260810 04:44:44.277102 19054 heartbeater.cc:644] Failed to heartbeat to 127.22.167.126:38209 (0 consecutive failures): Network error: Failed to ping master at 127.22.167.126:38209: Client connection negotiation failed: client connection to 127.22.167.126:38209: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260810 04:44:44.277446 19056 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:44:44.277337 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.126:38209 (local address 127.22.167.65:35877)
0810 04:44:44.277384 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0810 04:44:44.277425 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.167.126:38209: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":26}
I20260810 04:44:44.277765 19056 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:44:44.277683 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.126:38209 (local address 127.22.167.65:45267)
0810 04:44:44.277721 (+ 38us) negotiation.cc:107] Waiting for socket to connect
0810 04:44:44.277744 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.167.126:38209: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":24}
W20260810 04:44:44.277864 19054 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260810 04:44:44.318524 19059 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 04:44:44.318708 19059 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 04:44:44.318729 19059 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 04:44:44.319782 19059 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 04:44:44.319813 19059 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 04:44:44.319835 19059 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.167.66
I20260810 04:44:44.321167 19059 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.167.84:33205
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786336863388938-23197-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786336863388938-23197-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786336863388938-23197-0/krb5kdc/kudu_127.22.167.66.keytab
--principal=kudu/127.22.167.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.167.66:45913
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786336863388938-23197-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.22.167.66
--webserver_port=35009
--tserver_master_addrs=127.22.167.126:38209
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19059
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.167.66
--log_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786336863388938-23197-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 03:54:01 UTC on ccycloud.quasar-icijmt.root.comops.site
build id 4577009
I20260810 04:44:44.321334 19059 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 04:44:44.321497 19059 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 04:44:44.326309 19066 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 04:44:44.326270 19059 server_base.cc:1034] running on GCE node
W20260810 04:44:44.326304 19071 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 04:44:44.326274 19068 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 04:44:44.326732 19059 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 04:44:44.326948 19059 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 04:44:44.328097 19059 hybrid_clock.cc:648] HybridClock initialized: now 1786337084328082 us; error 28 us; skew 500 ppm
Aug 10 04:44:44 dist-test-slave-bqcl krb5kdc[16801](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786337084, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.167.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 04:44:44.330551 19059 init.cc:377] Logged in from keytab as kudu/127.22.167.66@KRBTEST.COM (short username kudu)
I20260810 04:44:44.330932 19059 webserver.cc:460] Webserver started at http://127.22.167.66:35009/ using document root <none> and password file <none>
I20260810 04:44:44.331125 19059 fs_manager.cc:359] Metadata directory not provided
I20260810 04:44:44.331173 19059 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 04:44:44.332374 19059 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260810 04:44:44.332865 19081 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 04:44:44.333012 19059 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260810 04:44:44.333102 19059 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786336863388938-23197-0/minicluster-data/ts-1/data,/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786336863388938-23197-0/minicluster-data/ts-1/wal
uuid: "6ee21ed53fbc49d284bbecab757bb9db"
format_stamp: "Formatted at 2026-08-10 04:44:02 on dist-test-slave-bqcl"
server_key: "b47c454c9f14a9d83f9fcc747b87c8e1"
server_key_iv: "454b1c4173200cffd387da3c134da7d4"
server_key_version: "encryptionkey@0"
I20260810 04:44:44.333388 19059 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786336863388938-23197-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786336863388938-23197-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786336863388938-23197-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 04:44:44.344985 19059 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 04:44:44.346599 19059 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 04:44:44.346724 19059 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 04:44:44.346904 19059 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 04:44:44.347174 19059 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 04:44:44.347208 19059 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 04:44:44.347234 19059 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 04:44:44.347257 19059 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 04:44:44.352792 19059 rpc_server.cc:307] RPC server started. Bound to: 127.22.167.66:45913
I20260810 04:44:44.352854 19288 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.167.66:45913 every 8 connection(s)
I20260810 04:44:44.353143 19059 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786336863388938-23197-0/minicluster-data/ts-1/data/info.pb
I20260810 04:44:44.354063 19296 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:44:44.353709 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.126:38209 (local address 127.22.167.66:44843)
0810 04:44:44.353853 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0810 04:44:44.353917 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.167.126:38209: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":78,"threads_started":1}
W20260810 04:44:44.354455 19290 heartbeater.cc:644] Failed to heartbeat to 127.22.167.126:38209 (0 consecutive failures): Network error: Failed to ping master at 127.22.167.126:38209: Client connection negotiation failed: client connection to 127.22.167.126:38209: connect: Connection refused (error 111)
I20260810 04:44:44.354784 19296 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:44:44.354688 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.126:38209 (local address 127.22.167.66:37047)
0810 04:44:44.354735 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0810 04:44:44.354763 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.167.126:38209: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
I20260810 04:44:44.355136 19296 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:44:44.355016 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.126:38209 (local address 127.22.167.66:56535)
0810 04:44:44.355080 (+ 64us) negotiation.cc:107] Waiting for socket to connect
0810 04:44:44.355112 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.167.126:38209: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260810 04:44:44.355243 19290 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260810 04:44:44.360139 23197 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskemv_AM/build/release/bin/kudu as pid 19059
I20260810 04:44:44.360375 23197 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskemv_AM/build/release/bin/kudu
/tmp/dist-test-taskemv_AM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786336863388938-23197-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786336863388938-23197-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786336863388938-23197-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786336863388938-23197-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.167.67:37313
--local_ip_for_outbound_sockets=127.22.167.67
--tserver_master_addrs=127.22.167.126:38209
--webserver_port=46093
--webserver_interface=127.22.167.67
--keytab_file=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786336863388938-23197-0/krb5kdc/kudu_127.22.167.67.keytab
--principal=kudu/127.22.167.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.167.84:33205
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786336863388938-23197-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786336863388938-23197-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786336863388938-23197-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 04:44:44.440531 19299 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 04:44:44.440739 19299 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 04:44:44.440760 19299 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 04:44:44.441864 19299 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 04:44:44.441908 19299 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 04:44:44.441933 19299 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.167.67
I20260810 04:44:44.443360 19299 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.167.84:33205
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786336863388938-23197-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786336863388938-23197-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786336863388938-23197-0/krb5kdc/kudu_127.22.167.67.keytab
--principal=kudu/127.22.167.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.167.67:37313
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786336863388938-23197-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.22.167.67
--webserver_port=46093
--tserver_master_addrs=127.22.167.126:38209
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19299
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.167.67
--log_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786336863388938-23197-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 03:54:01 UTC on ccycloud.quasar-icijmt.root.comops.site
build id 4577009
I20260810 04:44:44.443588 19299 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 04:44:44.443790 19299 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 04:44:44.448484 19311 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 04:44:44.448518 19306 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 04:44:44.448536 19309 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 04:44:44.448577 19299 server_base.cc:1034] running on GCE node
I20260810 04:44:44.448866 19299 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 04:44:44.449077 19299 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 04:44:44.450181 19299 hybrid_clock.cc:648] HybridClock initialized: now 1786337084450169 us; error 27 us; skew 500 ppm
Aug 10 04:44:44 dist-test-slave-bqcl krb5kdc[16801](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786337084, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.167.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 04:44:44.452641 19299 init.cc:377] Logged in from keytab as kudu/127.22.167.67@KRBTEST.COM (short username kudu)
I20260810 04:44:44.453022 19299 webserver.cc:460] Webserver started at http://127.22.167.67:46093/ using document root <none> and password file <none>
I20260810 04:44:44.453213 19299 fs_manager.cc:359] Metadata directory not provided
I20260810 04:44:44.453258 19299 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 04:44:44.454536 19299 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260810 04:44:44.455060 19325 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 04:44:44.455238 19299 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260810 04:44:44.455332 19299 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786336863388938-23197-0/minicluster-data/ts-2/data,/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786336863388938-23197-0/minicluster-data/ts-2/wal
uuid: "671e8749ce234fd9a175013a5198a496"
format_stamp: "Formatted at 2026-08-10 04:44:02 on dist-test-slave-bqcl"
server_key: "162bd4b4c572401ddcc172ca46430063"
server_key_iv: "b4da5ca9ac9870928944a17c8ae016ea"
server_key_version: "encryptionkey@0"
I20260810 04:44:44.455631 19299 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786336863388938-23197-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786336863388938-23197-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786336863388938-23197-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 04:44:44.464754 19299 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 04:44:44.466593 19299 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 04:44:44.466719 19299 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 04:44:44.466908 19299 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 04:44:44.467201 19299 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 04:44:44.467237 19299 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 04:44:44.467264 19299 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 04:44:44.467285 19299 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 04:44:44.472846 19299 rpc_server.cc:307] RPC server started. Bound to: 127.22.167.67:37313
I20260810 04:44:44.472906 19512 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.167.67:37313 every 8 connection(s)
I20260810 04:44:44.473198 19299 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786336863388938-23197-0/minicluster-data/ts-2/data/info.pb
I20260810 04:44:44.474217 19517 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:44:44.473792 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.126:38209 (local address 127.22.167.67:33963)
0810 04:44:44.473960 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0810 04:44:44.474029 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.167.126:38209: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":68,"threads_started":1}
W20260810 04:44:44.474562 19514 heartbeater.cc:644] Failed to heartbeat to 127.22.167.126:38209 (0 consecutive failures): Network error: Failed to ping master at 127.22.167.126:38209: Client connection negotiation failed: client connection to 127.22.167.126:38209: connect: Connection refused (error 111)
I20260810 04:44:44.474802 19517 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:44:44.474721 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.126:38209 (local address 127.22.167.67:54329)
0810 04:44:44.474758 (+ 37us) negotiation.cc:107] Waiting for socket to connect
0810 04:44:44.474783 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.167.126:38209: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":24}
I20260810 04:44:44.475011 19517 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:44:44.474964 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.126:38209 (local address 127.22.167.67:57555)
0810 04:44:44.474984 (+ 20us) negotiation.cc:107] Waiting for socket to connect
0810 04:44:44.475000 (+ 16us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.167.126:38209: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":11}
W20260810 04:44:44.475081 19514 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260810 04:44:44.475607 23197 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskemv_AM/build/release/bin/kudu as pid 19299
I20260810 04:44:45.278439 19522 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:44:45.278218 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.126:38209 (local address 127.22.167.65:46959)
0810 04:44:45.278337 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0810 04:44:45.278373 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.167.126:38209: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":101,"thread_start_us":62,"threads_started":1}
I20260810 04:44:45.355783 19523 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:44:45.355507 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.126:38209 (local address 127.22.167.66:47163)
0810 04:44:45.355678 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0810 04:44:45.355721 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.167.126:38209: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":153,"thread_start_us":53,"threads_started":1}
I20260810 04:44:45.475581 19527 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:44:45.475358 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.126:38209 (local address 127.22.167.67:34109)
0810 04:44:45.475468 (+ 110us) negotiation.cc:107] Waiting for socket to connect
0810 04:44:45.475528 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.167.126:38209: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":96,"thread_start_us":58,"threads_started":1}
I20260810 04:44:45.681970 19532 raft_consensus.cc:482] T 5ddb3715fece4198b35167574f1fc53c P 7e7834fc8be34efcb685d3a20756d41b [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 04:44:45.682111 19532 raft_consensus.cc:504] T 5ddb3715fece4198b35167574f1fc53c P 7e7834fc8be34efcb685d3a20756d41b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "671e8749ce234fd9a175013a5198a496" member_type: VOTER last_known_addr { host: "127.22.167.67" port: 37313 } } peers { permanent_uuid: "7e7834fc8be34efcb685d3a20756d41b" member_type: VOTER last_known_addr { host: "127.22.167.65" port: 37211 } } peers { permanent_uuid: "6ee21ed53fbc49d284bbecab757bb9db" member_type: VOTER last_known_addr { host: "127.22.167.66" port: 45913 } }
I20260810 04:44:45.682449 19532 leader_election.cc:288] T 5ddb3715fece4198b35167574f1fc53c P 7e7834fc8be34efcb685d3a20756d41b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 671e8749ce234fd9a175013a5198a496 (127.22.167.67:37313), 6ee21ed53fbc49d284bbecab757bb9db (127.22.167.66:45913)
Aug 10 04:44:45 dist-test-slave-bqcl krb5kdc[16801](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786337084, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.167.65@KRBTEST.COM for kudu/127.22.167.66@KRBTEST.COM
Aug 10 04:44:45 dist-test-slave-bqcl krb5kdc[16801](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786337084, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.167.65@KRBTEST.COM for kudu/127.22.167.67@KRBTEST.COM
I20260810 04:44:45.699435 19522 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:44:45.682589 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.67:37313 (local address 127.22.167.65:40513)
0810 04:44:45.682731 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0810 04:44:45.682742 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0810 04:44:45.682800 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 04:44:45.683807 (+ 1007us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 04:44:45.683810 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 04:44:45.684283 (+ 473us) client_negotiation.cc:189] Negotiated authn=SASL
0810 04:44:45.684565 (+ 282us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:44:45.684573 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:44:45.686288 (+ 1715us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:44:45.686290 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:44:45.687600 (+ 1310us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:44:45.687603 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:44:45.688443 (+ 840us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:44:45.688444 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:44:45.688615 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:44:45.693428 (+ 4813us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 04:44:45.693438 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0810 04:44:45.695719 (+ 2281us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 04:44:45.697435 (+ 1716us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 04:44:45.697436 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 04:44:45.697436 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 04:44:45.697765 (+ 329us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 04:44:45.698078 (+ 313us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 04:44:45.698079 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 04:44:45.698080 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 04:44:45.698303 (+ 223us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 04:44:45.698847 (+ 544us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 04:44:45.698854 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 04:44:45.699020 (+ 166us) client_negotiation.cc:769] Sending connection context
0810 04:44:45.699140 (+ 120us) client_negotiation.cc:240] Negotiation successful
0810 04:44:45.699319 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":18,"mutex_wait_us":82}
I20260810 04:44:45.699463 19535 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:44:45.682693 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.167.65:40513 (local address 127.22.167.67:37313)
0810 04:44:45.682882 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0810 04:44:45.682889 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 04:44:45.682920 (+ 31us) server_negotiation.cc:407] Connection header received
0810 04:44:45.683508 (+ 588us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:44:45.683512 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:44:45.683575 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:44:45.683762 (+ 187us) server_negotiation.cc:226] Negotiated authn=SASL
0810 04:44:45.684708 (+ 946us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:44:45.686180 (+ 1472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:44:45.687688 (+ 1508us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:44:45.688323 (+ 635us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:44:45.688428 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:44:45.695820 (+ 7392us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 04:44:45.695824 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 04:44:45.695824 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 04:44:45.695835 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0810 04:44:45.697379 (+ 1544us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 04:44:45.697851 (+ 472us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 04:44:45.697854 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 04:44:45.697854 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 04:44:45.697987 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 04:44:45.698400 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 04:44:45.698401 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 04:44:45.698402 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 04:44:45.698767 (+ 365us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 04:44:45.698858 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0810 04:44:45.699202 (+ 344us) server_negotiation.cc:299] Negotiation successful
0810 04:44:45.699346 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":145,"thread_start_us":55,"threads_started":1}
I20260810 04:44:45.699492 19534 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:44:45.682594 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.66:45913 (local address 127.22.167.65:49459)
0810 04:44:45.682872 (+ 278us) negotiation.cc:107] Waiting for socket to connect
0810 04:44:45.682882 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0810 04:44:45.682929 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 04:44:45.683807 (+ 878us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 04:44:45.683810 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 04:44:45.684295 (+ 485us) client_negotiation.cc:189] Negotiated authn=SASL
0810 04:44:45.684565 (+ 270us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:44:45.684573 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:44:45.686288 (+ 1715us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:44:45.686290 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:44:45.687600 (+ 1310us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:44:45.687603 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:44:45.688444 (+ 841us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:44:45.688445 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:44:45.688615 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:44:45.693428 (+ 4813us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 04:44:45.693438 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0810 04:44:45.695300 (+ 1862us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 04:44:45.697358 (+ 2058us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 04:44:45.697366 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 04:44:45.697368 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 04:44:45.697774 (+ 406us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 04:44:45.698078 (+ 304us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 04:44:45.698079 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 04:44:45.698080 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 04:44:45.698303 (+ 223us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 04:44:45.698847 (+ 544us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 04:44:45.698854 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 04:44:45.699020 (+ 166us) client_negotiation.cc:769] Sending connection context
0810 04:44:45.699135 (+ 115us) client_negotiation.cc:240] Negotiation successful
0810 04:44:45.699383 (+ 248us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":267,"mutex_wait_us":234,"thread_start_us":72,"threads_started":1}
I20260810 04:44:45.699463 19537 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:44:45.682694 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.167.65:49459 (local address 127.22.167.66:45913)
0810 04:44:45.682835 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0810 04:44:45.682848 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0810 04:44:45.682925 (+ 77us) server_negotiation.cc:407] Connection header received
0810 04:44:45.683518 (+ 593us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:44:45.683521 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:44:45.683575 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:44:45.683733 (+ 158us) server_negotiation.cc:226] Negotiated authn=SASL
0810 04:44:45.684712 (+ 979us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:44:45.686180 (+ 1468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:44:45.687689 (+ 1509us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:44:45.688317 (+ 628us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:44:45.688428 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:44:45.695427 (+ 6999us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 04:44:45.695431 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 04:44:45.695432 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 04:44:45.695445 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0810 04:44:45.697269 (+ 1824us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 04:44:45.697853 (+ 584us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 04:44:45.697855 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 04:44:45.697856 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 04:44:45.697994 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 04:44:45.698400 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 04:44:45.698401 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 04:44:45.698402 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 04:44:45.698767 (+ 365us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 04:44:45.698858 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0810 04:44:45.699207 (+ 349us) server_negotiation.cc:299] Negotiation successful
0810 04:44:45.699346 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":96,"thread_start_us":63,"threads_started":1}
W20260810 04:44:45.700332 18864 leader_election.cc:341] T 5ddb3715fece4198b35167574f1fc53c P 7e7834fc8be34efcb685d3a20756d41b [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 671e8749ce234fd9a175013a5198a496 (127.22.167.67:37313): Not found: Tablet not found: 5ddb3715fece4198b35167574f1fc53c
W20260810 04:44:45.700424 18866 leader_election.cc:341] T 5ddb3715fece4198b35167574f1fc53c P 7e7834fc8be34efcb685d3a20756d41b [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 6ee21ed53fbc49d284bbecab757bb9db (127.22.167.66:45913): Not found: Tablet not found: 5ddb3715fece4198b35167574f1fc53c
I20260810 04:44:45.700500 18866 leader_election.cc:302] T 5ddb3715fece4198b35167574f1fc53c P 7e7834fc8be34efcb685d3a20756d41b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e7834fc8be34efcb685d3a20756d41b; no voters: 671e8749ce234fd9a175013a5198a496, 6ee21ed53fbc49d284bbecab757bb9db
I20260810 04:44:45.700670 19532 raft_consensus.cc:2726] T 5ddb3715fece4198b35167574f1fc53c P 7e7834fc8be34efcb685d3a20756d41b [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260810 04:44:46.279085 19538 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:44:46.278876 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.126:38209 (local address 127.22.167.65:39915)
0810 04:44:46.278995 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0810 04:44:46.279034 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.167.126:38209: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":100,"thread_start_us":61,"threads_started":1}
I20260810 04:44:46.356349 19541 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:44:46.356158 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.126:38209 (local address 127.22.167.66:38269)
0810 04:44:46.356273 (+ 115us) negotiation.cc:107] Waiting for socket to connect
0810 04:44:46.356307 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.167.126:38209: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":100,"thread_start_us":58,"threads_started":1}
I20260810 04:44:46.476163 19542 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:44:46.475956 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.126:38209 (local address 127.22.167.67:60081)
0810 04:44:46.476088 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0810 04:44:46.476123 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.167.126:38209: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":71,"threads_started":1}
I20260810 04:44:47.279691 19543 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:44:47.279465 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.126:38209 (local address 127.22.167.65:38823)
0810 04:44:47.279600 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0810 04:44:47.279640 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.167.126:38209: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":68,"threads_started":1}
I20260810 04:44:47.348749 19546 raft_consensus.cc:482] T 5ddb3715fece4198b35167574f1fc53c P 7e7834fc8be34efcb685d3a20756d41b [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 04:44:47.348855 19546 raft_consensus.cc:504] T 5ddb3715fece4198b35167574f1fc53c P 7e7834fc8be34efcb685d3a20756d41b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "671e8749ce234fd9a175013a5198a496" member_type: VOTER last_known_addr { host: "127.22.167.67" port: 37313 } } peers { permanent_uuid: "7e7834fc8be34efcb685d3a20756d41b" member_type: VOTER last_known_addr { host: "127.22.167.65" port: 37211 } } peers { permanent_uuid: "6ee21ed53fbc49d284bbecab757bb9db" member_type: VOTER last_known_addr { host: "127.22.167.66" port: 45913 } }
I20260810 04:44:47.349056 19546 leader_election.cc:288] T 5ddb3715fece4198b35167574f1fc53c P 7e7834fc8be34efcb685d3a20756d41b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 671e8749ce234fd9a175013a5198a496 (127.22.167.67:37313), 6ee21ed53fbc49d284bbecab757bb9db (127.22.167.66:45913)
W20260810 04:44:47.349514 18866 leader_election.cc:341] T 5ddb3715fece4198b35167574f1fc53c P 7e7834fc8be34efcb685d3a20756d41b [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 6ee21ed53fbc49d284bbecab757bb9db (127.22.167.66:45913): Not found: Tablet not found: 5ddb3715fece4198b35167574f1fc53c
W20260810 04:44:47.349612 18864 leader_election.cc:341] T 5ddb3715fece4198b35167574f1fc53c P 7e7834fc8be34efcb685d3a20756d41b [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 671e8749ce234fd9a175013a5198a496 (127.22.167.67:37313): Not found: Tablet not found: 5ddb3715fece4198b35167574f1fc53c
I20260810 04:44:47.349655 18864 leader_election.cc:302] T 5ddb3715fece4198b35167574f1fc53c P 7e7834fc8be34efcb685d3a20756d41b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e7834fc8be34efcb685d3a20756d41b; no voters: 671e8749ce234fd9a175013a5198a496, 6ee21ed53fbc49d284bbecab757bb9db
I20260810 04:44:47.349738 19546 raft_consensus.cc:2726] T 5ddb3715fece4198b35167574f1fc53c P 7e7834fc8be34efcb685d3a20756d41b [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260810 04:44:47.356912 19549 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:44:47.356721 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.126:38209 (local address 127.22.167.66:50169)
0810 04:44:47.356849 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0810 04:44:47.356877 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.167.126:38209: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":71,"threads_started":1}
I20260810 04:44:47.476789 19550 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:44:47.476571 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.126:38209 (local address 127.22.167.67:38825)
0810 04:44:47.476710 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0810 04:44:47.476744 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.167.126:38209: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":82,"threads_started":1}
I20260810 04:44:48.280378 19551 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:44:48.280141 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.126:38209 (local address 127.22.167.65:37641)
0810 04:44:48.280288 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0810 04:44:48.280327 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.167.126:38209: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":80,"threads_started":1}
I20260810 04:44:48.357542 19553 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:44:48.357314 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.126:38209 (local address 127.22.167.66:56653)
0810 04:44:48.357460 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0810 04:44:48.357497 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.167.126:38209: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":60,"threads_started":1}
I20260810 04:44:48.477443 19555 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:44:48.477222 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.126:38209 (local address 127.22.167.67:36007)
0810 04:44:48.477362 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0810 04:44:48.477402 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.167.126:38209: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":76,"threads_started":1}
I20260810 04:44:49.281073 19556 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:44:49.280833 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.126:38209 (local address 127.22.167.65:55079)
0810 04:44:49.280981 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0810 04:44:49.281020 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.167.126:38209: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":61,"threads_started":1}
I20260810 04:44:49.358184 19557 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:44:49.357943 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.126:38209 (local address 127.22.167.66:55663)
0810 04:44:49.358079 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0810 04:44:49.358116 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.167.126:38209: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":60,"threads_started":1}
I20260810 04:44:49.478048 19558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:44:49.477819 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.126:38209 (local address 127.22.167.67:38413)
0810 04:44:49.477968 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0810 04:44:49.478005 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.167.126:38209: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":78,"threads_started":1}
I20260810 04:44:49.877547 19560 raft_consensus.cc:482] T 5ddb3715fece4198b35167574f1fc53c P 7e7834fc8be34efcb685d3a20756d41b [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 04:44:49.877648 19560 raft_consensus.cc:504] T 5ddb3715fece4198b35167574f1fc53c P 7e7834fc8be34efcb685d3a20756d41b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "671e8749ce234fd9a175013a5198a496" member_type: VOTER last_known_addr { host: "127.22.167.67" port: 37313 } } peers { permanent_uuid: "7e7834fc8be34efcb685d3a20756d41b" member_type: VOTER last_known_addr { host: "127.22.167.65" port: 37211 } } peers { permanent_uuid: "6ee21ed53fbc49d284bbecab757bb9db" member_type: VOTER last_known_addr { host: "127.22.167.66" port: 45913 } }
I20260810 04:44:49.877789 19560 leader_election.cc:288] T 5ddb3715fece4198b35167574f1fc53c P 7e7834fc8be34efcb685d3a20756d41b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 671e8749ce234fd9a175013a5198a496 (127.22.167.67:37313), 6ee21ed53fbc49d284bbecab757bb9db (127.22.167.66:45913)
W20260810 04:44:49.878208 18864 leader_election.cc:341] T 5ddb3715fece4198b35167574f1fc53c P 7e7834fc8be34efcb685d3a20756d41b [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 671e8749ce234fd9a175013a5198a496 (127.22.167.67:37313): Not found: Tablet not found: 5ddb3715fece4198b35167574f1fc53c
W20260810 04:44:49.878310 18866 leader_election.cc:341] T 5ddb3715fece4198b35167574f1fc53c P 7e7834fc8be34efcb685d3a20756d41b [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 6ee21ed53fbc49d284bbecab757bb9db (127.22.167.66:45913): Not found: Tablet not found: 5ddb3715fece4198b35167574f1fc53c
I20260810 04:44:49.878352 18866 leader_election.cc:302] T 5ddb3715fece4198b35167574f1fc53c P 7e7834fc8be34efcb685d3a20756d41b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e7834fc8be34efcb685d3a20756d41b; no voters: 671e8749ce234fd9a175013a5198a496, 6ee21ed53fbc49d284bbecab757bb9db
I20260810 04:44:49.878417 19560 raft_consensus.cc:2726] T 5ddb3715fece4198b35167574f1fc53c P 7e7834fc8be34efcb685d3a20756d41b [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260810 04:44:50.281759 19561 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:44:50.281534 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.126:38209 (local address 127.22.167.65:33779)
0810 04:44:50.281669 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0810 04:44:50.281706 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.167.126:38209: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":81,"threads_started":1}
I20260810 04:44:50.358745 19564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:44:50.358535 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.126:38209 (local address 127.22.167.66:37129)
0810 04:44:50.358668 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0810 04:44:50.358702 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.167.126:38209: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":75,"threads_started":1}
I20260810 04:44:50.478660 19565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:44:50.478465 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.126:38209 (local address 127.22.167.67:47169)
0810 04:44:50.478579 (+ 114us) negotiation.cc:107] Waiting for socket to connect
0810 04:44:50.478613 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.167.126:38209: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":98,"thread_start_us":59,"threads_started":1}
I20260810 04:44:51.282374 19567 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:44:51.282173 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.126:38209 (local address 127.22.167.65:58787)
0810 04:44:51.282292 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0810 04:44:51.282323 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.167.126:38209: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":103,"thread_start_us":67,"threads_started":1}
I20260810 04:44:51.359339 19569 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:44:51.359121 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.126:38209 (local address 127.22.167.66:60185)
0810 04:44:51.359257 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0810 04:44:51.359298 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.167.126:38209: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":72,"threads_started":1}
I20260810 04:44:51.479223 19571 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:44:51.479029 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.126:38209 (local address 127.22.167.67:37521)
0810 04:44:51.479151 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0810 04:44:51.479180 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.167.126:38209: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":67,"threads_started":1}
I20260810 04:44:52.282981 19577 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:44:52.282769 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.126:38209 (local address 127.22.167.65:34697)
0810 04:44:52.282893 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0810 04:44:52.282930 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.167.126:38209: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":106,"thread_start_us":63,"threads_started":1}
I20260810 04:44:52.359907 19578 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:44:52.359715 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.126:38209 (local address 127.22.167.66:55157)
0810 04:44:52.359824 (+ 109us) negotiation.cc:107] Waiting for socket to connect
0810 04:44:52.359859 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.167.126:38209: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":94,"thread_start_us":57,"threads_started":1}
I20260810 04:44:52.479888 19579 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:44:52.479695 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.126:38209 (local address 127.22.167.67:52633)
0810 04:44:52.479811 (+ 116us) negotiation.cc:107] Waiting for socket to connect
0810 04:44:52.479845 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.167.126:38209: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":103,"thread_start_us":68,"threads_started":1}
I20260810 04:44:53.283597 19580 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:44:53.283390 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.126:38209 (local address 127.22.167.65:49237)
0810 04:44:53.283512 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0810 04:44:53.283545 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.167.126:38209: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":106,"thread_start_us":65,"threads_started":1}
I20260810 04:44:53.360520 19581 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:44:53.360310 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.126:38209 (local address 127.22.167.66:39005)
0810 04:44:53.360440 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0810 04:44:53.360476 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.167.126:38209: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":78,"threads_started":1}
I20260810 04:44:53.480491 19584 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:44:53.480291 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.126:38209 (local address 127.22.167.67:41179)
0810 04:44:53.480414 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0810 04:44:53.480447 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.167.126:38209: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":76,"threads_started":1}
I20260810 04:44:54.284260 19585 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:44:54.284037 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.126:38209 (local address 127.22.167.65:35021)
0810 04:44:54.284172 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0810 04:44:54.284209 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.167.126:38209: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":89,"threads_started":1}
I20260810 04:44:54.361075 19586 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:44:54.360889 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.126:38209 (local address 127.22.167.66:58059)
0810 04:44:54.360995 (+ 106us) negotiation.cc:107] Waiting for socket to connect
0810 04:44:54.361030 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.167.126:38209: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":92,"thread_start_us":59,"threads_started":1}
I20260810 04:44:54.481112 19588 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:44:54.480860 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.126:38209 (local address 127.22.167.67:32855)
0810 04:44:54.481015 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0810 04:44:54.481067 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.167.126:38209: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":80,"threads_started":1}
I20260810 04:44:54.757695 19590 raft_consensus.cc:482] T 5ddb3715fece4198b35167574f1fc53c P 7e7834fc8be34efcb685d3a20756d41b [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 04:44:54.757790 19590 raft_consensus.cc:504] T 5ddb3715fece4198b35167574f1fc53c P 7e7834fc8be34efcb685d3a20756d41b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "671e8749ce234fd9a175013a5198a496" member_type: VOTER last_known_addr { host: "127.22.167.67" port: 37313 } } peers { permanent_uuid: "7e7834fc8be34efcb685d3a20756d41b" member_type: VOTER last_known_addr { host: "127.22.167.65" port: 37211 } } peers { permanent_uuid: "6ee21ed53fbc49d284bbecab757bb9db" member_type: VOTER last_known_addr { host: "127.22.167.66" port: 45913 } }
I20260810 04:44:54.757958 19590 leader_election.cc:288] T 5ddb3715fece4198b35167574f1fc53c P 7e7834fc8be34efcb685d3a20756d41b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 671e8749ce234fd9a175013a5198a496 (127.22.167.67:37313), 6ee21ed53fbc49d284bbecab757bb9db (127.22.167.66:45913)
W20260810 04:44:54.758430 18866 leader_election.cc:341] T 5ddb3715fece4198b35167574f1fc53c P 7e7834fc8be34efcb685d3a20756d41b [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 6ee21ed53fbc49d284bbecab757bb9db (127.22.167.66:45913): Not found: Tablet not found: 5ddb3715fece4198b35167574f1fc53c
W20260810 04:44:54.758543 18864 leader_election.cc:341] T 5ddb3715fece4198b35167574f1fc53c P 7e7834fc8be34efcb685d3a20756d41b [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 671e8749ce234fd9a175013a5198a496 (127.22.167.67:37313): Not found: Tablet not found: 5ddb3715fece4198b35167574f1fc53c
I20260810 04:44:54.758594 18864 leader_election.cc:302] T 5ddb3715fece4198b35167574f1fc53c P 7e7834fc8be34efcb685d3a20756d41b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e7834fc8be34efcb685d3a20756d41b; no voters: 671e8749ce234fd9a175013a5198a496, 6ee21ed53fbc49d284bbecab757bb9db
I20260810 04:44:54.758673 19590 raft_consensus.cc:2726] T 5ddb3715fece4198b35167574f1fc53c P 7e7834fc8be34efcb685d3a20756d41b [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260810 04:44:55.284830 19592 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:44:55.284603 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.126:38209 (local address 127.22.167.65:59613)
0810 04:44:55.284733 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0810 04:44:55.284774 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.167.126:38209: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":65,"threads_started":1}
I20260810 04:44:55.361637 19593 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:44:55.361432 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.126:38209 (local address 127.22.167.66:40477)
0810 04:44:55.361569 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0810 04:44:55.361597 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.167.126:38209: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":97,"threads_started":1}
I20260810 04:44:55.481706 19594 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:44:55.481498 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.126:38209 (local address 127.22.167.67:54807)
0810 04:44:55.481630 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0810 04:44:55.481664 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.167.126:38209: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":61,"threads_started":1}
I20260810 04:44:56.285472 19597 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:44:56.285259 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.126:38209 (local address 127.22.167.65:55171)
0810 04:44:56.285387 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0810 04:44:56.285420 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.167.126:38209: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":64,"threads_started":1}
I20260810 04:44:56.362269 19598 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:44:56.362058 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.126:38209 (local address 127.22.167.66:49369)
0810 04:44:56.362190 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0810 04:44:56.362224 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.167.126:38209: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":89,"threads_started":1}
I20260810 04:44:56.482275 19602 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:44:56.482056 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.126:38209 (local address 127.22.167.67:52227)
0810 04:44:56.482186 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0810 04:44:56.482233 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.167.126:38209: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":81,"threads_started":1}
I20260810 04:44:57.286099 19604 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:44:57.285880 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.126:38209 (local address 127.22.167.65:49153)
0810 04:44:57.286018 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0810 04:44:57.286049 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.167.126:38209: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":78,"threads_started":1}
I20260810 04:44:57.362887 19605 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:44:57.362682 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.126:38209 (local address 127.22.167.66:56321)
0810 04:44:57.362800 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0810 04:44:57.362841 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.167.126:38209: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":104,"thread_start_us":61,"threads_started":1}
I20260810 04:44:57.482875 19607 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:44:57.482663 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.126:38209 (local address 127.22.167.67:33647)
0810 04:44:57.482794 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0810 04:44:57.482832 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.167.126:38209: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":78,"threads_started":1}
I20260810 04:44:58.286775 19608 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:44:58.286558 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.126:38209 (local address 127.22.167.65:37651)
0810 04:44:58.286692 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0810 04:44:58.286726 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.167.126:38209: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":80,"threads_started":1}
I20260810 04:44:58.363458 19611 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:44:58.363266 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.126:38209 (local address 127.22.167.66:44137)
0810 04:44:58.363379 (+ 113us) negotiation.cc:107] Waiting for socket to connect
0810 04:44:58.363414 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.167.126:38209: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":97,"thread_start_us":57,"threads_started":1}
I20260810 04:44:58.483453 19613 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:44:58.483262 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.126:38209 (local address 127.22.167.67:56937)
0810 04:44:58.483373 (+ 111us) negotiation.cc:107] Waiting for socket to connect
0810 04:44:58.483406 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.167.126:38209: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":95,"thread_start_us":57,"threads_started":1}
I20260810 04:44:59.287403 19614 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:44:59.287191 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.126:38209 (local address 127.22.167.65:34437)
0810 04:44:59.287314 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0810 04:44:59.287352 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.167.126:38209: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":60,"threads_started":1}
I20260810 04:44:59.364101 19616 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:44:59.363897 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.126:38209 (local address 127.22.167.66:55163)
0810 04:44:59.364022 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0810 04:44:59.364054 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.167.126:38209: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":64,"threads_started":1}
I20260810 04:44:59.478359 23197 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskemv_AM/build/release/bin/kudu with pid 18779
I20260810 04:44:59.486004 19617 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:44:59.485585 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.126:38209 (local address 127.22.167.67:38085)
0810 04:44:59.485901 (+ 316us) negotiation.cc:107] Waiting for socket to connect
0810 04:44:59.485946 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.167.126:38209: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":292,"thread_start_us":237,"threads_started":1}
I20260810 04:44:59.490422 23197 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskemv_AM/build/release/bin/kudu with pid 19059
I20260810 04:44:59.506448 23197 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskemv_AM/build/release/bin/kudu with pid 19299
2026-08-10T04:44:59Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedTSK (58460 ms)
[----------] 30 tests from SecurityITest (236130 ms total)
[----------] 6 tests from EncryptionPolicyTest
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/0
Loading random data
Initializing database '/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786336863388938-23197-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 04:44:59 dist-test-slave-bqcl krb5kdc[19623](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 04:44:59 dist-test-slave-bqcl krb5kdc[19623](info): set up 2 sockets
Aug 10 04:44:59 dist-test-slave-bqcl krb5kdc[19623](info): commencing operation
krb5kdc: starting...
W20260810 04:45:00.676608 23197 mini_kdc.cc:121] Time spent starting KDC: real 1.145s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786336863388938-23197-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786336863388938-23197-0/krb5kdc/test-user.keytab.
Aug 10 04:45:00 dist-test-slave-bqcl krb5kdc[19623](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786337100, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T04:45:00Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T04:45:00Z Disabled control of system clock
WARNING: no policy specified for kudu/127.22.167.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.167.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.167.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786336863388938-23197-0/krb5kdc/kudu_127.22.167.126.keytab.
Entry for principal kudu/127.22.167.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786336863388938-23197-0/krb5kdc/kudu_127.22.167.126.keytab.
I20260810 04:45:00.804111 23197 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskemv_AM/build/release/bin/kudu
/tmp/dist-test-taskemv_AM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786336863388938-23197-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786336863388938-23197-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786336863388938-23197-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786336863388938-23197-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.167.126:38117
--webserver_interface=127.22.167.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786336863388938-23197-0/krb5kdc/kudu_127.22.167.126.keytab
--principal=kudu/127.22.167.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.167.84:35271
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.167.126:38117
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786336863388938-23197-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786336863388938-23197-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786336863388938-23197-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 04:45:00.884466 19645 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 04:45:00.884655 19645 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 04:45:00.884678 19645 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 04:45:00.885751 19645 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 04:45:00.885787 19645 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 04:45:00.885797 19645 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 04:45:00.885808 19645 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 04:45:00.885818 19645 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 04:45:00.887276 19645 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.167.84:35271
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786336863388938-23197-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786336863388938-23197-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.22.167.126:38117
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786336863388938-23197-0/krb5kdc/kudu_127.22.167.126.keytab
--principal=kudu/127.22.167.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.167.126:38117
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786336863388938-23197-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.167.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19645
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786336863388938-23197-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 03:54:01 UTC on ccycloud.quasar-icijmt.root.comops.site
build id 4577009
I20260810 04:45:00.887496 19645 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 04:45:00.887722 19645 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 04:45:00.892596 19658 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 04:45:00.892673 19645 server_base.cc:1034] running on GCE node
W20260810 04:45:00.892604 19657 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 04:45:00.892617 19662 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 04:45:00.893056 19645 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 04:45:00.893307 19645 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 04:45:00.894439 19645 hybrid_clock.cc:648] HybridClock initialized: now 1786337100894424 us; error 29 us; skew 500 ppm
Aug 10 04:45:00 dist-test-slave-bqcl krb5kdc[19623](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786337100, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.167.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 04:45:00.896919 19645 init.cc:377] Logged in from keytab as kudu/127.22.167.126@KRBTEST.COM (short username kudu)
I20260810 04:45:00.897364 19645 webserver.cc:460] Webserver started at http://127.22.167.126:40093/ using document root <none> and password file <none>
I20260810 04:45:00.897573 19645 fs_manager.cc:359] Metadata directory not provided
I20260810 04:45:00.897621 19645 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 04:45:00.897748 19645 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 04:45:00.898860 19645 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786336863388938-23197-0/minicluster-data/master-0/data/instance:
uuid: "233d162ff73d4c03b8d145ded2b1b68a"
format_stamp: "Formatted at 2026-08-10 04:45:00 on dist-test-slave-bqcl"
server_key: "04946d7afc8e33ac51bb24e04386c02b"
server_key_iv: "44bf445bb18d696fb6f4cfd20d484582"
server_key_version: "encryptionkey@0"
I20260810 04:45:00.899189 19645 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786336863388938-23197-0/minicluster-data/master-0/wal/instance:
uuid: "233d162ff73d4c03b8d145ded2b1b68a"
format_stamp: "Formatted at 2026-08-10 04:45:00 on dist-test-slave-bqcl"
server_key: "04946d7afc8e33ac51bb24e04386c02b"
server_key_iv: "44bf445bb18d696fb6f4cfd20d484582"
server_key_version: "encryptionkey@0"
I20260810 04:45:00.900386 19645 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260810 04:45:00.901084 19676 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 04:45:00.901260 19645 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260810 04:45:00.901346 19645 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786336863388938-23197-0/minicluster-data/master-0/data,/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786336863388938-23197-0/minicluster-data/master-0/wal
uuid: "233d162ff73d4c03b8d145ded2b1b68a"
format_stamp: "Formatted at 2026-08-10 04:45:00 on dist-test-slave-bqcl"
server_key: "04946d7afc8e33ac51bb24e04386c02b"
server_key_iv: "44bf445bb18d696fb6f4cfd20d484582"
server_key_version: "encryptionkey@0"
I20260810 04:45:00.901403 19645 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786336863388938-23197-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786336863388938-23197-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786336863388938-23197-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 04:45:00.919042 19645 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 04:45:00.920776 19645 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 04:45:00.920905 19645 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 04:45:00.924677 19645 rpc_server.cc:307] RPC server started. Bound to: 127.22.167.126:38117
I20260810 04:45:00.924710 19759 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.167.126:38117 every 8 connection(s)
I20260810 04:45:00.925024 19645 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786336863388938-23197-0/minicluster-data/master-0/data/info.pb
I20260810 04:45:00.925853 19764 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 04:45:00.928054 19764 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 233d162ff73d4c03b8d145ded2b1b68a: Bootstrap starting.
I20260810 04:45:00.928421 23197 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskemv_AM/build/release/bin/kudu as pid 19645
I20260810 04:45:00.928525 23197 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786336863388938-23197-0/minicluster-data/master-0/wal/instance
I20260810 04:45:00.928642 19764 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 233d162ff73d4c03b8d145ded2b1b68a: Neither blocks nor log segments found. Creating new log.
I20260810 04:45:00.928690 23197 external_mini_cluster.cc:1428] Setting key 2ebe4750d6a419867b910eca69acea01
I20260810 04:45:00.928941 19764 log.cc:824] T 00000000000000000000000000000000 P 233d162ff73d4c03b8d145ded2b1b68a: Log is configured to *not* fsync() on all Append() calls
I20260810 04:45:00.929709 19764 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 233d162ff73d4c03b8d145ded2b1b68a: No bootstrap required, opened a new log
I20260810 04:45:00.931951 19764 raft_consensus.cc:348] T 00000000000000000000000000000000 P 233d162ff73d4c03b8d145ded2b1b68a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "233d162ff73d4c03b8d145ded2b1b68a" member_type: VOTER last_known_addr { host: "127.22.167.126" port: 38117 } }
I20260810 04:45:00.932117 19764 raft_consensus.cc:374] T 00000000000000000000000000000000 P 233d162ff73d4c03b8d145ded2b1b68a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 04:45:00.932154 19764 raft_consensus.cc:729] T 00000000000000000000000000000000 P 233d162ff73d4c03b8d145ded2b1b68a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 233d162ff73d4c03b8d145ded2b1b68a, State: Initialized, Role: FOLLOWER
I20260810 04:45:00.932282 19764 consensus_queue.cc:260] T 00000000000000000000000000000000 P 233d162ff73d4c03b8d145ded2b1b68a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "233d162ff73d4c03b8d145ded2b1b68a" member_type: VOTER last_known_addr { host: "127.22.167.126" port: 38117 } }
I20260810 04:45:00.932364 19764 raft_consensus.cc:388] T 00000000000000000000000000000000 P 233d162ff73d4c03b8d145ded2b1b68a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 04:45:00.932399 19764 raft_consensus.cc:482] T 00000000000000000000000000000000 P 233d162ff73d4c03b8d145ded2b1b68a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 04:45:00.932446 19764 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 233d162ff73d4c03b8d145ded2b1b68a [term 0 FOLLOWER]: Advancing to term 1
I20260810 04:45:00.933096 19764 raft_consensus.cc:504] T 00000000000000000000000000000000 P 233d162ff73d4c03b8d145ded2b1b68a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "233d162ff73d4c03b8d145ded2b1b68a" member_type: VOTER last_known_addr { host: "127.22.167.126" port: 38117 } }
I20260810 04:45:00.933205 19764 leader_election.cc:302] T 00000000000000000000000000000000 P 233d162ff73d4c03b8d145ded2b1b68a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 233d162ff73d4c03b8d145ded2b1b68a; no voters:
I20260810 04:45:00.933357 19764 leader_election.cc:288] T 00000000000000000000000000000000 P 233d162ff73d4c03b8d145ded2b1b68a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 04:45:00.933380 19769 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 233d162ff73d4c03b8d145ded2b1b68a [term 1 FOLLOWER]: Leader election won for term 1
I20260810 04:45:00.933544 19769 raft_consensus.cc:686] T 00000000000000000000000000000000 P 233d162ff73d4c03b8d145ded2b1b68a [term 1 LEADER]: Becoming Leader. State: Replica: 233d162ff73d4c03b8d145ded2b1b68a, State: Running, Role: LEADER
I20260810 04:45:00.933621 19764 sys_catalog.cc:564] T 00000000000000000000000000000000 P 233d162ff73d4c03b8d145ded2b1b68a [sys.catalog]: configured and running, proceeding with master startup.
I20260810 04:45:00.933681 19769 consensus_queue.cc:237] T 00000000000000000000000000000000 P 233d162ff73d4c03b8d145ded2b1b68a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "233d162ff73d4c03b8d145ded2b1b68a" member_type: VOTER last_known_addr { host: "127.22.167.126" port: 38117 } }
I20260810 04:45:00.934039 19769 sys_catalog.cc:455] T 00000000000000000000000000000000 P 233d162ff73d4c03b8d145ded2b1b68a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 233d162ff73d4c03b8d145ded2b1b68a. Latest consensus state: current_term: 1 leader_uuid: "233d162ff73d4c03b8d145ded2b1b68a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "233d162ff73d4c03b8d145ded2b1b68a" member_type: VOTER last_known_addr { host: "127.22.167.126" port: 38117 } } }
I20260810 04:45:00.934237 19772 sys_catalog.cc:455] T 00000000000000000000000000000000 P 233d162ff73d4c03b8d145ded2b1b68a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "233d162ff73d4c03b8d145ded2b1b68a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "233d162ff73d4c03b8d145ded2b1b68a" member_type: VOTER last_known_addr { host: "127.22.167.126" port: 38117 } } }
I20260810 04:45:00.934311 19769 sys_catalog.cc:458] T 00000000000000000000000000000000 P 233d162ff73d4c03b8d145ded2b1b68a [sys.catalog]: This master's current role is: LEADER
I20260810 04:45:00.934377 19772 sys_catalog.cc:458] T 00000000000000000000000000000000 P 233d162ff73d4c03b8d145ded2b1b68a [sys.catalog]: This master's current role is: LEADER
I20260810 04:45:00.934633 19787 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 04:45:00.935206 19787 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Aug 10 04:45:00 dist-test-slave-bqcl krb5kdc[19623](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786337100, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.167.126@KRBTEST.COM
I20260810 04:45:00.936661 19787 catalog_manager.cc:1347] Generated new cluster ID: 859f7635383b486e9f6da5a6930386b5
I20260810 04:45:00.936712 19787 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 04:45:00.940208 19768 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:45:00.929147 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54032 (local address 127.22.167.126:38117)
0810 04:45:00.929321 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0810 04:45:00.929331 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 04:45:00.929362 (+ 31us) server_negotiation.cc:407] Connection header received
0810 04:45:00.929903 (+ 541us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:45:00.929913 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:45:00.929973 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:45:00.930148 (+ 175us) server_negotiation.cc:226] Negotiated authn=SASL
0810 04:45:00.930762 (+ 614us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:45:00.932126 (+ 1364us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:45:00.933182 (+ 1056us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:45:00.933811 (+ 629us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:45:00.933959 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:45:00.936861 (+ 2902us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 04:45:00.936865 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 04:45:00.936865 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 04:45:00.936876 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0810 04:45:00.938687 (+ 1811us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 04:45:00.939078 (+ 391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 04:45:00.939080 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 04:45:00.939081 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 04:45:00.939182 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 04:45:00.939374 (+ 192us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 04:45:00.939375 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 04:45:00.939375 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 04:45:00.939633 (+ 258us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 04:45:00.939778 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0810 04:45:00.939884 (+ 106us) server_negotiation.cc:299] Negotiation successful
0810 04:45:00.940003 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":125,"thread_start_us":82,"threads_started":1}
I20260810 04:45:00.944581 19787 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 04:45:00.945287 19787 catalog_manager.cc:1504] Loading token signing keys...
I20260810 04:45:00.953089 19787 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 233d162ff73d4c03b8d145ded2b1b68a: Generated new TSK 0
I20260810 04:45:00.953258 19787 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.22.167.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.167.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.167.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786336863388938-23197-0/krb5kdc/kudu_127.22.167.65.keytab.
Entry for principal kudu/127.22.167.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786336863388938-23197-0/krb5kdc/kudu_127.22.167.65.keytab.
I20260810 04:45:00.982738 23197 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskemv_AM/build/release/bin/kudu
/tmp/dist-test-taskemv_AM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786336863388938-23197-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786336863388938-23197-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786336863388938-23197-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786336863388938-23197-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.167.65:0
--local_ip_for_outbound_sockets=127.22.167.65
--webserver_interface=127.22.167.65
--webserver_port=0
--tserver_master_addrs=127.22.167.126:38117
--keytab_file=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786336863388938-23197-0/krb5kdc/kudu_127.22.167.65.keytab
--principal=kudu/127.22.167.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.167.84:35271
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786336863388938-23197-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786336863388938-23197-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786336863388938-23197-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 04:45:01.063611 19809 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 04:45:01.063817 19809 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 04:45:01.063838 19809 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 04:45:01.064924 19809 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 04:45:01.064965 19809 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 04:45:01.064988 19809 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.167.65
I20260810 04:45:01.066401 19809 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.167.84:35271
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786336863388938-23197-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786336863388938-23197-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786336863388938-23197-0/krb5kdc/kudu_127.22.167.65.keytab
--principal=kudu/127.22.167.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.167.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786336863388938-23197-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.22.167.65
--webserver_port=0
--tserver_master_addrs=127.22.167.126:38117
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19809
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.167.65
--log_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786336863388938-23197-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 03:54:01 UTC on ccycloud.quasar-icijmt.root.comops.site
build id 4577009
I20260810 04:45:01.066604 19809 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 04:45:01.066823 19809 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 04:45:01.071655 19817 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 04:45:01.071666 19815 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 04:45:01.071733 19809 server_base.cc:1034] running on GCE node
W20260810 04:45:01.071641 19814 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 04:45:01.072001 19809 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 04:45:01.072242 19809 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 04:45:01.073379 19809 hybrid_clock.cc:648] HybridClock initialized: now 1786337101073367 us; error 28 us; skew 500 ppm
Aug 10 04:45:01 dist-test-slave-bqcl krb5kdc[19623](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786337101, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.167.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 04:45:01.075786 19809 init.cc:377] Logged in from keytab as kudu/127.22.167.65@KRBTEST.COM (short username kudu)
I20260810 04:45:01.076192 19809 webserver.cc:460] Webserver started at http://127.22.167.65:34673/ using document root <none> and password file <none>
I20260810 04:45:01.076383 19809 fs_manager.cc:359] Metadata directory not provided
I20260810 04:45:01.076431 19809 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 04:45:01.076555 19809 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 04:45:01.077561 19809 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786336863388938-23197-0/minicluster-data/ts-0/data/instance:
uuid: "07f4cce57ae44844aa6a0af2105c6dad"
format_stamp: "Formatted at 2026-08-10 04:45:01 on dist-test-slave-bqcl"
server_key: "09ad068ab3ad1d74f00a137406e3bcdd"
server_key_iv: "8e77633cad5d2d58707bc2a01d5e05aa"
server_key_version: "encryptionkey@0"
I20260810 04:45:01.077891 19809 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786336863388938-23197-0/minicluster-data/ts-0/wal/instance:
uuid: "07f4cce57ae44844aa6a0af2105c6dad"
format_stamp: "Formatted at 2026-08-10 04:45:01 on dist-test-slave-bqcl"
server_key: "09ad068ab3ad1d74f00a137406e3bcdd"
server_key_iv: "8e77633cad5d2d58707bc2a01d5e05aa"
server_key_version: "encryptionkey@0"
I20260810 04:45:01.079159 19809 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260810 04:45:01.079866 19832 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 04:45:01.080020 19809 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260810 04:45:01.080103 19809 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786336863388938-23197-0/minicluster-data/ts-0/data,/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786336863388938-23197-0/minicluster-data/ts-0/wal
uuid: "07f4cce57ae44844aa6a0af2105c6dad"
format_stamp: "Formatted at 2026-08-10 04:45:01 on dist-test-slave-bqcl"
server_key: "09ad068ab3ad1d74f00a137406e3bcdd"
server_key_iv: "8e77633cad5d2d58707bc2a01d5e05aa"
server_key_version: "encryptionkey@0"
I20260810 04:45:01.080178 19809 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786336863388938-23197-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786336863388938-23197-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786336863388938-23197-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 04:45:01.098574 19809 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 04:45:01.100519 19809 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 04:45:01.100633 19809 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 04:45:01.100834 19809 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 04:45:01.101146 19809 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 04:45:01.101182 19809 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 04:45:01.101213 19809 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 04:45:01.101238 19809 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 04:45:01.106824 19809 rpc_server.cc:307] RPC server started. Bound to: 127.22.167.65:42983
I20260810 04:45:01.106895 20034 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.167.65:42983 every 8 connection(s)
I20260810 04:45:01.107184 19809 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786336863388938-23197-0/minicluster-data/ts-0/data/info.pb
I20260810 04:45:01.107550 23197 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskemv_AM/build/release/bin/kudu as pid 19809
I20260810 04:45:01.107651 23197 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786336863388938-23197-0/minicluster-data/ts-0/wal/instance
I20260810 04:45:01.107856 23197 external_mini_cluster.cc:1428] Setting key 23872ca09987375eda20395e2cc996f7
Aug 10 04:45:01 dist-test-slave-bqcl krb5kdc[19623](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786337101, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.167.65@KRBTEST.COM for kudu/127.22.167.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.22.167.66@KRBTEST.COM; defaulting to no policy
I20260810 04:45:01.117764 19768 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:45:01.108213 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.167.65:43941 (local address 127.22.167.126:38117)
0810 04:45:01.108258 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0810 04:45:01.108260 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 04:45:01.108327 (+ 67us) server_negotiation.cc:407] Connection header received
0810 04:45:01.108844 (+ 517us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:45:01.108845 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:45:01.108857 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:45:01.108915 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0810 04:45:01.109907 (+ 992us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:45:01.110554 (+ 647us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:45:01.111929 (+ 1375us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:45:01.112207 (+ 278us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:45:01.112276 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:45:01.114549 (+ 2273us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 04:45:01.114551 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 04:45:01.114551 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 04:45:01.114560 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0810 04:45:01.116210 (+ 1650us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 04:45:01.116601 (+ 391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 04:45:01.116603 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 04:45:01.116603 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 04:45:01.116664 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 04:45:01.116880 (+ 216us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 04:45:01.116881 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 04:45:01.116881 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 04:45:01.117024 (+ 143us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 04:45:01.117092 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0810 04:45:01.117603 (+ 511us) server_negotiation.cc:299] Negotiation successful
0810 04:45:01.117693 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260810 04:45:01.117883 20037 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:45:01.107906 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.126:38117 (local address 127.22.167.65:43941)
0810 04:45:01.108208 (+ 302us) negotiation.cc:107] Waiting for socket to connect
0810 04:45:01.108258 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0810 04:45:01.108752 (+ 494us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 04:45:01.108973 (+ 221us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 04:45:01.108976 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 04:45:01.109603 (+ 627us) client_negotiation.cc:189] Negotiated authn=SASL
0810 04:45:01.109810 (+ 207us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:45:01.109817 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:45:01.110650 (+ 833us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:45:01.110652 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:45:01.111837 (+ 1185us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:45:01.111841 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:45:01.112285 (+ 444us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:45:01.112286 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:45:01.112388 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:45:01.112938 (+ 550us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 04:45:01.112948 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0810 04:45:01.114437 (+ 1489us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 04:45:01.116286 (+ 1849us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 04:45:01.116294 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 04:45:01.116296 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 04:45:01.116526 (+ 230us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 04:45:01.116730 (+ 204us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 04:45:01.116731 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 04:45:01.116731 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 04:45:01.116824 (+ 93us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 04:45:01.117199 (+ 375us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 04:45:01.117208 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 04:45:01.117377 (+ 169us) client_negotiation.cc:769] Sending connection context
0810 04:45:01.117500 (+ 123us) client_negotiation.cc:240] Negotiation successful
0810 04:45:01.117688 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":249,"thread_start_us":90,"threads_started":1}
I20260810 04:45:01.118530 20035 heartbeater.cc:347] Connected to a master server at 127.22.167.126:38117
I20260810 04:45:01.118651 20035 heartbeater.cc:464] Registering TS with master...
I20260810 04:45:01.118829 20035 heartbeater.cc:511] Master 127.22.167.126:38117 requested a full tablet report, sending...
I20260810 04:45:01.119401 19703 ts_manager.cc:194] Registered new tserver with Master: 07f4cce57ae44844aa6a0af2105c6dad (127.22.167.65:42983)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.167.66@KRBTEST.COM" created.
I20260810 04:45:01.120168 19703 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.167.65@KRBTEST.COM'} at 127.22.167.65:43941
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.167.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786336863388938-23197-0/krb5kdc/kudu_127.22.167.66.keytab.
Entry for principal kudu/127.22.167.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786336863388938-23197-0/krb5kdc/kudu_127.22.167.66.keytab.
I20260810 04:45:01.132326 23197 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskemv_AM/build/release/bin/kudu
/tmp/dist-test-taskemv_AM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786336863388938-23197-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786336863388938-23197-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786336863388938-23197-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786336863388938-23197-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.167.66:0
--local_ip_for_outbound_sockets=127.22.167.66
--webserver_interface=127.22.167.66
--webserver_port=0
--tserver_master_addrs=127.22.167.126:38117
--keytab_file=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786336863388938-23197-0/krb5kdc/kudu_127.22.167.66.keytab
--principal=kudu/127.22.167.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.167.84:35271
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786336863388938-23197-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786336863388938-23197-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786336863388938-23197-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 04:45:01.213137 20041 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 04:45:01.213332 20041 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 04:45:01.213353 20041 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 04:45:01.214434 20041 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 04:45:01.214466 20041 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 04:45:01.214490 20041 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.167.66
I20260810 04:45:01.215843 20041 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.167.84:35271
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786336863388938-23197-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786336863388938-23197-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786336863388938-23197-0/krb5kdc/kudu_127.22.167.66.keytab
--principal=kudu/127.22.167.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.167.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786336863388938-23197-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.22.167.66
--webserver_port=0
--tserver_master_addrs=127.22.167.126:38117
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20041
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.167.66
--log_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786336863388938-23197-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 03:54:01 UTC on ccycloud.quasar-icijmt.root.comops.site
build id 4577009
I20260810 04:45:01.216015 20041 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 04:45:01.216207 20041 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 04:45:01.221316 20053 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 04:45:01.221323 20057 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 04:45:01.221355 20041 server_base.cc:1034] running on GCE node
W20260810 04:45:01.221351 20054 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 04:45:01.221755 20041 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 04:45:01.221983 20041 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 04:45:01.223115 20041 hybrid_clock.cc:648] HybridClock initialized: now 1786337101223089 us; error 41 us; skew 500 ppm
Aug 10 04:45:01 dist-test-slave-bqcl krb5kdc[19623](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786337101, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.167.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 04:45:01.225436 20041 init.cc:377] Logged in from keytab as kudu/127.22.167.66@KRBTEST.COM (short username kudu)
I20260810 04:45:01.225858 20041 webserver.cc:460] Webserver started at http://127.22.167.66:36815/ using document root <none> and password file <none>
I20260810 04:45:01.226054 20041 fs_manager.cc:359] Metadata directory not provided
I20260810 04:45:01.226100 20041 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 04:45:01.226248 20041 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 04:45:01.227227 20041 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786336863388938-23197-0/minicluster-data/ts-1/data/instance:
uuid: "f5f8a930e59447c7ae7e2edf2001ba65"
format_stamp: "Formatted at 2026-08-10 04:45:01 on dist-test-slave-bqcl"
server_key: "d65e47430003499654d56e964d79d4c9"
server_key_iv: "9658123e1801b78d4f0da8c3d7d65c3a"
server_key_version: "encryptionkey@0"
I20260810 04:45:01.227557 20041 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786336863388938-23197-0/minicluster-data/ts-1/wal/instance:
uuid: "f5f8a930e59447c7ae7e2edf2001ba65"
format_stamp: "Formatted at 2026-08-10 04:45:01 on dist-test-slave-bqcl"
server_key: "d65e47430003499654d56e964d79d4c9"
server_key_iv: "9658123e1801b78d4f0da8c3d7d65c3a"
server_key_version: "encryptionkey@0"
I20260810 04:45:01.228780 20041 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260810 04:45:01.229408 20071 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 04:45:01.229561 20041 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260810 04:45:01.229650 20041 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786336863388938-23197-0/minicluster-data/ts-1/data,/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786336863388938-23197-0/minicluster-data/ts-1/wal
uuid: "f5f8a930e59447c7ae7e2edf2001ba65"
format_stamp: "Formatted at 2026-08-10 04:45:01 on dist-test-slave-bqcl"
server_key: "d65e47430003499654d56e964d79d4c9"
server_key_iv: "9658123e1801b78d4f0da8c3d7d65c3a"
server_key_version: "encryptionkey@0"
I20260810 04:45:01.229713 20041 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786336863388938-23197-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786336863388938-23197-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786336863388938-23197-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 04:45:01.244823 20041 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 04:45:01.246708 20041 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 04:45:01.246850 20041 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 04:45:01.247069 20041 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 04:45:01.247373 20041 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 04:45:01.247407 20041 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 04:45:01.247437 20041 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 04:45:01.247454 20041 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 04:45:01.252967 20041 rpc_server.cc:307] RPC server started. Bound to: 127.22.167.66:43349
I20260810 04:45:01.253027 20272 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.167.66:43349 every 8 connection(s)
I20260810 04:45:01.253319 20041 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786336863388938-23197-0/minicluster-data/ts-1/data/info.pb
I20260810 04:45:01.257575 23197 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskemv_AM/build/release/bin/kudu as pid 20041
I20260810 04:45:01.257702 23197 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786336863388938-23197-0/minicluster-data/ts-1/wal/instance
I20260810 04:45:01.257895 23197 external_mini_cluster.cc:1428] Setting key fc746d692a2963bc7eff44bc6753fee3
Aug 10 04:45:01 dist-test-slave-bqcl krb5kdc[19623](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786337101, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.167.66@KRBTEST.COM for kudu/127.22.167.126@KRBTEST.COM
I20260810 04:45:01.262410 19768 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:45:01.254006 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.167.66:55249 (local address 127.22.167.126:38117)
0810 04:45:01.254069 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0810 04:45:01.254071 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 04:45:01.254267 (+ 196us) server_negotiation.cc:407] Connection header received
0810 04:45:01.254754 (+ 487us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:45:01.254756 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:45:01.254766 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:45:01.254806 (+ 40us) server_negotiation.cc:226] Negotiated authn=SASL
0810 04:45:01.255706 (+ 900us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:45:01.256196 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:45:01.257147 (+ 951us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:45:01.257322 (+ 175us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:45:01.257364 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:45:01.259785 (+ 2421us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 04:45:01.259787 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 04:45:01.259787 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 04:45:01.259794 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 04:45:01.261030 (+ 1236us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 04:45:01.261482 (+ 452us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 04:45:01.261482 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 04:45:01.261482 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 04:45:01.261532 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 04:45:01.261775 (+ 243us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 04:45:01.261776 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 04:45:01.261777 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 04:45:01.261887 (+ 110us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 04:45:01.261945 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0810 04:45:01.262235 (+ 290us) server_negotiation.cc:299] Negotiation successful
0810 04:45:01.262321 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260810 04:45:01.262584 20278 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:45:01.253975 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.126:38117 (local address 127.22.167.66:55249)
0810 04:45:01.254146 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0810 04:45:01.254191 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0810 04:45:01.254662 (+ 471us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 04:45:01.254871 (+ 209us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 04:45:01.254875 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 04:45:01.255436 (+ 561us) client_negotiation.cc:189] Negotiated authn=SASL
0810 04:45:01.255633 (+ 197us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:45:01.255638 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:45:01.256261 (+ 623us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:45:01.256262 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:45:01.257086 (+ 824us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:45:01.257088 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:45:01.257388 (+ 300us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:45:01.257389 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:45:01.257552 (+ 163us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:45:01.258078 (+ 526us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 04:45:01.258089 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0810 04:45:01.259699 (+ 1610us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 04:45:01.261105 (+ 1406us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 04:45:01.261112 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 04:45:01.261114 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 04:45:01.261409 (+ 295us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 04:45:01.261602 (+ 193us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 04:45:01.261603 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 04:45:01.261604 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 04:45:01.261713 (+ 109us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 04:45:01.261944 (+ 231us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 04:45:01.261951 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 04:45:01.262086 (+ 135us) client_negotiation.cc:769] Sending connection context
0810 04:45:01.262203 (+ 117us) client_negotiation.cc:240] Negotiation successful
0810 04:45:01.262363 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":81,"threads_started":1}
I20260810 04:45:01.262996 20275 heartbeater.cc:347] Connected to a master server at 127.22.167.126:38117
I20260810 04:45:01.263092 20275 heartbeater.cc:464] Registering TS with master...
I20260810 04:45:01.263268 20275 heartbeater.cc:511] Master 127.22.167.126:38117 requested a full tablet report, sending...
I20260810 04:45:01.263614 19703 ts_manager.cc:194] Registered new tserver with Master: f5f8a930e59447c7ae7e2edf2001ba65 (127.22.167.66:43349)
I20260810 04:45:01.263974 19703 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.167.66@KRBTEST.COM'} at 127.22.167.66:55249
WARNING: no policy specified for kudu/127.22.167.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.167.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.167.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786336863388938-23197-0/krb5kdc/kudu_127.22.167.67.keytab.
Entry for principal kudu/127.22.167.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786336863388938-23197-0/krb5kdc/kudu_127.22.167.67.keytab.
I20260810 04:45:01.282635 23197 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskemv_AM/build/release/bin/kudu
/tmp/dist-test-taskemv_AM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786336863388938-23197-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786336863388938-23197-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786336863388938-23197-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786336863388938-23197-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.167.67:0
--local_ip_for_outbound_sockets=127.22.167.67
--webserver_interface=127.22.167.67
--webserver_port=0
--tserver_master_addrs=127.22.167.126:38117
--keytab_file=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786336863388938-23197-0/krb5kdc/kudu_127.22.167.67.keytab
--principal=kudu/127.22.167.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.167.84:35271
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786336863388938-23197-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786336863388938-23197-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786336863388938-23197-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 04:45:01.363593 20284 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 04:45:01.363780 20284 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 04:45:01.363801 20284 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 04:45:01.364881 20284 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 04:45:01.364921 20284 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 04:45:01.364950 20284 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.167.67
I20260810 04:45:01.366375 20284 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.167.84:35271
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786336863388938-23197-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786336863388938-23197-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786336863388938-23197-0/krb5kdc/kudu_127.22.167.67.keytab
--principal=kudu/127.22.167.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.167.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786336863388938-23197-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.22.167.67
--webserver_port=0
--tserver_master_addrs=127.22.167.126:38117
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20284
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.167.67
--log_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786336863388938-23197-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 03:54:01 UTC on ccycloud.quasar-icijmt.root.comops.site
build id 4577009
I20260810 04:45:01.366582 20284 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 04:45:01.366802 20284 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 04:45:01.371482 20300 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 04:45:01.371543 20284 server_base.cc:1034] running on GCE node
W20260810 04:45:01.371515 20295 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 04:45:01.371507 20294 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 04:45:01.371917 20284 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 04:45:01.372133 20284 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 04:45:01.373271 20284 hybrid_clock.cc:648] HybridClock initialized: now 1786337101373257 us; error 28 us; skew 500 ppm
Aug 10 04:45:01 dist-test-slave-bqcl krb5kdc[19623](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786337101, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.167.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 04:45:01.375664 20284 init.cc:377] Logged in from keytab as kudu/127.22.167.67@KRBTEST.COM (short username kudu)
I20260810 04:45:01.376057 20284 webserver.cc:460] Webserver started at http://127.22.167.67:44489/ using document root <none> and password file <none>
I20260810 04:45:01.376255 20284 fs_manager.cc:359] Metadata directory not provided
I20260810 04:45:01.376293 20284 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 04:45:01.376385 20284 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 04:45:01.377349 20284 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786336863388938-23197-0/minicluster-data/ts-2/data/instance:
uuid: "07b6a9892d4a46b79343dd317d4efdbc"
format_stamp: "Formatted at 2026-08-10 04:45:01 on dist-test-slave-bqcl"
server_key: "d1201a408c131e861069934709fd9b3e"
server_key_iv: "2dbcccf70d537d86583b936963a542af"
server_key_version: "encryptionkey@0"
I20260810 04:45:01.377645 20284 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786336863388938-23197-0/minicluster-data/ts-2/wal/instance:
uuid: "07b6a9892d4a46b79343dd317d4efdbc"
format_stamp: "Formatted at 2026-08-10 04:45:01 on dist-test-slave-bqcl"
server_key: "d1201a408c131e861069934709fd9b3e"
server_key_iv: "2dbcccf70d537d86583b936963a542af"
server_key_version: "encryptionkey@0"
I20260810 04:45:01.378937 20284 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260810 04:45:01.379594 20311 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 04:45:01.379748 20284 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260810 04:45:01.379822 20284 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786336863388938-23197-0/minicluster-data/ts-2/data,/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786336863388938-23197-0/minicluster-data/ts-2/wal
uuid: "07b6a9892d4a46b79343dd317d4efdbc"
format_stamp: "Formatted at 2026-08-10 04:45:01 on dist-test-slave-bqcl"
server_key: "d1201a408c131e861069934709fd9b3e"
server_key_iv: "2dbcccf70d537d86583b936963a542af"
server_key_version: "encryptionkey@0"
I20260810 04:45:01.379881 20284 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786336863388938-23197-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786336863388938-23197-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786336863388938-23197-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 04:45:01.395387 20284 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 04:45:01.397358 20284 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 04:45:01.397483 20284 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 04:45:01.397680 20284 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 04:45:01.397951 20284 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 04:45:01.397984 20284 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 04:45:01.398003 20284 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 04:45:01.398025 20284 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 04:45:01.403260 20284 rpc_server.cc:307] RPC server started. Bound to: 127.22.167.67:44303
I20260810 04:45:01.403323 20515 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.167.67:44303 every 8 connection(s)
I20260810 04:45:01.403594 20284 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786336863388938-23197-0/minicluster-data/ts-2/data/info.pb
I20260810 04:45:01.407373 23197 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskemv_AM/build/release/bin/kudu as pid 20284
I20260810 04:45:01.407481 23197 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786336863388938-23197-0/minicluster-data/ts-2/wal/instance
I20260810 04:45:01.407694 23197 external_mini_cluster.cc:1428] Setting key fb0a306aa63934ac3a43b96d23d7b114
Aug 10 04:45:01 dist-test-slave-bqcl krb5kdc[19623](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786337101, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.167.67@KRBTEST.COM for kudu/127.22.167.126@KRBTEST.COM
I20260810 04:45:01.412631 19768 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:45:01.404270 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.167.67:34489 (local address 127.22.167.126:38117)
0810 04:45:01.404328 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0810 04:45:01.404330 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 04:45:01.404509 (+ 179us) server_negotiation.cc:407] Connection header received
0810 04:45:01.405044 (+ 535us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:45:01.405045 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:45:01.405058 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:45:01.405110 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0810 04:45:01.406025 (+ 915us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:45:01.406572 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:45:01.407760 (+ 1188us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:45:01.408055 (+ 295us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:45:01.408130 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:45:01.410213 (+ 2083us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 04:45:01.410214 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 04:45:01.410214 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 04:45:01.410222 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0810 04:45:01.411333 (+ 1111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 04:45:01.411754 (+ 421us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 04:45:01.411755 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 04:45:01.411755 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 04:45:01.411824 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 04:45:01.412104 (+ 280us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 04:45:01.412104 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 04:45:01.412104 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 04:45:01.412211 (+ 107us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 04:45:01.412265 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0810 04:45:01.412464 (+ 199us) server_negotiation.cc:299] Negotiation successful
0810 04:45:01.412565 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260810 04:45:01.412748 20520 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:45:01.404233 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.126:38117 (local address 127.22.167.67:34489)
0810 04:45:01.404397 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0810 04:45:01.404431 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0810 04:45:01.404936 (+ 505us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 04:45:01.405195 (+ 259us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 04:45:01.405200 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 04:45:01.405745 (+ 545us) client_negotiation.cc:189] Negotiated authn=SASL
0810 04:45:01.405927 (+ 182us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:45:01.405932 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:45:01.406664 (+ 732us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:45:01.406665 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:45:01.407643 (+ 978us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:45:01.407647 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:45:01.408167 (+ 520us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:45:01.408169 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:45:01.408307 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:45:01.408795 (+ 488us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 04:45:01.408803 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0810 04:45:01.410086 (+ 1283us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 04:45:01.411421 (+ 1335us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 04:45:01.411428 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 04:45:01.411429 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 04:45:01.411681 (+ 252us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 04:45:01.411920 (+ 239us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 04:45:01.411921 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 04:45:01.411921 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 04:45:01.412043 (+ 122us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 04:45:01.412259 (+ 216us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 04:45:01.412261 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 04:45:01.412368 (+ 107us) client_negotiation.cc:769] Sending connection context
0810 04:45:01.412444 (+ 76us) client_negotiation.cc:240] Negotiation successful
0810 04:45:01.412576 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":87,"threads_started":1}
I20260810 04:45:01.413214 20516 heartbeater.cc:347] Connected to a master server at 127.22.167.126:38117
I20260810 04:45:01.413300 20516 heartbeater.cc:464] Registering TS with master...
I20260810 04:45:01.413467 20516 heartbeater.cc:511] Master 127.22.167.126:38117 requested a full tablet report, sending...
I20260810 04:45:01.413852 19703 ts_manager.cc:194] Registered new tserver with Master: 07b6a9892d4a46b79343dd317d4efdbc (127.22.167.67:44303)
I20260810 04:45:01.414243 19703 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.167.67@KRBTEST.COM'} at 127.22.167.67:34489
I20260810 04:45:01.419313 23197 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260810 04:45:01.425554 19768 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:45:01.420765 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54042 (local address 127.22.167.126:38117)
0810 04:45:01.420814 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0810 04:45:01.420817 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 04:45:01.420934 (+ 117us) server_negotiation.cc:407] Connection header received
0810 04:45:01.420973 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:45:01.420973 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:45:01.420982 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:45:01.421021 (+ 39us) server_negotiation.cc:226] Negotiated authn=SASL
0810 04:45:01.421490 (+ 469us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:45:01.421944 (+ 454us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:45:01.422623 (+ 679us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:45:01.422803 (+ 180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:45:01.422853 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:45:01.423760 (+ 907us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 04:45:01.423761 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 04:45:01.423761 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 04:45:01.423766 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0810 04:45:01.424752 (+ 986us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 04:45:01.425018 (+ 266us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 04:45:01.425019 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 04:45:01.425019 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 04:45:01.425061 (+ 42us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 04:45:01.425168 (+ 107us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 04:45:01.425169 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 04:45:01.425169 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 04:45:01.425297 (+ 128us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 04:45:01.425367 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0810 04:45:01.425438 (+ 71us) server_negotiation.cc:299] Negotiation successful
0810 04:45:01.425496 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260810 04:45:01.426843 19703 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54042:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 04:45:01.428340 19703 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 04:45:01.434569 20535 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:45:01.430471 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.66:43349 (local address 127.0.0.1:46490)
0810 04:45:01.430662 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0810 04:45:01.430671 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0810 04:45:01.430714 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 04:45:01.431140 (+ 426us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 04:45:01.431141 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 04:45:01.431143 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 04:45:01.431186 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:45:01.431188 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:45:01.432108 (+ 920us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:45:01.432109 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:45:01.433408 (+ 1299us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:45:01.433411 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:45:01.434333 (+ 922us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:45:01.434334 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:45:01.434393 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:45:01.434396 (+ 3us) client_negotiation.cc:769] Sending connection context
0810 04:45:01.434451 (+ 55us) client_negotiation.cc:240] Negotiation successful
0810 04:45:01.434489 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":180,"mutex_wait_us":31,"thread_start_us":60,"threads_started":1}
I20260810 04:45:01.434589 20540 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:45:01.430682 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46490 (local address 127.22.167.66:43349)
0810 04:45:01.430875 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0810 04:45:01.430880 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 04:45:01.430890 (+ 10us) server_negotiation.cc:407] Connection header received
0810 04:45:01.430924 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:45:01.430926 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:45:01.430938 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:45:01.431257 (+ 319us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 04:45:01.431320 (+ 63us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:45:01.432006 (+ 686us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:45:01.433486 (+ 1480us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:45:01.434271 (+ 785us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:45:01.434335 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:45:01.434393 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0810 04:45:01.434466 (+ 73us) server_negotiation.cc:299] Negotiation successful
0810 04:45:01.434508 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":162,"thread_start_us":111,"threads_started":1}
I20260810 04:45:01.434657 20527 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:45:01.430275 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.65:42983 (local address 127.0.0.1:48758)
0810 04:45:01.430516 (+ 241us) negotiation.cc:107] Waiting for socket to connect
0810 04:45:01.430557 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0810 04:45:01.430621 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 04:45:01.430947 (+ 326us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 04:45:01.430949 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 04:45:01.430954 (+ 5us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 04:45:01.431024 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:45:01.431025 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:45:01.432343 (+ 1318us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:45:01.432344 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:45:01.433311 (+ 967us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:45:01.433314 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:45:01.434424 (+ 1110us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:45:01.434425 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:45:01.434498 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:45:01.434502 (+ 4us) client_negotiation.cc:769] Sending connection context
0810 04:45:01.434553 (+ 51us) client_negotiation.cc:240] Negotiation successful
0810 04:45:01.434593 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":216,"mutex_wait_us":9,"thread_start_us":55,"threads_started":1}
I20260810 04:45:01.434804 20532 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:45:01.430362 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44010 (local address 127.22.167.67:44303)
0810 04:45:01.430520 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0810 04:45:01.430525 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 04:45:01.430743 (+ 218us) server_negotiation.cc:407] Connection header received
0810 04:45:01.430784 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:45:01.430786 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:45:01.430803 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:45:01.430873 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 04:45:01.431099 (+ 226us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:45:01.432183 (+ 1084us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:45:01.433521 (+ 1338us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:45:01.434092 (+ 571us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:45:01.434185 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:45:01.434245 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0810 04:45:01.434551 (+ 306us) server_negotiation.cc:299] Negotiation successful
0810 04:45:01.434590 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":125,"thread_start_us":76,"threads_started":1}
I20260810 04:45:01.434877 20533 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:45:01.430434 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48758 (local address 127.22.167.65:42983)
0810 04:45:01.430784 (+ 350us) server_negotiation.cc:206] Beginning negotiation
0810 04:45:01.430788 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 04:45:01.430797 (+ 9us) server_negotiation.cc:407] Connection header received
0810 04:45:01.430841 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:45:01.430843 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:45:01.430858 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:45:01.430933 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 04:45:01.431107 (+ 174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:45:01.432251 (+ 1144us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:45:01.433384 (+ 1133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:45:01.434169 (+ 785us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:45:01.434336 (+ 167us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:45:01.434396 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0810 04:45:01.434739 (+ 343us) server_negotiation.cc:299] Negotiation successful
0810 04:45:01.434785 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":312,"thread_start_us":88,"threads_started":1}
I20260810 04:45:01.434962 20529 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:45:01.430280 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.67:44303 (local address 127.0.0.1:44010)
0810 04:45:01.430602 (+ 322us) negotiation.cc:107] Waiting for socket to connect
0810 04:45:01.430612 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0810 04:45:01.430685 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 04:45:01.430946 (+ 261us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 04:45:01.430949 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 04:45:01.430951 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 04:45:01.431026 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:45:01.431027 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:45:01.432268 (+ 1241us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:45:01.432269 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:45:01.433429 (+ 1160us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:45:01.433430 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:45:01.434277 (+ 847us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:45:01.434279 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:45:01.434357 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:45:01.434371 (+ 14us) client_negotiation.cc:769] Sending connection context
0810 04:45:01.434439 (+ 68us) client_negotiation.cc:240] Negotiation successful
0810 04:45:01.434489 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":308,"mutex_wait_us":11,"thread_start_us":55,"threads_started":1}
I20260810 04:45:01.435580 19928 tablet_service.cc:1467] Processing CreateTablet for tablet f4630c147fb343b395c4ac5c8ff024d8 (DEFAULT_TABLE table=test-table [id=ea4ebe13a299400a8e94beb0b673e9f9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 04:45:01.435583 20156 tablet_service.cc:1467] Processing CreateTablet for tablet f4630c147fb343b395c4ac5c8ff024d8 (DEFAULT_TABLE table=test-table [id=ea4ebe13a299400a8e94beb0b673e9f9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 04:45:01.435643 20393 tablet_service.cc:1467] Processing CreateTablet for tablet f4630c147fb343b395c4ac5c8ff024d8 (DEFAULT_TABLE table=test-table [id=ea4ebe13a299400a8e94beb0b673e9f9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 04:45:01.435936 20393 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f4630c147fb343b395c4ac5c8ff024d8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 04:45:01.435941 19928 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f4630c147fb343b395c4ac5c8ff024d8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 04:45:01.435941 20156 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f4630c147fb343b395c4ac5c8ff024d8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 04:45:01.438572 20541 tablet_bootstrap.cc:492] T f4630c147fb343b395c4ac5c8ff024d8 P 07b6a9892d4a46b79343dd317d4efdbc: Bootstrap starting.
I20260810 04:45:01.439129 20542 tablet_bootstrap.cc:492] T f4630c147fb343b395c4ac5c8ff024d8 P 07f4cce57ae44844aa6a0af2105c6dad: Bootstrap starting.
I20260810 04:45:01.439172 20541 tablet_bootstrap.cc:654] T f4630c147fb343b395c4ac5c8ff024d8 P 07b6a9892d4a46b79343dd317d4efdbc: Neither blocks nor log segments found. Creating new log.
I20260810 04:45:01.439702 20541 log.cc:824] T f4630c147fb343b395c4ac5c8ff024d8 P 07b6a9892d4a46b79343dd317d4efdbc: Log is configured to *not* fsync() on all Append() calls
I20260810 04:45:01.439886 20542 tablet_bootstrap.cc:654] T f4630c147fb343b395c4ac5c8ff024d8 P 07f4cce57ae44844aa6a0af2105c6dad: Neither blocks nor log segments found. Creating new log.
I20260810 04:45:01.440056 20544 tablet_bootstrap.cc:492] T f4630c147fb343b395c4ac5c8ff024d8 P f5f8a930e59447c7ae7e2edf2001ba65: Bootstrap starting.
I20260810 04:45:01.440415 20541 tablet_bootstrap.cc:492] T f4630c147fb343b395c4ac5c8ff024d8 P 07b6a9892d4a46b79343dd317d4efdbc: No bootstrap required, opened a new log
I20260810 04:45:01.440477 20541 ts_tablet_manager.cc:1397] T f4630c147fb343b395c4ac5c8ff024d8 P 07b6a9892d4a46b79343dd317d4efdbc: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260810 04:45:01.440485 20542 log.cc:824] T f4630c147fb343b395c4ac5c8ff024d8 P 07f4cce57ae44844aa6a0af2105c6dad: Log is configured to *not* fsync() on all Append() calls
I20260810 04:45:01.440697 20544 tablet_bootstrap.cc:654] T f4630c147fb343b395c4ac5c8ff024d8 P f5f8a930e59447c7ae7e2edf2001ba65: Neither blocks nor log segments found. Creating new log.
I20260810 04:45:01.441349 20542 tablet_bootstrap.cc:492] T f4630c147fb343b395c4ac5c8ff024d8 P 07f4cce57ae44844aa6a0af2105c6dad: No bootstrap required, opened a new log
I20260810 04:45:01.441383 20544 log.cc:824] T f4630c147fb343b395c4ac5c8ff024d8 P f5f8a930e59447c7ae7e2edf2001ba65: Log is configured to *not* fsync() on all Append() calls
I20260810 04:45:01.441421 20542 ts_tablet_manager.cc:1397] T f4630c147fb343b395c4ac5c8ff024d8 P 07f4cce57ae44844aa6a0af2105c6dad: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260810 04:45:01.442179 20544 tablet_bootstrap.cc:492] T f4630c147fb343b395c4ac5c8ff024d8 P f5f8a930e59447c7ae7e2edf2001ba65: No bootstrap required, opened a new log
I20260810 04:45:01.442248 20544 ts_tablet_manager.cc:1397] T f4630c147fb343b395c4ac5c8ff024d8 P f5f8a930e59447c7ae7e2edf2001ba65: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260810 04:45:01.442248 20541 raft_consensus.cc:348] T f4630c147fb343b395c4ac5c8ff024d8 P 07b6a9892d4a46b79343dd317d4efdbc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07f4cce57ae44844aa6a0af2105c6dad" member_type: VOTER last_known_addr { host: "127.22.167.65" port: 42983 } } peers { permanent_uuid: "07b6a9892d4a46b79343dd317d4efdbc" member_type: VOTER last_known_addr { host: "127.22.167.67" port: 44303 } } peers { permanent_uuid: "f5f8a930e59447c7ae7e2edf2001ba65" member_type: VOTER last_known_addr { host: "127.22.167.66" port: 43349 } }
I20260810 04:45:01.442416 20541 raft_consensus.cc:374] T f4630c147fb343b395c4ac5c8ff024d8 P 07b6a9892d4a46b79343dd317d4efdbc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 04:45:01.442458 20541 raft_consensus.cc:729] T f4630c147fb343b395c4ac5c8ff024d8 P 07b6a9892d4a46b79343dd317d4efdbc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 07b6a9892d4a46b79343dd317d4efdbc, State: Initialized, Role: FOLLOWER
I20260810 04:45:01.442579 20541 consensus_queue.cc:260] T f4630c147fb343b395c4ac5c8ff024d8 P 07b6a9892d4a46b79343dd317d4efdbc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07f4cce57ae44844aa6a0af2105c6dad" member_type: VOTER last_known_addr { host: "127.22.167.65" port: 42983 } } peers { permanent_uuid: "07b6a9892d4a46b79343dd317d4efdbc" member_type: VOTER last_known_addr { host: "127.22.167.67" port: 44303 } } peers { permanent_uuid: "f5f8a930e59447c7ae7e2edf2001ba65" member_type: VOTER last_known_addr { host: "127.22.167.66" port: 43349 } }
I20260810 04:45:01.442868 20541 ts_tablet_manager.cc:1428] T f4630c147fb343b395c4ac5c8ff024d8 P 07b6a9892d4a46b79343dd317d4efdbc: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260810 04:45:01.442883 20516 heartbeater.cc:503] Master 127.22.167.126:38117 was elected leader, sending a full tablet report...
I20260810 04:45:01.443315 20542 raft_consensus.cc:348] T f4630c147fb343b395c4ac5c8ff024d8 P 07f4cce57ae44844aa6a0af2105c6dad [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07f4cce57ae44844aa6a0af2105c6dad" member_type: VOTER last_known_addr { host: "127.22.167.65" port: 42983 } } peers { permanent_uuid: "07b6a9892d4a46b79343dd317d4efdbc" member_type: VOTER last_known_addr { host: "127.22.167.67" port: 44303 } } peers { permanent_uuid: "f5f8a930e59447c7ae7e2edf2001ba65" member_type: VOTER last_known_addr { host: "127.22.167.66" port: 43349 } }
I20260810 04:45:01.443485 20542 raft_consensus.cc:374] T f4630c147fb343b395c4ac5c8ff024d8 P 07f4cce57ae44844aa6a0af2105c6dad [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 04:45:01.443538 20542 raft_consensus.cc:729] T f4630c147fb343b395c4ac5c8ff024d8 P 07f4cce57ae44844aa6a0af2105c6dad [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 07f4cce57ae44844aa6a0af2105c6dad, State: Initialized, Role: FOLLOWER
I20260810 04:45:01.443646 20542 consensus_queue.cc:260] T f4630c147fb343b395c4ac5c8ff024d8 P 07f4cce57ae44844aa6a0af2105c6dad [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07f4cce57ae44844aa6a0af2105c6dad" member_type: VOTER last_known_addr { host: "127.22.167.65" port: 42983 } } peers { permanent_uuid: "07b6a9892d4a46b79343dd317d4efdbc" member_type: VOTER last_known_addr { host: "127.22.167.67" port: 44303 } } peers { permanent_uuid: "f5f8a930e59447c7ae7e2edf2001ba65" member_type: VOTER last_known_addr { host: "127.22.167.66" port: 43349 } }
I20260810 04:45:01.443905 20542 ts_tablet_manager.cc:1428] T f4630c147fb343b395c4ac5c8ff024d8 P 07f4cce57ae44844aa6a0af2105c6dad: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260810 04:45:01.444041 20035 heartbeater.cc:503] Master 127.22.167.126:38117 was elected leader, sending a full tablet report...
I20260810 04:45:01.444077 20544 raft_consensus.cc:348] T f4630c147fb343b395c4ac5c8ff024d8 P f5f8a930e59447c7ae7e2edf2001ba65 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07f4cce57ae44844aa6a0af2105c6dad" member_type: VOTER last_known_addr { host: "127.22.167.65" port: 42983 } } peers { permanent_uuid: "07b6a9892d4a46b79343dd317d4efdbc" member_type: VOTER last_known_addr { host: "127.22.167.67" port: 44303 } } peers { permanent_uuid: "f5f8a930e59447c7ae7e2edf2001ba65" member_type: VOTER last_known_addr { host: "127.22.167.66" port: 43349 } }
I20260810 04:45:01.444221 20544 raft_consensus.cc:374] T f4630c147fb343b395c4ac5c8ff024d8 P f5f8a930e59447c7ae7e2edf2001ba65 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 04:45:01.444245 20544 raft_consensus.cc:729] T f4630c147fb343b395c4ac5c8ff024d8 P f5f8a930e59447c7ae7e2edf2001ba65 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f5f8a930e59447c7ae7e2edf2001ba65, State: Initialized, Role: FOLLOWER
I20260810 04:45:01.444348 20544 consensus_queue.cc:260] T f4630c147fb343b395c4ac5c8ff024d8 P f5f8a930e59447c7ae7e2edf2001ba65 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07f4cce57ae44844aa6a0af2105c6dad" member_type: VOTER last_known_addr { host: "127.22.167.65" port: 42983 } } peers { permanent_uuid: "07b6a9892d4a46b79343dd317d4efdbc" member_type: VOTER last_known_addr { host: "127.22.167.67" port: 44303 } } peers { permanent_uuid: "f5f8a930e59447c7ae7e2edf2001ba65" member_type: VOTER last_known_addr { host: "127.22.167.66" port: 43349 } }
I20260810 04:45:01.444614 20544 ts_tablet_manager.cc:1428] T f4630c147fb343b395c4ac5c8ff024d8 P f5f8a930e59447c7ae7e2edf2001ba65: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260810 04:45:01.444651 20275 heartbeater.cc:503] Master 127.22.167.126:38117 was elected leader, sending a full tablet report...
W20260810 04:45:01.503854 20276 tablet.cc:2369] T f4630c147fb343b395c4ac5c8ff024d8 P f5f8a930e59447c7ae7e2edf2001ba65: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 04:45:01.508353 20551 raft_consensus.cc:482] T f4630c147fb343b395c4ac5c8ff024d8 P 07b6a9892d4a46b79343dd317d4efdbc [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 04:45:01.508460 20551 raft_consensus.cc:504] T f4630c147fb343b395c4ac5c8ff024d8 P 07b6a9892d4a46b79343dd317d4efdbc [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07f4cce57ae44844aa6a0af2105c6dad" member_type: VOTER last_known_addr { host: "127.22.167.65" port: 42983 } } peers { permanent_uuid: "07b6a9892d4a46b79343dd317d4efdbc" member_type: VOTER last_known_addr { host: "127.22.167.67" port: 44303 } } peers { permanent_uuid: "f5f8a930e59447c7ae7e2edf2001ba65" member_type: VOTER last_known_addr { host: "127.22.167.66" port: 43349 } }
I20260810 04:45:01.508793 20551 leader_election.cc:288] T f4630c147fb343b395c4ac5c8ff024d8 P 07b6a9892d4a46b79343dd317d4efdbc [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 07f4cce57ae44844aa6a0af2105c6dad (127.22.167.65:42983), f5f8a930e59447c7ae7e2edf2001ba65 (127.22.167.66:43349)
I20260810 04:45:01.511721 20520 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:45:01.508949 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.65:42983 (local address 127.22.167.67:43357)
0810 04:45:01.509005 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0810 04:45:01.509028 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0810 04:45:01.509080 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 04:45:01.509206 (+ 126us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 04:45:01.509207 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 04:45:01.509208 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 04:45:01.509283 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:45:01.509285 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:45:01.510187 (+ 902us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:45:01.510188 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:45:01.510931 (+ 743us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:45:01.510933 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:45:01.511468 (+ 535us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:45:01.511469 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:45:01.511548 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:45:01.511551 (+ 3us) client_negotiation.cc:769] Sending connection context
0810 04:45:01.511597 (+ 46us) client_negotiation.cc:240] Negotiation successful
0810 04:45:01.511638 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":28}
I20260810 04:45:01.511721 20533 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:45:01.509036 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.167.67:43357 (local address 127.22.167.65:42983)
0810 04:45:01.509086 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0810 04:45:01.509088 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 04:45:01.509115 (+ 27us) server_negotiation.cc:407] Connection header received
0810 04:45:01.509140 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:45:01.509141 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:45:01.509150 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:45:01.509194 (+ 44us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 04:45:01.509346 (+ 152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:45:01.510101 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:45:01.511019 (+ 918us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:45:01.511401 (+ 382us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:45:01.511464 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:45:01.511474 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0810 04:45:01.511614 (+ 140us) server_negotiation.cc:299] Negotiation successful
0810 04:45:01.511655 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260810 04:45:01.511902 20540 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:45:01.509135 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.167.67:58821 (local address 127.22.167.66:43349)
0810 04:45:01.509193 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0810 04:45:01.509194 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0810 04:45:01.509320 (+ 126us) server_negotiation.cc:407] Connection header received
0810 04:45:01.509374 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:45:01.509375 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:45:01.509383 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:45:01.509433 (+ 50us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 04:45:01.509545 (+ 112us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:45:01.510248 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:45:01.511183 (+ 935us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:45:01.511576 (+ 393us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:45:01.511638 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:45:01.511646 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0810 04:45:01.511785 (+ 139us) server_negotiation.cc:299] Negotiation successful
0810 04:45:01.511825 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260810 04:45:01.512001 20554 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:45:01.509052 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.66:43349 (local address 127.22.167.67:58821)
0810 04:45:01.509252 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0810 04:45:01.509263 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0810 04:45:01.509323 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 04:45:01.509438 (+ 115us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 04:45:01.509439 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 04:45:01.509440 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 04:45:01.509494 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:45:01.509495 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:45:01.510317 (+ 822us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:45:01.510318 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:45:01.511097 (+ 779us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:45:01.511099 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:45:01.511650 (+ 551us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:45:01.511651 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:45:01.511723 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:45:01.511727 (+ 4us) client_negotiation.cc:769] Sending connection context
0810 04:45:01.511890 (+ 163us) client_negotiation.cc:240] Negotiation successful
0810 04:45:01.511930 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":185,"thread_start_us":120,"threads_started":1}
I20260810 04:45:01.512080 19958 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f4630c147fb343b395c4ac5c8ff024d8" candidate_uuid: "07b6a9892d4a46b79343dd317d4efdbc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "07f4cce57ae44844aa6a0af2105c6dad" is_pre_election: true
I20260810 04:45:01.512236 19958 raft_consensus.cc:2454] T f4630c147fb343b395c4ac5c8ff024d8 P 07f4cce57ae44844aa6a0af2105c6dad [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 07b6a9892d4a46b79343dd317d4efdbc in term 0.
I20260810 04:45:01.512472 20318 leader_election.cc:302] T f4630c147fb343b395c4ac5c8ff024d8 P 07b6a9892d4a46b79343dd317d4efdbc [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 07b6a9892d4a46b79343dd317d4efdbc, 07f4cce57ae44844aa6a0af2105c6dad; no voters:
I20260810 04:45:01.512609 20190 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f4630c147fb343b395c4ac5c8ff024d8" candidate_uuid: "07b6a9892d4a46b79343dd317d4efdbc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f5f8a930e59447c7ae7e2edf2001ba65" is_pre_election: true
I20260810 04:45:01.512635 20551 raft_consensus.cc:2781] T f4630c147fb343b395c4ac5c8ff024d8 P 07b6a9892d4a46b79343dd317d4efdbc [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 04:45:01.512696 20551 raft_consensus.cc:482] T f4630c147fb343b395c4ac5c8ff024d8 P 07b6a9892d4a46b79343dd317d4efdbc [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 04:45:01.512719 20551 raft_consensus.cc:3037] T f4630c147fb343b395c4ac5c8ff024d8 P 07b6a9892d4a46b79343dd317d4efdbc [term 0 FOLLOWER]: Advancing to term 1
I20260810 04:45:01.512754 20190 raft_consensus.cc:2454] T f4630c147fb343b395c4ac5c8ff024d8 P f5f8a930e59447c7ae7e2edf2001ba65 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 07b6a9892d4a46b79343dd317d4efdbc in term 0.
I20260810 04:45:01.513459 20551 raft_consensus.cc:504] T f4630c147fb343b395c4ac5c8ff024d8 P 07b6a9892d4a46b79343dd317d4efdbc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07f4cce57ae44844aa6a0af2105c6dad" member_type: VOTER last_known_addr { host: "127.22.167.65" port: 42983 } } peers { permanent_uuid: "07b6a9892d4a46b79343dd317d4efdbc" member_type: VOTER last_known_addr { host: "127.22.167.67" port: 44303 } } peers { permanent_uuid: "f5f8a930e59447c7ae7e2edf2001ba65" member_type: VOTER last_known_addr { host: "127.22.167.66" port: 43349 } }
I20260810 04:45:01.513563 20551 leader_election.cc:288] T f4630c147fb343b395c4ac5c8ff024d8 P 07b6a9892d4a46b79343dd317d4efdbc [CANDIDATE]: Term 1 election: Requested vote from peers 07f4cce57ae44844aa6a0af2105c6dad (127.22.167.65:42983), f5f8a930e59447c7ae7e2edf2001ba65 (127.22.167.66:43349)
I20260810 04:45:01.513749 20190 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f4630c147fb343b395c4ac5c8ff024d8" candidate_uuid: "07b6a9892d4a46b79343dd317d4efdbc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f5f8a930e59447c7ae7e2edf2001ba65"
I20260810 04:45:01.513742 19958 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f4630c147fb343b395c4ac5c8ff024d8" candidate_uuid: "07b6a9892d4a46b79343dd317d4efdbc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "07f4cce57ae44844aa6a0af2105c6dad"
I20260810 04:45:01.513825 20190 raft_consensus.cc:3037] T f4630c147fb343b395c4ac5c8ff024d8 P f5f8a930e59447c7ae7e2edf2001ba65 [term 0 FOLLOWER]: Advancing to term 1
I20260810 04:45:01.513824 19958 raft_consensus.cc:3037] T f4630c147fb343b395c4ac5c8ff024d8 P 07f4cce57ae44844aa6a0af2105c6dad [term 0 FOLLOWER]: Advancing to term 1
I20260810 04:45:01.514727 20190 raft_consensus.cc:2454] T f4630c147fb343b395c4ac5c8ff024d8 P f5f8a930e59447c7ae7e2edf2001ba65 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 07b6a9892d4a46b79343dd317d4efdbc in term 1.
I20260810 04:45:01.514727 19958 raft_consensus.cc:2454] T f4630c147fb343b395c4ac5c8ff024d8 P 07f4cce57ae44844aa6a0af2105c6dad [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 07b6a9892d4a46b79343dd317d4efdbc in term 1.
I20260810 04:45:01.514904 20318 leader_election.cc:302] T f4630c147fb343b395c4ac5c8ff024d8 P 07b6a9892d4a46b79343dd317d4efdbc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 07b6a9892d4a46b79343dd317d4efdbc, 07f4cce57ae44844aa6a0af2105c6dad; no voters:
I20260810 04:45:01.514986 20551 raft_consensus.cc:2781] T f4630c147fb343b395c4ac5c8ff024d8 P 07b6a9892d4a46b79343dd317d4efdbc [term 1 FOLLOWER]: Leader election won for term 1
I20260810 04:45:01.515128 20551 raft_consensus.cc:686] T f4630c147fb343b395c4ac5c8ff024d8 P 07b6a9892d4a46b79343dd317d4efdbc [term 1 LEADER]: Becoming Leader. State: Replica: 07b6a9892d4a46b79343dd317d4efdbc, State: Running, Role: LEADER
I20260810 04:45:01.515223 20551 consensus_queue.cc:237] T f4630c147fb343b395c4ac5c8ff024d8 P 07b6a9892d4a46b79343dd317d4efdbc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07f4cce57ae44844aa6a0af2105c6dad" member_type: VOTER last_known_addr { host: "127.22.167.65" port: 42983 } } peers { permanent_uuid: "07b6a9892d4a46b79343dd317d4efdbc" member_type: VOTER last_known_addr { host: "127.22.167.67" port: 44303 } } peers { permanent_uuid: "f5f8a930e59447c7ae7e2edf2001ba65" member_type: VOTER last_known_addr { host: "127.22.167.66" port: 43349 } }
I20260810 04:45:01.516019 19702 catalog_manager.cc:5495] T f4630c147fb343b395c4ac5c8ff024d8 P 07b6a9892d4a46b79343dd317d4efdbc reported cstate change: term changed from 0 to 1, leader changed from <none> to 07b6a9892d4a46b79343dd317d4efdbc (127.22.167.67). New cstate: current_term: 1 leader_uuid: "07b6a9892d4a46b79343dd317d4efdbc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07f4cce57ae44844aa6a0af2105c6dad" member_type: VOTER last_known_addr { host: "127.22.167.65" port: 42983 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "07b6a9892d4a46b79343dd317d4efdbc" member_type: VOTER last_known_addr { host: "127.22.167.67" port: 44303 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f5f8a930e59447c7ae7e2edf2001ba65" member_type: VOTER last_known_addr { host: "127.22.167.66" port: 43349 } health_report { overall_health: UNKNOWN } } }
I20260810 04:45:01.527112 20532 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:45:01.524948 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44020 (local address 127.22.167.67:44303)
0810 04:45:01.525001 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0810 04:45:01.525003 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 04:45:01.525011 (+ 8us) server_negotiation.cc:407] Connection header received
0810 04:45:01.525060 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:45:01.525061 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:45:01.525076 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:45:01.525133 (+ 57us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 04:45:01.525261 (+ 128us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:45:01.525693 (+ 432us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:45:01.526447 (+ 754us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:45:01.526624 (+ 177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:45:01.526676 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:45:01.526818 (+ 142us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 04:45:01.526908 (+ 90us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 04:45:01.526977 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0810 04:45:01.527013 (+ 36us) server_negotiation.cc:299] Negotiation successful
0810 04:45:01.527032 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260810 04:45:01.528641 20190 raft_consensus.cc:1261] T f4630c147fb343b395c4ac5c8ff024d8 P f5f8a930e59447c7ae7e2edf2001ba65 [term 1 FOLLOWER]: Refusing update from remote peer 07b6a9892d4a46b79343dd317d4efdbc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 04:45:01.528640 19958 raft_consensus.cc:1261] T f4630c147fb343b395c4ac5c8ff024d8 P 07f4cce57ae44844aa6a0af2105c6dad [term 1 FOLLOWER]: Refusing update from remote peer 07b6a9892d4a46b79343dd317d4efdbc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 04:45:01.528939 20551 consensus_queue.cc:1035] T f4630c147fb343b395c4ac5c8ff024d8 P 07b6a9892d4a46b79343dd317d4efdbc [LEADER]: Connected to new peer: Peer: permanent_uuid: "07f4cce57ae44844aa6a0af2105c6dad" member_type: VOTER last_known_addr { host: "127.22.167.65" port: 42983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 04:45:01.529070 20555 consensus_queue.cc:1035] T f4630c147fb343b395c4ac5c8ff024d8 P 07b6a9892d4a46b79343dd317d4efdbc [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5f8a930e59447c7ae7e2edf2001ba65" member_type: VOTER last_known_addr { host: "127.22.167.66" port: 43349 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 04:45:01.531287 20561 mvcc.cc:205] Tried to move back new op lower bound from 7316836767859204096 to 7316836767806996480. Current Snapshot: MvccSnapshot[applied={T|T < 7316836767859204096}]
I20260810 04:45:01.531651 20560 mvcc.cc:205] Tried to move back new op lower bound from 7316836767859204096 to 7316836767806996480. Current Snapshot: MvccSnapshot[applied={T|T < 7316836767859204096}]
I20260810 04:45:01.533622 19702 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54042:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 04:45:01.533704 19702 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54042:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 04:45:01.534446 19702 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 233d162ff73d4c03b8d145ded2b1b68a: Sending DeleteTablet for 3 replicas of tablet f4630c147fb343b395c4ac5c8ff024d8
I20260810 04:45:01.534768 20393 tablet_service.cc:1514] Processing DeleteTablet for tablet f4630c147fb343b395c4ac5c8ff024d8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 04:45:01 UTC) from {username='kudu', principal='kudu/127.22.167.126@KRBTEST.COM'} at 127.0.0.1:44010
I20260810 04:45:01.534765 19928 tablet_service.cc:1514] Processing DeleteTablet for tablet f4630c147fb343b395c4ac5c8ff024d8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 04:45:01 UTC) from {username='kudu', principal='kudu/127.22.167.126@KRBTEST.COM'} at 127.0.0.1:48758
I20260810 04:45:01.535022 20577 tablet_replica.cc:330] T f4630c147fb343b395c4ac5c8ff024d8 P 07f4cce57ae44844aa6a0af2105c6dad: stopping tablet replica
I20260810 04:45:01.535054 20156 tablet_service.cc:1514] Processing DeleteTablet for tablet f4630c147fb343b395c4ac5c8ff024d8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 04:45:01 UTC) from {username='kudu', principal='kudu/127.22.167.126@KRBTEST.COM'} at 127.0.0.1:46490
I20260810 04:45:01.535184 20577 raft_consensus.cc:2229] T f4630c147fb343b395c4ac5c8ff024d8 P 07f4cce57ae44844aa6a0af2105c6dad [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 04:45:01.535272 20582 tablet_replica.cc:330] T f4630c147fb343b395c4ac5c8ff024d8 P f5f8a930e59447c7ae7e2edf2001ba65: stopping tablet replica
I20260810 04:45:01.535383 20577 raft_consensus.cc:2258] T f4630c147fb343b395c4ac5c8ff024d8 P 07f4cce57ae44844aa6a0af2105c6dad [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 04:45:01.535454 20575 tablet_replica.cc:330] T f4630c147fb343b395c4ac5c8ff024d8 P 07b6a9892d4a46b79343dd317d4efdbc: stopping tablet replica
I20260810 04:45:01.535523 23197 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskemv_AM/build/release/bin/kudu with pid 19809
I20260810 04:45:01.535542 20575 raft_consensus.cc:2229] T f4630c147fb343b395c4ac5c8ff024d8 P 07b6a9892d4a46b79343dd317d4efdbc [term 1 LEADER]: Raft consensus shutting down.
I20260810 04:45:01.535640 20575 raft_consensus.cc:2258] T f4630c147fb343b395c4ac5c8ff024d8 P 07b6a9892d4a46b79343dd317d4efdbc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 04:45:01.560768 20575 ts_tablet_manager.cc:1905] T f4630c147fb343b395c4ac5c8ff024d8 P 07b6a9892d4a46b79343dd317d4efdbc: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 04:45:01.562856 20575 ts_tablet_manager.cc:1918] T f4630c147fb343b395c4ac5c8ff024d8 P 07b6a9892d4a46b79343dd317d4efdbc: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 04:45:01.562947 20575 log.cc:1196] T f4630c147fb343b395c4ac5c8ff024d8 P 07b6a9892d4a46b79343dd317d4efdbc: Deleting WAL directory at /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786336863388938-23197-0/minicluster-data/ts-2/wal/wals/f4630c147fb343b395c4ac5c8ff024d8
I20260810 04:45:01.563311 20575 ts_tablet_manager.cc:1939] T f4630c147fb343b395c4ac5c8ff024d8 P 07b6a9892d4a46b79343dd317d4efdbc: Deleting consensus metadata
I20260810 04:45:01.566968 19677 catalog_manager.cc:4841] TS 07b6a9892d4a46b79343dd317d4efdbc (127.22.167.67:44303): tablet f4630c147fb343b395c4ac5c8ff024d8 (table test-table [id=ea4ebe13a299400a8e94beb0b673e9f9]) successfully deleted
I20260810 04:45:01.568454 20582 raft_consensus.cc:2229] T f4630c147fb343b395c4ac5c8ff024d8 P f5f8a930e59447c7ae7e2edf2001ba65 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 04:45:01.568635 20582 raft_consensus.cc:2258] T f4630c147fb343b395c4ac5c8ff024d8 P f5f8a930e59447c7ae7e2edf2001ba65 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 04:45:01.569154 20582 ts_tablet_manager.cc:1905] T f4630c147fb343b395c4ac5c8ff024d8 P f5f8a930e59447c7ae7e2edf2001ba65: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 04:45:01.570757 20582 ts_tablet_manager.cc:1918] T f4630c147fb343b395c4ac5c8ff024d8 P f5f8a930e59447c7ae7e2edf2001ba65: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 04:45:01.570837 20582 log.cc:1196] T f4630c147fb343b395c4ac5c8ff024d8 P f5f8a930e59447c7ae7e2edf2001ba65: Deleting WAL directory at /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786336863388938-23197-0/minicluster-data/ts-1/wal/wals/f4630c147fb343b395c4ac5c8ff024d8
I20260810 04:45:01.571096 20582 ts_tablet_manager.cc:1939] T f4630c147fb343b395c4ac5c8ff024d8 P f5f8a930e59447c7ae7e2edf2001ba65: Deleting consensus metadata
I20260810 04:45:01.571538 19683 catalog_manager.cc:4841] TS f5f8a930e59447c7ae7e2edf2001ba65 (127.22.167.66:43349): tablet f4630c147fb343b395c4ac5c8ff024d8 (table test-table [id=ea4ebe13a299400a8e94beb0b673e9f9]) successfully deleted
W20260810 04:45:01.572613 19683 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.22.167.65:42983 (error 108)
I20260810 04:45:01.572825 23197 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskemv_AM/build/release/bin/kudu with pid 20041
I20260810 04:45:01.578454 20529 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:45:01.572970 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.65:42983 (local address 127.0.0.1:48764)
0810 04:45:01.578339 (+ 5369us) negotiation.cc:107] Waiting for socket to connect
0810 04:45:01.578410 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.167.65:42983: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":5336}
W20260810 04:45:01.581039 19683 catalog_manager.cc:4584] TS 07f4cce57ae44844aa6a0af2105c6dad (127.22.167.65:42983): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet f4630c147fb343b395c4ac5c8ff024d8: Network error: Client connection negotiation failed: client connection to 127.22.167.65:42983: connect: Connection refused (error 111)
I20260810 04:45:01.596575 23197 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskemv_AM/build/release/bin/kudu with pid 20284
I20260810 04:45:01.611265 23197 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskemv_AM/build/release/bin/kudu with pid 19645
2026-08-10T04:45:01Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/0 (2102 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/1
2026-08-10T04:45:01Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T04:45:01Z Disabled control of system clock
I20260810 04:45:01.641793 23197 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskemv_AM/build/release/bin/kudu
/tmp/dist-test-taskemv_AM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786336863388938-23197-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786336863388938-23197-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786336863388938-23197-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786336863388938-23197-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.167.126:40521
--webserver_interface=127.22.167.126
--webserver_port=0
--builtin_ntp_servers=127.22.167.84:39297
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.167.126:40521
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260810 04:45:01.722508 20595 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 04:45:01.722730 20595 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 04:45:01.722751 20595 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 04:45:01.723817 20595 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 04:45:01.723850 20595 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 04:45:01.723860 20595 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 04:45:01.723871 20595 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 04:45:01.723882 20595 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 04:45:01.725132 20595 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.167.84:39297
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786336863388938-23197-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786336863388938-23197-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.22.167.126:40521
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.167.126:40521
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786336863388938-23197-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.167.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20595
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786336863388938-23197-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 03:54:01 UTC on ccycloud.quasar-icijmt.root.comops.site
build id 4577009
I20260810 04:45:01.725339 20595 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 04:45:01.725580 20595 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 04:45:01.730072 20603 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 04:45:01.730154 20605 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 04:45:01.730158 20602 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 04:45:01.730284 20595 server_base.cc:1034] running on GCE node
I20260810 04:45:01.730528 20595 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 04:45:01.730792 20595 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 04:45:01.731921 20595 hybrid_clock.cc:648] HybridClock initialized: now 1786337101731912 us; error 28 us; skew 500 ppm
I20260810 04:45:01.733166 20595 webserver.cc:460] Webserver started at http://127.22.167.126:38453/ using document root <none> and password file <none>
I20260810 04:45:01.733393 20595 fs_manager.cc:359] Metadata directory not provided
I20260810 04:45:01.733438 20595 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 04:45:01.733562 20595 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 04:45:01.734635 20595 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786336863388938-23197-0/minicluster-data/master-0/data/instance:
uuid: "4770ad600718452a9358497e0e772fee"
format_stamp: "Formatted at 2026-08-10 04:45:01 on dist-test-slave-bqcl"
server_key: "bafc7a2468dea0ec1c44d796934260d9"
server_key_iv: "65566dc7f1cfe8a43dc5a97eed502429"
server_key_version: "encryptionkey@0"
I20260810 04:45:01.734969 20595 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786336863388938-23197-0/minicluster-data/master-0/wal/instance:
uuid: "4770ad600718452a9358497e0e772fee"
format_stamp: "Formatted at 2026-08-10 04:45:01 on dist-test-slave-bqcl"
server_key: "bafc7a2468dea0ec1c44d796934260d9"
server_key_iv: "65566dc7f1cfe8a43dc5a97eed502429"
server_key_version: "encryptionkey@0"
I20260810 04:45:01.736228 20595 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260810 04:45:01.736980 20616 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 04:45:01.737130 20595 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 04:45:01.737210 20595 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786336863388938-23197-0/minicluster-data/master-0/data,/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786336863388938-23197-0/minicluster-data/master-0/wal
uuid: "4770ad600718452a9358497e0e772fee"
format_stamp: "Formatted at 2026-08-10 04:45:01 on dist-test-slave-bqcl"
server_key: "bafc7a2468dea0ec1c44d796934260d9"
server_key_iv: "65566dc7f1cfe8a43dc5a97eed502429"
server_key_version: "encryptionkey@0"
I20260810 04:45:01.737275 20595 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786336863388938-23197-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786336863388938-23197-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786336863388938-23197-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 04:45:01.752007 20595 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 04:45:01.752300 20595 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 04:45:01.752431 20595 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 04:45:01.756422 20595 rpc_server.cc:307] RPC server started. Bound to: 127.22.167.126:40521
I20260810 04:45:01.756456 20700 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.167.126:40521 every 8 connection(s)
I20260810 04:45:01.756790 20595 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786336863388938-23197-0/minicluster-data/master-0/data/info.pb
I20260810 04:45:01.757799 20701 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 04:45:01.760205 20701 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4770ad600718452a9358497e0e772fee: Bootstrap starting.
I20260810 04:45:01.760780 20701 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4770ad600718452a9358497e0e772fee: Neither blocks nor log segments found. Creating new log.
I20260810 04:45:01.761080 20701 log.cc:824] T 00000000000000000000000000000000 P 4770ad600718452a9358497e0e772fee: Log is configured to *not* fsync() on all Append() calls
I20260810 04:45:01.761735 20701 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4770ad600718452a9358497e0e772fee: No bootstrap required, opened a new log
I20260810 04:45:01.763425 20701 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4770ad600718452a9358497e0e772fee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4770ad600718452a9358497e0e772fee" member_type: VOTER last_known_addr { host: "127.22.167.126" port: 40521 } }
I20260810 04:45:01.763576 20701 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4770ad600718452a9358497e0e772fee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 04:45:01.763617 20701 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4770ad600718452a9358497e0e772fee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4770ad600718452a9358497e0e772fee, State: Initialized, Role: FOLLOWER
I20260810 04:45:01.763767 20701 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4770ad600718452a9358497e0e772fee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4770ad600718452a9358497e0e772fee" member_type: VOTER last_known_addr { host: "127.22.167.126" port: 40521 } }
I20260810 04:45:01.763847 20701 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4770ad600718452a9358497e0e772fee [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 04:45:01.763886 20701 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4770ad600718452a9358497e0e772fee [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 04:45:01.763933 20701 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4770ad600718452a9358497e0e772fee [term 0 FOLLOWER]: Advancing to term 1
I20260810 04:45:01.764464 20701 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4770ad600718452a9358497e0e772fee [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4770ad600718452a9358497e0e772fee" member_type: VOTER last_known_addr { host: "127.22.167.126" port: 40521 } }
I20260810 04:45:01.764568 20701 leader_election.cc:302] T 00000000000000000000000000000000 P 4770ad600718452a9358497e0e772fee [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4770ad600718452a9358497e0e772fee; no voters:
I20260810 04:45:01.764737 20701 leader_election.cc:288] T 00000000000000000000000000000000 P 4770ad600718452a9358497e0e772fee [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 04:45:01.764775 20706 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4770ad600718452a9358497e0e772fee [term 1 FOLLOWER]: Leader election won for term 1
I20260810 04:45:01.764906 20706 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4770ad600718452a9358497e0e772fee [term 1 LEADER]: Becoming Leader. State: Replica: 4770ad600718452a9358497e0e772fee, State: Running, Role: LEADER
I20260810 04:45:01.765012 20701 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4770ad600718452a9358497e0e772fee [sys.catalog]: configured and running, proceeding with master startup.
I20260810 04:45:01.765038 20706 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4770ad600718452a9358497e0e772fee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4770ad600718452a9358497e0e772fee" member_type: VOTER last_known_addr { host: "127.22.167.126" port: 40521 } }
I20260810 04:45:01.765425 20711 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4770ad600718452a9358497e0e772fee [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4770ad600718452a9358497e0e772fee" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4770ad600718452a9358497e0e772fee" member_type: VOTER last_known_addr { host: "127.22.167.126" port: 40521 } } }
I20260810 04:45:01.765506 20711 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4770ad600718452a9358497e0e772fee [sys.catalog]: This master's current role is: LEADER
I20260810 04:45:01.765622 20712 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4770ad600718452a9358497e0e772fee [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4770ad600718452a9358497e0e772fee. Latest consensus state: current_term: 1 leader_uuid: "4770ad600718452a9358497e0e772fee" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4770ad600718452a9358497e0e772fee" member_type: VOTER last_known_addr { host: "127.22.167.126" port: 40521 } } }
I20260810 04:45:01.765720 20720 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 04:45:01.766304 23197 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskemv_AM/build/release/bin/kudu as pid 20595
I20260810 04:45:01.766407 23197 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786336863388938-23197-0/minicluster-data/master-0/wal/instance
I20260810 04:45:01.766332 20720 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 04:45:01.766599 23197 external_mini_cluster.cc:1428] Setting key 90d6500e42f48ac6366efdbcb9684af3
I20260810 04:45:01.766630 20712 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4770ad600718452a9358497e0e772fee [sys.catalog]: This master's current role is: LEADER
I20260810 04:45:01.768479 20720 catalog_manager.cc:1347] Generated new cluster ID: 1447e8e8340a4395a11d76186c82989b
I20260810 04:45:01.768544 20720 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 04:45:01.772671 20742 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:45:01.767064 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50708 (local address 127.22.167.126:40521)
0810 04:45:01.767308 (+ 244us) server_negotiation.cc:206] Beginning negotiation
0810 04:45:01.767317 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 04:45:01.767519 (+ 202us) server_negotiation.cc:407] Connection header received
0810 04:45:01.767635 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:45:01.767641 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:45:01.767695 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:45:01.767826 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0810 04:45:01.768033 (+ 207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:45:01.769184 (+ 1151us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:45:01.770308 (+ 1124us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:45:01.770923 (+ 615us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:45:01.771043 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:45:01.771777 (+ 734us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 04:45:01.771781 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 04:45:01.771781 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 04:45:01.771994 (+ 213us) server_negotiation.cc:922] calling sasl_server_start()
0810 04:45:01.772036 (+ 42us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 04:45:01.772050 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 04:45:01.772118 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0810 04:45:01.772200 (+ 82us) server_negotiation.cc:299] Negotiation successful
0810 04:45:01.772274 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":158,"thread_start_us":95,"threads_started":1}
I20260810 04:45:01.776583 20720 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 04:45:01.777092 20720 catalog_manager.cc:1504] Loading token signing keys...
I20260810 04:45:01.780279 20720 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4770ad600718452a9358497e0e772fee: Generated new TSK 0
I20260810 04:45:01.780434 20720 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 04:45:01.784426 23197 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskemv_AM/build/release/bin/kudu
/tmp/dist-test-taskemv_AM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786336863388938-23197-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786336863388938-23197-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786336863388938-23197-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786336863388938-23197-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.167.65:0
--local_ip_for_outbound_sockets=127.22.167.65
--webserver_interface=127.22.167.65
--webserver_port=0
--tserver_master_addrs=127.22.167.126:40521
--builtin_ntp_servers=127.22.167.84:39297
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260810 04:45:01.882311 20744 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 04:45:01.882520 20744 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 04:45:01.882542 20744 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 04:45:01.883679 20744 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 04:45:01.883718 20744 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 04:45:01.883741 20744 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.167.65
I20260810 04:45:01.885058 20744 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.167.84:39297
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786336863388938-23197-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786336863388938-23197-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.167.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786336863388938-23197-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.22.167.65
--webserver_port=0
--tserver_master_addrs=127.22.167.126:40521
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20744
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.167.65
--log_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786336863388938-23197-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 03:54:01 UTC on ccycloud.quasar-icijmt.root.comops.site
build id 4577009
I20260810 04:45:01.885293 20744 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 04:45:01.885577 20744 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 04:45:01.891302 20752 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 04:45:01.891275 20754 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 04:45:01.891296 20759 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 04:45:01.891690 20744 server_base.cc:1034] running on GCE node
I20260810 04:45:01.891885 20744 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 04:45:01.892117 20744 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 04:45:01.893280 20744 hybrid_clock.cc:648] HybridClock initialized: now 1786337101893263 us; error 30 us; skew 500 ppm
I20260810 04:45:01.894579 20744 webserver.cc:460] Webserver started at http://127.22.167.65:33195/ using document root <none> and password file <none>
I20260810 04:45:01.894800 20744 fs_manager.cc:359] Metadata directory not provided
I20260810 04:45:01.894848 20744 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 04:45:01.894977 20744 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 04:45:01.896021 20744 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786336863388938-23197-0/minicluster-data/ts-0/data/instance:
uuid: "dbe5d51105f142bb9a45d0be0fbf3f52"
format_stamp: "Formatted at 2026-08-10 04:45:01 on dist-test-slave-bqcl"
server_key: "ffac28e51c150965f4fbbc91d149ede0"
server_key_iv: "c36a5a70947e3a555731a2ec7797fe4e"
server_key_version: "encryptionkey@0"
I20260810 04:45:01.896360 20744 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786336863388938-23197-0/minicluster-data/ts-0/wal/instance:
uuid: "dbe5d51105f142bb9a45d0be0fbf3f52"
format_stamp: "Formatted at 2026-08-10 04:45:01 on dist-test-slave-bqcl"
server_key: "ffac28e51c150965f4fbbc91d149ede0"
server_key_iv: "c36a5a70947e3a555731a2ec7797fe4e"
server_key_version: "encryptionkey@0"
I20260810 04:45:01.897585 20744 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260810 04:45:01.898339 20771 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 04:45:01.898505 20744 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 04:45:01.898591 20744 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786336863388938-23197-0/minicluster-data/ts-0/data,/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786336863388938-23197-0/minicluster-data/ts-0/wal
uuid: "dbe5d51105f142bb9a45d0be0fbf3f52"
format_stamp: "Formatted at 2026-08-10 04:45:01 on dist-test-slave-bqcl"
server_key: "ffac28e51c150965f4fbbc91d149ede0"
server_key_iv: "c36a5a70947e3a555731a2ec7797fe4e"
server_key_version: "encryptionkey@0"
I20260810 04:45:01.898660 20744 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786336863388938-23197-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786336863388938-23197-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786336863388938-23197-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 04:45:01.924335 20744 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 04:45:01.924679 20744 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 04:45:01.924817 20744 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 04:45:01.925064 20744 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 04:45:01.925417 20744 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 04:45:01.925452 20744 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 04:45:01.925484 20744 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 04:45:01.925500 20744 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 04:45:01.931089 20744 rpc_server.cc:307] RPC server started. Bound to: 127.22.167.65:39099
I20260810 04:45:01.931136 20972 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.167.65:39099 every 8 connection(s)
I20260810 04:45:01.931443 20744 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786336863388938-23197-0/minicluster-data/ts-0/data/info.pb
I20260810 04:45:01.933957 23197 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskemv_AM/build/release/bin/kudu as pid 20744
I20260810 04:45:01.934041 23197 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786336863388938-23197-0/minicluster-data/ts-0/wal/instance
I20260810 04:45:01.934233 23197 external_mini_cluster.cc:1428] Setting key d58602cf363f234fded196bbfb63c7ca
I20260810 04:45:01.935307 23197 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskemv_AM/build/release/bin/kudu
/tmp/dist-test-taskemv_AM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786336863388938-23197-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786336863388938-23197-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786336863388938-23197-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786336863388938-23197-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.167.66:0
--local_ip_for_outbound_sockets=127.22.167.66
--webserver_interface=127.22.167.66
--webserver_port=0
--tserver_master_addrs=127.22.167.126:40521
--builtin_ntp_servers=127.22.167.84:39297
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260810 04:45:01.937004 20742 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:45:01.932124 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.167.65:50055 (local address 127.22.167.126:40521)
0810 04:45:01.932205 (+ 81us) server_negotiation.cc:206] Beginning negotiation
0810 04:45:01.932207 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 04:45:01.932396 (+ 189us) server_negotiation.cc:407] Connection header received
0810 04:45:01.932883 (+ 487us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:45:01.932883 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:45:01.932896 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:45:01.932946 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0810 04:45:01.933588 (+ 642us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:45:01.934191 (+ 603us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:45:01.935478 (+ 1287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:45:01.935713 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:45:01.935752 (+ 39us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:45:01.936718 (+ 966us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 04:45:01.936720 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 04:45:01.936720 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 04:45:01.936725 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0810 04:45:01.936742 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 04:45:01.936746 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 04:45:01.936796 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0810 04:45:01.936885 (+ 89us) server_negotiation.cc:299] Negotiation successful
0810 04:45:01.936917 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260810 04:45:01.937230 20979 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:45:01.932102 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.126:40521 (local address 127.22.167.65:50055)
0810 04:45:01.932279 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0810 04:45:01.932314 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0810 04:45:01.932805 (+ 491us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 04:45:01.933006 (+ 201us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 04:45:01.933010 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 04:45:01.933019 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0810 04:45:01.933514 (+ 495us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:45:01.933520 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:45:01.934310 (+ 790us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:45:01.934312 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:45:01.935368 (+ 1056us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:45:01.935372 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:45:01.935848 (+ 476us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:45:01.935849 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:45:01.936002 (+ 153us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:45:01.936562 (+ 560us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 04:45:01.936563 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0810 04:45:01.936600 (+ 37us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 04:45:01.936631 (+ 31us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 04:45:01.936805 (+ 174us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 04:45:01.936810 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 04:45:01.936813 (+ 3us) client_negotiation.cc:769] Sending connection context
0810 04:45:01.936880 (+ 67us) client_negotiation.cc:240] Negotiation successful
0810 04:45:01.936995 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":83,"threads_started":1}
I20260810 04:45:01.937707 20973 heartbeater.cc:347] Connected to a master server at 127.22.167.126:40521
I20260810 04:45:01.937796 20973 heartbeater.cc:464] Registering TS with master...
I20260810 04:45:01.937963 20973 heartbeater.cc:511] Master 127.22.167.126:40521 requested a full tablet report, sending...
I20260810 04:45:01.938505 20641 ts_manager.cc:194] Registered new tserver with Master: dbe5d51105f142bb9a45d0be0fbf3f52 (127.22.167.65:39099)
I20260810 04:45:01.939332 20641 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.22.167.65:50055
W20260810 04:45:02.018604 20980 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 04:45:02.018802 20980 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 04:45:02.018826 20980 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 04:45:02.019882 20980 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 04:45:02.019922 20980 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 04:45:02.019948 20980 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.167.66
I20260810 04:45:02.021219 20980 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.167.84:39297
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786336863388938-23197-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786336863388938-23197-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.167.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786336863388938-23197-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.22.167.66
--webserver_port=0
--tserver_master_addrs=127.22.167.126:40521
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20980
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.167.66
--log_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786336863388938-23197-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 03:54:01 UTC on ccycloud.quasar-icijmt.root.comops.site
build id 4577009
I20260810 04:45:02.021414 20980 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 04:45:02.021668 20980 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 04:45:02.026382 20991 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 04:45:02.026391 20988 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 04:45:02.026450 20987 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 04:45:02.026531 20980 server_base.cc:1034] running on GCE node
I20260810 04:45:02.026789 20980 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 04:45:02.027029 20980 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 04:45:02.028182 20980 hybrid_clock.cc:648] HybridClock initialized: now 1786337102028165 us; error 29 us; skew 500 ppm
I20260810 04:45:02.029357 20980 webserver.cc:460] Webserver started at http://127.22.167.66:38225/ using document root <none> and password file <none>
I20260810 04:45:02.029580 20980 fs_manager.cc:359] Metadata directory not provided
I20260810 04:45:02.029628 20980 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 04:45:02.029758 20980 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 04:45:02.030789 20980 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786336863388938-23197-0/minicluster-data/ts-1/data/instance:
uuid: "73c1b9365985479796d9bdd3b0b4461c"
format_stamp: "Formatted at 2026-08-10 04:45:02 on dist-test-slave-bqcl"
server_key: "6c22421882839733a5c8992dcd1d6a63"
server_key_iv: "102f80a4c6cf12c7aa3292946179b3f0"
server_key_version: "encryptionkey@0"
I20260810 04:45:02.031116 20980 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786336863388938-23197-0/minicluster-data/ts-1/wal/instance:
uuid: "73c1b9365985479796d9bdd3b0b4461c"
format_stamp: "Formatted at 2026-08-10 04:45:02 on dist-test-slave-bqcl"
server_key: "6c22421882839733a5c8992dcd1d6a63"
server_key_iv: "102f80a4c6cf12c7aa3292946179b3f0"
server_key_version: "encryptionkey@0"
I20260810 04:45:02.032336 20980 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260810 04:45:02.032959 21002 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 04:45:02.033113 20980 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260810 04:45:02.033191 20980 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786336863388938-23197-0/minicluster-data/ts-1/data,/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786336863388938-23197-0/minicluster-data/ts-1/wal
uuid: "73c1b9365985479796d9bdd3b0b4461c"
format_stamp: "Formatted at 2026-08-10 04:45:02 on dist-test-slave-bqcl"
server_key: "6c22421882839733a5c8992dcd1d6a63"
server_key_iv: "102f80a4c6cf12c7aa3292946179b3f0"
server_key_version: "encryptionkey@0"
I20260810 04:45:02.033257 20980 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786336863388938-23197-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786336863388938-23197-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786336863388938-23197-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 04:45:02.056463 20980 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 04:45:02.056780 20980 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 04:45:02.056902 20980 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 04:45:02.057114 20980 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 04:45:02.057430 20980 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 04:45:02.057463 20980 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 04:45:02.057492 20980 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 04:45:02.057512 20980 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 04:45:02.062968 20980 rpc_server.cc:307] RPC server started. Bound to: 127.22.167.66:34647
I20260810 04:45:02.063021 21197 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.167.66:34647 every 8 connection(s)
I20260810 04:45:02.063328 20980 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786336863388938-23197-0/minicluster-data/ts-1/data/info.pb
I20260810 04:45:02.068266 20742 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:45:02.063959 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.167.66:52607 (local address 127.22.167.126:40521)
0810 04:45:02.064011 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0810 04:45:02.064014 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 04:45:02.064215 (+ 201us) server_negotiation.cc:407] Connection header received
0810 04:45:02.064742 (+ 527us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:45:02.064744 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:45:02.064756 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:45:02.064805 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0810 04:45:02.065505 (+ 700us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:45:02.066018 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:45:02.067042 (+ 1024us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:45:02.067246 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:45:02.067302 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:45:02.068002 (+ 700us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 04:45:02.068004 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 04:45:02.068004 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 04:45:02.068009 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0810 04:45:02.068027 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 04:45:02.068032 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 04:45:02.068076 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0810 04:45:02.068156 (+ 80us) server_negotiation.cc:299] Negotiation successful
0810 04:45:02.068186 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260810 04:45:02.068439 21201 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:45:02.063945 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.126:40521 (local address 127.22.167.66:52607)
0810 04:45:02.064106 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0810 04:45:02.064139 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0810 04:45:02.064647 (+ 508us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 04:45:02.064881 (+ 234us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 04:45:02.064884 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 04:45:02.064892 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0810 04:45:02.065408 (+ 516us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:45:02.065414 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:45:02.066100 (+ 686us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:45:02.066101 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:45:02.066954 (+ 853us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:45:02.066957 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:45:02.067315 (+ 358us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:45:02.067316 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:45:02.067447 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:45:02.067870 (+ 423us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 04:45:02.067870 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0810 04:45:02.067897 (+ 27us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 04:45:02.067920 (+ 23us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 04:45:02.068083 (+ 163us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 04:45:02.068088 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 04:45:02.068091 (+ 3us) client_negotiation.cc:769] Sending connection context
0810 04:45:02.068150 (+ 59us) client_negotiation.cc:240] Negotiation successful
0810 04:45:02.068247 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":103,"thread_start_us":74,"threads_started":1}
I20260810 04:45:02.068923 21198 heartbeater.cc:347] Connected to a master server at 127.22.167.126:40521
I20260810 04:45:02.069015 21198 heartbeater.cc:464] Registering TS with master...
I20260810 04:45:02.069190 21198 heartbeater.cc:511] Master 127.22.167.126:40521 requested a full tablet report, sending...
I20260810 04:45:02.069603 20641 ts_manager.cc:194] Registered new tserver with Master: 73c1b9365985479796d9bdd3b0b4461c (127.22.167.66:34647)
I20260810 04:45:02.069967 20641 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.22.167.66:52607
I20260810 04:45:02.071493 23197 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskemv_AM/build/release/bin/kudu as pid 20980
I20260810 04:45:02.071578 23197 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786336863388938-23197-0/minicluster-data/ts-1/wal/instance
I20260810 04:45:02.071751 23197 external_mini_cluster.cc:1428] Setting key 46086832a8a9bd198fe2b307e7374049
I20260810 04:45:02.072539 23197 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskemv_AM/build/release/bin/kudu
/tmp/dist-test-taskemv_AM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786336863388938-23197-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786336863388938-23197-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786336863388938-23197-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786336863388938-23197-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.167.67:0
--local_ip_for_outbound_sockets=127.22.167.67
--webserver_interface=127.22.167.67
--webserver_port=0
--tserver_master_addrs=127.22.167.126:40521
--builtin_ntp_servers=127.22.167.84:39297
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260810 04:45:02.153357 21202 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 04:45:02.153548 21202 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 04:45:02.153578 21202 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 04:45:02.154704 21202 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 04:45:02.154743 21202 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 04:45:02.154775 21202 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.167.67
I20260810 04:45:02.156054 21202 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.167.84:39297
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786336863388938-23197-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786336863388938-23197-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.167.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786336863388938-23197-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.22.167.67
--webserver_port=0
--tserver_master_addrs=127.22.167.126:40521
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21202
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.167.67
--log_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786336863388938-23197-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 03:54:01 UTC on ccycloud.quasar-icijmt.root.comops.site
build id 4577009
I20260810 04:45:02.156286 21202 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 04:45:02.156548 21202 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 04:45:02.161168 21216 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 04:45:02.161214 21202 server_base.cc:1034] running on GCE node
W20260810 04:45:02.161216 21209 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 04:45:02.161173 21211 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 04:45:02.161571 21202 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 04:45:02.161805 21202 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 04:45:02.162940 21202 hybrid_clock.cc:648] HybridClock initialized: now 1786337102162921 us; error 31 us; skew 500 ppm
I20260810 04:45:02.164127 21202 webserver.cc:460] Webserver started at http://127.22.167.67:42533/ using document root <none> and password file <none>
I20260810 04:45:02.164353 21202 fs_manager.cc:359] Metadata directory not provided
I20260810 04:45:02.164402 21202 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 04:45:02.164557 21202 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 04:45:02.165678 21202 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786336863388938-23197-0/minicluster-data/ts-2/data/instance:
uuid: "0e7179322de84d3a8f8b2b39fc42fbdc"
format_stamp: "Formatted at 2026-08-10 04:45:02 on dist-test-slave-bqcl"
server_key: "fd74bd9f12b8af7b4d6e9b7da8845577"
server_key_iv: "9d2e3eed9e90b8545750076df8505cbe"
server_key_version: "encryptionkey@0"
I20260810 04:45:02.166019 21202 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786336863388938-23197-0/minicluster-data/ts-2/wal/instance:
uuid: "0e7179322de84d3a8f8b2b39fc42fbdc"
format_stamp: "Formatted at 2026-08-10 04:45:02 on dist-test-slave-bqcl"
server_key: "fd74bd9f12b8af7b4d6e9b7da8845577"
server_key_iv: "9d2e3eed9e90b8545750076df8505cbe"
server_key_version: "encryptionkey@0"
I20260810 04:45:02.167311 21202 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260810 04:45:02.167990 21223 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 04:45:02.168138 21202 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260810 04:45:02.168223 21202 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786336863388938-23197-0/minicluster-data/ts-2/data,/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786336863388938-23197-0/minicluster-data/ts-2/wal
uuid: "0e7179322de84d3a8f8b2b39fc42fbdc"
format_stamp: "Formatted at 2026-08-10 04:45:02 on dist-test-slave-bqcl"
server_key: "fd74bd9f12b8af7b4d6e9b7da8845577"
server_key_iv: "9d2e3eed9e90b8545750076df8505cbe"
server_key_version: "encryptionkey@0"
I20260810 04:45:02.168288 21202 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786336863388938-23197-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786336863388938-23197-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786336863388938-23197-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 04:45:02.183863 21202 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 04:45:02.184124 21202 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 04:45:02.184250 21202 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 04:45:02.184453 21202 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 04:45:02.184760 21202 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 04:45:02.184795 21202 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 04:45:02.184825 21202 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 04:45:02.184849 21202 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 04:45:02.190267 21202 rpc_server.cc:307] RPC server started. Bound to: 127.22.167.67:34745
I20260810 04:45:02.190307 21417 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.167.67:34745 every 8 connection(s)
I20260810 04:45:02.190634 21202 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786336863388938-23197-0/minicluster-data/ts-2/data/info.pb
I20260810 04:45:02.195279 20742 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:45:02.191332 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.167.67:44197 (local address 127.22.167.126:40521)
0810 04:45:02.191380 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0810 04:45:02.191382 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 04:45:02.191595 (+ 213us) server_negotiation.cc:407] Connection header received
0810 04:45:02.192046 (+ 451us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:45:02.192047 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:45:02.192057 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:45:02.192112 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0810 04:45:02.192736 (+ 624us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:45:02.193234 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:45:02.194159 (+ 925us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:45:02.194358 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:45:02.194413 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:45:02.195073 (+ 660us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 04:45:02.195074 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 04:45:02.195074 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 04:45:02.195079 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0810 04:45:02.195093 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 04:45:02.195097 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 04:45:02.195136 (+ 39us) server_negotiation.cc:1035] Waiting for connection context
0810 04:45:02.195193 (+ 57us) server_negotiation.cc:299] Negotiation successful
0810 04:45:02.195218 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260810 04:45:02.195456 21423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:45:02.191302 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.126:40521 (local address 127.22.167.67:44197)
0810 04:45:02.191476 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0810 04:45:02.191516 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0810 04:45:02.191972 (+ 456us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 04:45:02.192167 (+ 195us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 04:45:02.192170 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 04:45:02.192178 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0810 04:45:02.192658 (+ 480us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:45:02.192661 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:45:02.193297 (+ 636us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:45:02.193297 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:45:02.194063 (+ 766us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:45:02.194065 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:45:02.194420 (+ 355us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:45:02.194420 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:45:02.194522 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:45:02.194961 (+ 439us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 04:45:02.194961 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0810 04:45:02.194986 (+ 25us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 04:45:02.195007 (+ 21us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 04:45:02.195141 (+ 134us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 04:45:02.195144 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 04:45:02.195146 (+ 2us) client_negotiation.cc:769] Sending connection context
0810 04:45:02.195189 (+ 43us) client_negotiation.cc:240] Negotiation successful
0810 04:45:02.195263 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":81,"threads_started":1}
I20260810 04:45:02.195894 21421 heartbeater.cc:347] Connected to a master server at 127.22.167.126:40521
I20260810 04:45:02.195976 21421 heartbeater.cc:464] Registering TS with master...
I20260810 04:45:02.196149 21421 heartbeater.cc:511] Master 127.22.167.126:40521 requested a full tablet report, sending...
I20260810 04:45:02.196487 20641 ts_manager.cc:194] Registered new tserver with Master: 0e7179322de84d3a8f8b2b39fc42fbdc (127.22.167.67:34745)
I20260810 04:45:02.196820 20641 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.22.167.67:44197
I20260810 04:45:02.197942 23197 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskemv_AM/build/release/bin/kudu as pid 21202
I20260810 04:45:02.198026 23197 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786336863388938-23197-0/minicluster-data/ts-2/wal/instance
I20260810 04:45:02.198225 23197 external_mini_cluster.cc:1428] Setting key d75e97b5389285516744b15782ae7f5d
I20260810 04:45:02.199033 23197 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260810 04:45:02.202973 20742 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:45:02.200254 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50710 (local address 127.22.167.126:40521)
0810 04:45:02.200289 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0810 04:45:02.200291 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 04:45:02.200441 (+ 150us) server_negotiation.cc:407] Connection header received
0810 04:45:02.200500 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:45:02.200501 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:45:02.200513 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:45:02.200561 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0810 04:45:02.200707 (+ 146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:45:02.201163 (+ 456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:45:02.201915 (+ 752us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:45:02.202118 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:45:02.202194 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:45:02.202762 (+ 568us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 04:45:02.202764 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 04:45:02.202764 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 04:45:02.202768 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0810 04:45:02.202782 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 04:45:02.202786 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 04:45:02.202823 (+ 37us) server_negotiation.cc:1035] Waiting for connection context
0810 04:45:02.202882 (+ 59us) server_negotiation.cc:299] Negotiation successful
0810 04:45:02.202904 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260810 04:45:02.204236 20641 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:50710:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 04:45:02.205651 20641 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 04:45:02.211521 21431 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:45:02.207544 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46744 (local address 127.22.167.67:34745)
0810 04:45:02.207674 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0810 04:45:02.207678 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 04:45:02.207689 (+ 11us) server_negotiation.cc:407] Connection header received
0810 04:45:02.207737 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:45:02.207739 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:45:02.207752 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:45:02.207832 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 04:45:02.208064 (+ 232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:45:02.209130 (+ 1066us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:45:02.210469 (+ 1339us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:45:02.211182 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:45:02.211250 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:45:02.211335 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0810 04:45:02.211429 (+ 94us) server_negotiation.cc:299] Negotiation successful
0810 04:45:02.211457 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":100,"thread_start_us":66,"threads_started":1}
I20260810 04:45:02.211546 21430 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:45:02.207467 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.67:34745 (local address 127.0.0.1:46744)
0810 04:45:02.207577 (+ 110us) negotiation.cc:107] Waiting for socket to connect
0810 04:45:02.207613 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0810 04:45:02.207674 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 04:45:02.207938 (+ 264us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 04:45:02.207941 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 04:45:02.207943 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 04:45:02.208002 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:45:02.208004 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:45:02.209212 (+ 1208us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:45:02.209213 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:45:02.210400 (+ 1187us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:45:02.210403 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:45:02.211262 (+ 859us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:45:02.211263 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:45:02.211339 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:45:02.211352 (+ 13us) client_negotiation.cc:769] Sending connection context
0810 04:45:02.211421 (+ 69us) client_negotiation.cc:240] Negotiation successful
0810 04:45:02.211467 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":89,"thread_start_us":53,"threads_started":1}
I20260810 04:45:02.211880 21432 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:45:02.207663 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.66:34647 (local address 127.0.0.1:46022)
0810 04:45:02.207804 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0810 04:45:02.207814 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0810 04:45:02.207859 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 04:45:02.208355 (+ 496us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 04:45:02.208356 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 04:45:02.208358 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 04:45:02.208399 (+ 41us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:45:02.208400 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:45:02.209509 (+ 1109us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:45:02.209510 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:45:02.210374 (+ 864us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:45:02.210377 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:45:02.211674 (+ 1297us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:45:02.211675 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:45:02.211730 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:45:02.211733 (+ 3us) client_negotiation.cc:769] Sending connection context
0810 04:45:02.211778 (+ 45us) client_negotiation.cc:240] Negotiation successful
0810 04:45:02.211812 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":79,"threads_started":1}
I20260810 04:45:02.212149 21439 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:45:02.208034 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46022 (local address 127.22.167.66:34647)
0810 04:45:02.208206 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0810 04:45:02.208211 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 04:45:02.208220 (+ 9us) server_negotiation.cc:407] Connection header received
0810 04:45:02.208256 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:45:02.208258 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:45:02.208272 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:45:02.208348 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 04:45:02.208451 (+ 103us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:45:02.209450 (+ 999us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:45:02.210461 (+ 1011us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:45:02.211176 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:45:02.211250 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:45:02.211336 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0810 04:45:02.212024 (+ 688us) server_negotiation.cc:299] Negotiation successful
0810 04:45:02.212071 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":134,"thread_start_us":82,"threads_started":1}
I20260810 04:45:02.212276 21298 tablet_service.cc:1467] Processing CreateTablet for tablet 10101be113b64201816c5e509eefa810 (DEFAULT_TABLE table=test-table [id=b5ab37a929cb400da125f123954549a4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 04:45:02.212613 21298 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 10101be113b64201816c5e509eefa810. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 04:45:02.212780 21434 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:45:02.207821 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.65:39099 (local address 127.0.0.1:52800)
0810 04:45:02.208133 (+ 312us) negotiation.cc:107] Waiting for socket to connect
0810 04:45:02.208142 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0810 04:45:02.208192 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 04:45:02.209421 (+ 1229us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 04:45:02.209422 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 04:45:02.209424 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 04:45:02.209459 (+ 35us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:45:02.209460 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:45:02.210704 (+ 1244us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:45:02.210705 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:45:02.211458 (+ 753us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:45:02.211461 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:45:02.212544 (+ 1083us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:45:02.212546 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:45:02.212607 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:45:02.212611 (+ 4us) client_negotiation.cc:769] Sending connection context
0810 04:45:02.212677 (+ 66us) client_negotiation.cc:240] Negotiation successful
0810 04:45:02.212716 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":296,"thread_start_us":62,"threads_started":1}
I20260810 04:45:02.212823 21437 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:45:02.207931 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52800 (local address 127.22.167.65:39099)
0810 04:45:02.209269 (+ 1338us) server_negotiation.cc:206] Beginning negotiation
0810 04:45:02.209274 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 04:45:02.209283 (+ 9us) server_negotiation.cc:407] Connection header received
0810 04:45:02.209319 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:45:02.209321 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:45:02.209335 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:45:02.209404 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 04:45:02.209579 (+ 175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:45:02.210640 (+ 1061us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:45:02.211537 (+ 897us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:45:02.212354 (+ 817us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:45:02.212419 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:45:02.212507 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0810 04:45:02.212704 (+ 197us) server_negotiation.cc:299] Negotiation successful
0810 04:45:02.212748 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1302,"thread_start_us":103,"threads_started":1}
I20260810 04:45:02.212808 21081 tablet_service.cc:1467] Processing CreateTablet for tablet 10101be113b64201816c5e509eefa810 (DEFAULT_TABLE table=test-table [id=b5ab37a929cb400da125f123954549a4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 04:45:02.213105 21081 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 10101be113b64201816c5e509eefa810. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 04:45:02.215123 21440 tablet_bootstrap.cc:492] T 10101be113b64201816c5e509eefa810 P 0e7179322de84d3a8f8b2b39fc42fbdc: Bootstrap starting.
I20260810 04:45:02.215658 21441 tablet_bootstrap.cc:492] T 10101be113b64201816c5e509eefa810 P 73c1b9365985479796d9bdd3b0b4461c: Bootstrap starting.
I20260810 04:45:02.215785 21440 tablet_bootstrap.cc:654] T 10101be113b64201816c5e509eefa810 P 0e7179322de84d3a8f8b2b39fc42fbdc: Neither blocks nor log segments found. Creating new log.
I20260810 04:45:02.215679 20851 tablet_service.cc:1467] Processing CreateTablet for tablet 10101be113b64201816c5e509eefa810 (DEFAULT_TABLE table=test-table [id=b5ab37a929cb400da125f123954549a4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 04:45:02.216019 20851 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 10101be113b64201816c5e509eefa810. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 04:45:02.216099 21440 log.cc:824] T 10101be113b64201816c5e509eefa810 P 0e7179322de84d3a8f8b2b39fc42fbdc: Log is configured to *not* fsync() on all Append() calls
I20260810 04:45:02.216404 21441 tablet_bootstrap.cc:654] T 10101be113b64201816c5e509eefa810 P 73c1b9365985479796d9bdd3b0b4461c: Neither blocks nor log segments found. Creating new log.
I20260810 04:45:02.216727 21441 log.cc:824] T 10101be113b64201816c5e509eefa810 P 73c1b9365985479796d9bdd3b0b4461c: Log is configured to *not* fsync() on all Append() calls
I20260810 04:45:02.216823 21440 tablet_bootstrap.cc:492] T 10101be113b64201816c5e509eefa810 P 0e7179322de84d3a8f8b2b39fc42fbdc: No bootstrap required, opened a new log
I20260810 04:45:02.216892 21440 ts_tablet_manager.cc:1397] T 10101be113b64201816c5e509eefa810 P 0e7179322de84d3a8f8b2b39fc42fbdc: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260810 04:45:02.217522 21441 tablet_bootstrap.cc:492] T 10101be113b64201816c5e509eefa810 P 73c1b9365985479796d9bdd3b0b4461c: No bootstrap required, opened a new log
I20260810 04:45:02.217594 21441 ts_tablet_manager.cc:1397] T 10101be113b64201816c5e509eefa810 P 73c1b9365985479796d9bdd3b0b4461c: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260810 04:45:02.218678 21448 tablet_bootstrap.cc:492] T 10101be113b64201816c5e509eefa810 P dbe5d51105f142bb9a45d0be0fbf3f52: Bootstrap starting.
I20260810 04:45:02.219231 21448 tablet_bootstrap.cc:654] T 10101be113b64201816c5e509eefa810 P dbe5d51105f142bb9a45d0be0fbf3f52: Neither blocks nor log segments found. Creating new log.
I20260810 04:45:02.219503 21448 log.cc:824] T 10101be113b64201816c5e509eefa810 P dbe5d51105f142bb9a45d0be0fbf3f52: Log is configured to *not* fsync() on all Append() calls
I20260810 04:45:02.219614 21440 raft_consensus.cc:348] T 10101be113b64201816c5e509eefa810 P 0e7179322de84d3a8f8b2b39fc42fbdc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e7179322de84d3a8f8b2b39fc42fbdc" member_type: VOTER last_known_addr { host: "127.22.167.67" port: 34745 } } peers { permanent_uuid: "73c1b9365985479796d9bdd3b0b4461c" member_type: VOTER last_known_addr { host: "127.22.167.66" port: 34647 } } peers { permanent_uuid: "dbe5d51105f142bb9a45d0be0fbf3f52" member_type: VOTER last_known_addr { host: "127.22.167.65" port: 39099 } }
I20260810 04:45:02.219789 21440 raft_consensus.cc:374] T 10101be113b64201816c5e509eefa810 P 0e7179322de84d3a8f8b2b39fc42fbdc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 04:45:02.219830 21440 raft_consensus.cc:729] T 10101be113b64201816c5e509eefa810 P 0e7179322de84d3a8f8b2b39fc42fbdc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0e7179322de84d3a8f8b2b39fc42fbdc, State: Initialized, Role: FOLLOWER
I20260810 04:45:02.219944 21440 consensus_queue.cc:260] T 10101be113b64201816c5e509eefa810 P 0e7179322de84d3a8f8b2b39fc42fbdc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e7179322de84d3a8f8b2b39fc42fbdc" member_type: VOTER last_known_addr { host: "127.22.167.67" port: 34745 } } peers { permanent_uuid: "73c1b9365985479796d9bdd3b0b4461c" member_type: VOTER last_known_addr { host: "127.22.167.66" port: 34647 } } peers { permanent_uuid: "dbe5d51105f142bb9a45d0be0fbf3f52" member_type: VOTER last_known_addr { host: "127.22.167.65" port: 39099 } }
I20260810 04:45:02.220151 21448 tablet_bootstrap.cc:492] T 10101be113b64201816c5e509eefa810 P dbe5d51105f142bb9a45d0be0fbf3f52: No bootstrap required, opened a new log
I20260810 04:45:02.220091 21441 raft_consensus.cc:348] T 10101be113b64201816c5e509eefa810 P 73c1b9365985479796d9bdd3b0b4461c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e7179322de84d3a8f8b2b39fc42fbdc" member_type: VOTER last_known_addr { host: "127.22.167.67" port: 34745 } } peers { permanent_uuid: "73c1b9365985479796d9bdd3b0b4461c" member_type: VOTER last_known_addr { host: "127.22.167.66" port: 34647 } } peers { permanent_uuid: "dbe5d51105f142bb9a45d0be0fbf3f52" member_type: VOTER last_known_addr { host: "127.22.167.65" port: 39099 } }
I20260810 04:45:02.220213 21448 ts_tablet_manager.cc:1397] T 10101be113b64201816c5e509eefa810 P dbe5d51105f142bb9a45d0be0fbf3f52: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260810 04:45:02.220232 21440 ts_tablet_manager.cc:1428] T 10101be113b64201816c5e509eefa810 P 0e7179322de84d3a8f8b2b39fc42fbdc: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260810 04:45:02.220247 21441 raft_consensus.cc:374] T 10101be113b64201816c5e509eefa810 P 73c1b9365985479796d9bdd3b0b4461c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 04:45:02.220283 21441 raft_consensus.cc:729] T 10101be113b64201816c5e509eefa810 P 73c1b9365985479796d9bdd3b0b4461c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 73c1b9365985479796d9bdd3b0b4461c, State: Initialized, Role: FOLLOWER
I20260810 04:45:02.220350 21421 heartbeater.cc:503] Master 127.22.167.126:40521 was elected leader, sending a full tablet report...
I20260810 04:45:02.220392 21441 consensus_queue.cc:260] T 10101be113b64201816c5e509eefa810 P 73c1b9365985479796d9bdd3b0b4461c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e7179322de84d3a8f8b2b39fc42fbdc" member_type: VOTER last_known_addr { host: "127.22.167.67" port: 34745 } } peers { permanent_uuid: "73c1b9365985479796d9bdd3b0b4461c" member_type: VOTER last_known_addr { host: "127.22.167.66" port: 34647 } } peers { permanent_uuid: "dbe5d51105f142bb9a45d0be0fbf3f52" member_type: VOTER last_known_addr { host: "127.22.167.65" port: 39099 } }
I20260810 04:45:02.220659 21441 ts_tablet_manager.cc:1428] T 10101be113b64201816c5e509eefa810 P 73c1b9365985479796d9bdd3b0b4461c: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260810 04:45:02.221009 21198 heartbeater.cc:503] Master 127.22.167.126:40521 was elected leader, sending a full tablet report...
I20260810 04:45:02.222312 21448 raft_consensus.cc:348] T 10101be113b64201816c5e509eefa810 P dbe5d51105f142bb9a45d0be0fbf3f52 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e7179322de84d3a8f8b2b39fc42fbdc" member_type: VOTER last_known_addr { host: "127.22.167.67" port: 34745 } } peers { permanent_uuid: "73c1b9365985479796d9bdd3b0b4461c" member_type: VOTER last_known_addr { host: "127.22.167.66" port: 34647 } } peers { permanent_uuid: "dbe5d51105f142bb9a45d0be0fbf3f52" member_type: VOTER last_known_addr { host: "127.22.167.65" port: 39099 } }
I20260810 04:45:02.222426 21448 raft_consensus.cc:374] T 10101be113b64201816c5e509eefa810 P dbe5d51105f142bb9a45d0be0fbf3f52 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 04:45:02.222457 21448 raft_consensus.cc:729] T 10101be113b64201816c5e509eefa810 P dbe5d51105f142bb9a45d0be0fbf3f52 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dbe5d51105f142bb9a45d0be0fbf3f52, State: Initialized, Role: FOLLOWER
I20260810 04:45:02.222560 21448 consensus_queue.cc:260] T 10101be113b64201816c5e509eefa810 P dbe5d51105f142bb9a45d0be0fbf3f52 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e7179322de84d3a8f8b2b39fc42fbdc" member_type: VOTER last_known_addr { host: "127.22.167.67" port: 34745 } } peers { permanent_uuid: "73c1b9365985479796d9bdd3b0b4461c" member_type: VOTER last_known_addr { host: "127.22.167.66" port: 34647 } } peers { permanent_uuid: "dbe5d51105f142bb9a45d0be0fbf3f52" member_type: VOTER last_known_addr { host: "127.22.167.65" port: 39099 } }
I20260810 04:45:02.222765 21448 ts_tablet_manager.cc:1428] T 10101be113b64201816c5e509eefa810 P dbe5d51105f142bb9a45d0be0fbf3f52: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260810 04:45:02.222833 20973 heartbeater.cc:503] Master 127.22.167.126:40521 was elected leader, sending a full tablet report...
W20260810 04:45:02.313869 21199 tablet.cc:2369] T 10101be113b64201816c5e509eefa810 P 73c1b9365985479796d9bdd3b0b4461c: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 04:45:02.321033 21453 raft_consensus.cc:482] T 10101be113b64201816c5e509eefa810 P dbe5d51105f142bb9a45d0be0fbf3f52 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 04:45:02.321158 21453 raft_consensus.cc:504] T 10101be113b64201816c5e509eefa810 P dbe5d51105f142bb9a45d0be0fbf3f52 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e7179322de84d3a8f8b2b39fc42fbdc" member_type: VOTER last_known_addr { host: "127.22.167.67" port: 34745 } } peers { permanent_uuid: "73c1b9365985479796d9bdd3b0b4461c" member_type: VOTER last_known_addr { host: "127.22.167.66" port: 34647 } } peers { permanent_uuid: "dbe5d51105f142bb9a45d0be0fbf3f52" member_type: VOTER last_known_addr { host: "127.22.167.65" port: 39099 } }
I20260810 04:45:02.321475 21453 leader_election.cc:288] T 10101be113b64201816c5e509eefa810 P dbe5d51105f142bb9a45d0be0fbf3f52 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0e7179322de84d3a8f8b2b39fc42fbdc (127.22.167.67:34745), 73c1b9365985479796d9bdd3b0b4461c (127.22.167.66:34647)
I20260810 04:45:02.324371 20979 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:45:02.321674 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.67:34745 (local address 127.22.167.65:36937)
0810 04:45:02.321727 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0810 04:45:02.321740 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0810 04:45:02.321788 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 04:45:02.321952 (+ 164us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 04:45:02.321953 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 04:45:02.321955 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 04:45:02.322025 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:45:02.322027 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:45:02.322777 (+ 750us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:45:02.322778 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:45:02.323561 (+ 783us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:45:02.323563 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:45:02.324131 (+ 568us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:45:02.324133 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:45:02.324201 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:45:02.324204 (+ 3us) client_negotiation.cc:769] Sending connection context
0810 04:45:02.324258 (+ 54us) client_negotiation.cc:240] Negotiation successful
0810 04:45:02.324297 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":30}
I20260810 04:45:02.324380 21431 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:45:02.321765 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.167.65:36937 (local address 127.22.167.67:34745)
0810 04:45:02.321814 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0810 04:45:02.321816 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 04:45:02.321826 (+ 10us) server_negotiation.cc:407] Connection header received
0810 04:45:02.321859 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:45:02.321860 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:45:02.321869 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:45:02.321939 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 04:45:02.322116 (+ 177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:45:02.322701 (+ 585us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:45:02.323660 (+ 959us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:45:02.324056 (+ 396us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:45:02.324111 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:45:02.324118 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0810 04:45:02.324266 (+ 148us) server_negotiation.cc:299] Negotiation successful
0810 04:45:02.324311 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260810 04:45:02.324456 21439 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:45:02.321867 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.167.65:45515 (local address 127.22.167.66:34647)
0810 04:45:02.321931 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0810 04:45:02.321933 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 04:45:02.322067 (+ 134us) server_negotiation.cc:407] Connection header received
0810 04:45:02.322106 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:45:02.322107 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:45:02.322116 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:45:02.322198 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 04:45:02.322348 (+ 150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:45:02.322805 (+ 457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:45:02.323736 (+ 931us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:45:02.324092 (+ 356us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:45:02.324180 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:45:02.324189 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0810 04:45:02.324354 (+ 165us) server_negotiation.cc:299] Negotiation successful
0810 04:45:02.324399 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260810 04:45:02.324656 21457 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:45:02.321786 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.66:34647 (local address 127.22.167.65:45515)
0810 04:45:02.322002 (+ 216us) negotiation.cc:107] Waiting for socket to connect
0810 04:45:02.322009 (+ 7us) client_negotiation.cc:174] Beginning negotiation
0810 04:45:02.322061 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 04:45:02.322209 (+ 148us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 04:45:02.322210 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 04:45:02.322211 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 04:45:02.322282 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:45:02.322284 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:45:02.322895 (+ 611us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:45:02.322897 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:45:02.323653 (+ 756us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:45:02.323655 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:45:02.324194 (+ 539us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:45:02.324195 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:45:02.324271 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:45:02.324275 (+ 4us) client_negotiation.cc:769] Sending connection context
0810 04:45:02.324525 (+ 250us) client_negotiation.cc:240] Negotiation successful
0810 04:45:02.324573 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":201,"thread_start_us":134,"threads_started":1}
I20260810 04:45:02.324783 21327 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "10101be113b64201816c5e509eefa810" candidate_uuid: "dbe5d51105f142bb9a45d0be0fbf3f52" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e7179322de84d3a8f8b2b39fc42fbdc" is_pre_election: true
I20260810 04:45:02.324951 21327 raft_consensus.cc:2454] T 10101be113b64201816c5e509eefa810 P 0e7179322de84d3a8f8b2b39fc42fbdc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dbe5d51105f142bb9a45d0be0fbf3f52 in term 0.
I20260810 04:45:02.324990 21114 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "10101be113b64201816c5e509eefa810" candidate_uuid: "dbe5d51105f142bb9a45d0be0fbf3f52" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "73c1b9365985479796d9bdd3b0b4461c" is_pre_election: true
I20260810 04:45:02.325157 21114 raft_consensus.cc:2454] T 10101be113b64201816c5e509eefa810 P 73c1b9365985479796d9bdd3b0b4461c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dbe5d51105f142bb9a45d0be0fbf3f52 in term 0.
I20260810 04:45:02.325248 20772 leader_election.cc:302] T 10101be113b64201816c5e509eefa810 P dbe5d51105f142bb9a45d0be0fbf3f52 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e7179322de84d3a8f8b2b39fc42fbdc, dbe5d51105f142bb9a45d0be0fbf3f52; no voters:
I20260810 04:45:02.325390 21453 raft_consensus.cc:2781] T 10101be113b64201816c5e509eefa810 P dbe5d51105f142bb9a45d0be0fbf3f52 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 04:45:02.325456 21453 raft_consensus.cc:482] T 10101be113b64201816c5e509eefa810 P dbe5d51105f142bb9a45d0be0fbf3f52 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 04:45:02.325471 21453 raft_consensus.cc:3037] T 10101be113b64201816c5e509eefa810 P dbe5d51105f142bb9a45d0be0fbf3f52 [term 0 FOLLOWER]: Advancing to term 1
I20260810 04:45:02.326218 21453 raft_consensus.cc:504] T 10101be113b64201816c5e509eefa810 P dbe5d51105f142bb9a45d0be0fbf3f52 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e7179322de84d3a8f8b2b39fc42fbdc" member_type: VOTER last_known_addr { host: "127.22.167.67" port: 34745 } } peers { permanent_uuid: "73c1b9365985479796d9bdd3b0b4461c" member_type: VOTER last_known_addr { host: "127.22.167.66" port: 34647 } } peers { permanent_uuid: "dbe5d51105f142bb9a45d0be0fbf3f52" member_type: VOTER last_known_addr { host: "127.22.167.65" port: 39099 } }
I20260810 04:45:02.326333 21453 leader_election.cc:288] T 10101be113b64201816c5e509eefa810 P dbe5d51105f142bb9a45d0be0fbf3f52 [CANDIDATE]: Term 1 election: Requested vote from peers 0e7179322de84d3a8f8b2b39fc42fbdc (127.22.167.67:34745), 73c1b9365985479796d9bdd3b0b4461c (127.22.167.66:34647)
I20260810 04:45:02.326475 21327 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "10101be113b64201816c5e509eefa810" candidate_uuid: "dbe5d51105f142bb9a45d0be0fbf3f52" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e7179322de84d3a8f8b2b39fc42fbdc"
I20260810 04:45:02.326495 21114 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "10101be113b64201816c5e509eefa810" candidate_uuid: "dbe5d51105f142bb9a45d0be0fbf3f52" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "73c1b9365985479796d9bdd3b0b4461c"
I20260810 04:45:02.326555 21327 raft_consensus.cc:3037] T 10101be113b64201816c5e509eefa810 P 0e7179322de84d3a8f8b2b39fc42fbdc [term 0 FOLLOWER]: Advancing to term 1
I20260810 04:45:02.326555 21114 raft_consensus.cc:3037] T 10101be113b64201816c5e509eefa810 P 73c1b9365985479796d9bdd3b0b4461c [term 0 FOLLOWER]: Advancing to term 1
I20260810 04:45:02.327265 21114 raft_consensus.cc:2454] T 10101be113b64201816c5e509eefa810 P 73c1b9365985479796d9bdd3b0b4461c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dbe5d51105f142bb9a45d0be0fbf3f52 in term 1.
I20260810 04:45:02.327278 21327 raft_consensus.cc:2454] T 10101be113b64201816c5e509eefa810 P 0e7179322de84d3a8f8b2b39fc42fbdc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dbe5d51105f142bb9a45d0be0fbf3f52 in term 1.
I20260810 04:45:02.327407 20772 leader_election.cc:302] T 10101be113b64201816c5e509eefa810 P dbe5d51105f142bb9a45d0be0fbf3f52 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 73c1b9365985479796d9bdd3b0b4461c, dbe5d51105f142bb9a45d0be0fbf3f52; no voters:
I20260810 04:45:02.327508 21453 raft_consensus.cc:2781] T 10101be113b64201816c5e509eefa810 P dbe5d51105f142bb9a45d0be0fbf3f52 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 04:45:02.327656 21453 raft_consensus.cc:686] T 10101be113b64201816c5e509eefa810 P dbe5d51105f142bb9a45d0be0fbf3f52 [term 1 LEADER]: Becoming Leader. State: Replica: dbe5d51105f142bb9a45d0be0fbf3f52, State: Running, Role: LEADER
I20260810 04:45:02.327764 21453 consensus_queue.cc:237] T 10101be113b64201816c5e509eefa810 P dbe5d51105f142bb9a45d0be0fbf3f52 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e7179322de84d3a8f8b2b39fc42fbdc" member_type: VOTER last_known_addr { host: "127.22.167.67" port: 34745 } } peers { permanent_uuid: "73c1b9365985479796d9bdd3b0b4461c" member_type: VOTER last_known_addr { host: "127.22.167.66" port: 34647 } } peers { permanent_uuid: "dbe5d51105f142bb9a45d0be0fbf3f52" member_type: VOTER last_known_addr { host: "127.22.167.65" port: 39099 } }
I20260810 04:45:02.328569 20638 catalog_manager.cc:5495] T 10101be113b64201816c5e509eefa810 P dbe5d51105f142bb9a45d0be0fbf3f52 reported cstate change: term changed from 0 to 1, leader changed from <none> to dbe5d51105f142bb9a45d0be0fbf3f52 (127.22.167.65). New cstate: current_term: 1 leader_uuid: "dbe5d51105f142bb9a45d0be0fbf3f52" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e7179322de84d3a8f8b2b39fc42fbdc" member_type: VOTER last_known_addr { host: "127.22.167.67" port: 34745 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "73c1b9365985479796d9bdd3b0b4461c" member_type: VOTER last_known_addr { host: "127.22.167.66" port: 34647 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dbe5d51105f142bb9a45d0be0fbf3f52" member_type: VOTER last_known_addr { host: "127.22.167.65" port: 39099 } health_report { overall_health: HEALTHY } } }
I20260810 04:45:02.357410 21437 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:45:02.355312 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52814 (local address 127.22.167.65:39099)
0810 04:45:02.355364 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0810 04:45:02.355366 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 04:45:02.355374 (+ 8us) server_negotiation.cc:407] Connection header received
0810 04:45:02.355410 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:45:02.355411 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:45:02.355429 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:45:02.355491 (+ 62us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 04:45:02.355605 (+ 114us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:45:02.356066 (+ 461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:45:02.356768 (+ 702us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:45:02.356940 (+ 172us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:45:02.356980 (+ 40us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:45:02.357077 (+ 97us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 04:45:02.357188 (+ 111us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 04:45:02.357242 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0810 04:45:02.357314 (+ 72us) server_negotiation.cc:299] Negotiation successful
0810 04:45:02.357333 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260810 04:45:02.359295 21327 raft_consensus.cc:1261] T 10101be113b64201816c5e509eefa810 P 0e7179322de84d3a8f8b2b39fc42fbdc [term 1 FOLLOWER]: Refusing update from remote peer dbe5d51105f142bb9a45d0be0fbf3f52: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 04:45:02.359319 21114 raft_consensus.cc:1261] T 10101be113b64201816c5e509eefa810 P 73c1b9365985479796d9bdd3b0b4461c [term 1 FOLLOWER]: Refusing update from remote peer dbe5d51105f142bb9a45d0be0fbf3f52: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 04:45:02.359690 21458 consensus_queue.cc:1035] T 10101be113b64201816c5e509eefa810 P dbe5d51105f142bb9a45d0be0fbf3f52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e7179322de84d3a8f8b2b39fc42fbdc" member_type: VOTER last_known_addr { host: "127.22.167.67" port: 34745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 04:45:02.359823 21453 consensus_queue.cc:1035] T 10101be113b64201816c5e509eefa810 P dbe5d51105f142bb9a45d0be0fbf3f52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "73c1b9365985479796d9bdd3b0b4461c" member_type: VOTER last_known_addr { host: "127.22.167.66" port: 34647 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 04:45:02.363041 21463 mvcc.cc:205] Tried to move back new op lower bound from 7316836771260801024 to 7316836771135115264. Current Snapshot: MvccSnapshot[applied={T|T < 7316836771260801024}]
I20260810 04:45:02.364816 20638 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:50710:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 04:45:02.364914 20638 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:50710:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 04:45:02.365656 20638 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 4770ad600718452a9358497e0e772fee: Sending DeleteTablet for 3 replicas of tablet 10101be113b64201816c5e509eefa810
I20260810 04:45:02.366016 21298 tablet_service.cc:1514] Processing DeleteTablet for tablet 10101be113b64201816c5e509eefa810 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 04:45:02 UTC) from {username='slave'} at 127.0.0.1:46744
I20260810 04:45:02.366072 21081 tablet_service.cc:1514] Processing DeleteTablet for tablet 10101be113b64201816c5e509eefa810 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 04:45:02 UTC) from {username='slave'} at 127.0.0.1:46022
I20260810 04:45:02.366209 20851 tablet_service.cc:1514] Processing DeleteTablet for tablet 10101be113b64201816c5e509eefa810 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 04:45:02 UTC) from {username='slave'} at 127.0.0.1:52800
I20260810 04:45:02.366446 21479 tablet_replica.cc:330] T 10101be113b64201816c5e509eefa810 P 73c1b9365985479796d9bdd3b0b4461c: stopping tablet replica
I20260810 04:45:02.366451 23197 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskemv_AM/build/release/bin/kudu with pid 20744
I20260810 04:45:02.366503 21481 tablet_replica.cc:330] T 10101be113b64201816c5e509eefa810 P dbe5d51105f142bb9a45d0be0fbf3f52: stopping tablet replica
I20260810 04:45:02.366606 21481 raft_consensus.cc:2229] T 10101be113b64201816c5e509eefa810 P dbe5d51105f142bb9a45d0be0fbf3f52 [term 1 LEADER]: Raft consensus shutting down.
I20260810 04:45:02.366691 21479 raft_consensus.cc:2229] T 10101be113b64201816c5e509eefa810 P 73c1b9365985479796d9bdd3b0b4461c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 04:45:02.366729 21481 raft_consensus.cc:2258] T 10101be113b64201816c5e509eefa810 P dbe5d51105f142bb9a45d0be0fbf3f52 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 04:45:02.366981 21477 tablet_replica.cc:330] T 10101be113b64201816c5e509eefa810 P 0e7179322de84d3a8f8b2b39fc42fbdc: stopping tablet replica
I20260810 04:45:02.367071 21477 raft_consensus.cc:2229] T 10101be113b64201816c5e509eefa810 P 0e7179322de84d3a8f8b2b39fc42fbdc [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 04:45:02.367177 21477 raft_consensus.cc:2258] T 10101be113b64201816c5e509eefa810 P 0e7179322de84d3a8f8b2b39fc42fbdc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 04:45:02.369284 21479 raft_consensus.cc:2258] T 10101be113b64201816c5e509eefa810 P 73c1b9365985479796d9bdd3b0b4461c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 04:45:02.372584 21477 ts_tablet_manager.cc:1905] T 10101be113b64201816c5e509eefa810 P 0e7179322de84d3a8f8b2b39fc42fbdc: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 04:45:02.373910 21477 ts_tablet_manager.cc:1918] T 10101be113b64201816c5e509eefa810 P 0e7179322de84d3a8f8b2b39fc42fbdc: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 04:45:02.373979 21477 log.cc:1196] T 10101be113b64201816c5e509eefa810 P 0e7179322de84d3a8f8b2b39fc42fbdc: Deleting WAL directory at /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786336863388938-23197-0/minicluster-data/ts-2/wal/wals/10101be113b64201816c5e509eefa810
I20260810 04:45:02.374243 21477 ts_tablet_manager.cc:1939] T 10101be113b64201816c5e509eefa810 P 0e7179322de84d3a8f8b2b39fc42fbdc: Deleting consensus metadata
I20260810 04:45:02.374611 20617 catalog_manager.cc:4841] TS 0e7179322de84d3a8f8b2b39fc42fbdc (127.22.167.67:34745): tablet 10101be113b64201816c5e509eefa810 (table test-table [id=b5ab37a929cb400da125f123954549a4]) successfully deleted
I20260810 04:45:02.377946 21479 ts_tablet_manager.cc:1905] T 10101be113b64201816c5e509eefa810 P 73c1b9365985479796d9bdd3b0b4461c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 04:45:02.379213 21479 ts_tablet_manager.cc:1918] T 10101be113b64201816c5e509eefa810 P 73c1b9365985479796d9bdd3b0b4461c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 04:45:02.379284 21479 log.cc:1196] T 10101be113b64201816c5e509eefa810 P 73c1b9365985479796d9bdd3b0b4461c: Deleting WAL directory at /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786336863388938-23197-0/minicluster-data/ts-1/wal/wals/10101be113b64201816c5e509eefa810
I20260810 04:45:02.379480 21479 ts_tablet_manager.cc:1939] T 10101be113b64201816c5e509eefa810 P 73c1b9365985479796d9bdd3b0b4461c: Deleting consensus metadata
I20260810 04:45:02.379869 20617 catalog_manager.cc:4841] TS 73c1b9365985479796d9bdd3b0b4461c (127.22.167.66:34647): tablet 10101be113b64201816c5e509eefa810 (table test-table [id=b5ab37a929cb400da125f123954549a4]) successfully deleted
W20260810 04:45:02.381953 20617 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.22.167.65:39099 (error 108)
I20260810 04:45:02.382462 21434 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:45:02.382351 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.65:39099 (local address 127.0.0.1:52830)
0810 04:45:02.382396 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0810 04:45:02.382433 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.167.65:39099: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
I20260810 04:45:02.382579 23197 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskemv_AM/build/release/bin/kudu with pid 20980
W20260810 04:45:02.382576 20617 catalog_manager.cc:4584] TS dbe5d51105f142bb9a45d0be0fbf3f52 (127.22.167.65:39099): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 10101be113b64201816c5e509eefa810: Network error: Client connection negotiation failed: client connection to 127.22.167.65:39099: connect: Connection refused (error 111)
I20260810 04:45:02.389993 23197 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskemv_AM/build/release/bin/kudu with pid 21202
I20260810 04:45:02.402820 23197 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskemv_AM/build/release/bin/kudu with pid 20595
2026-08-10T04:45:02Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/1 (793 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/2
Loading random data
Initializing database '/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786336863388938-23197-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 04:45:02 dist-test-slave-bqcl krb5kdc[21489](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 04:45:02 dist-test-slave-bqcl krb5kdc[21489](info): set up 2 sockets
Aug 10 04:45:02 dist-test-slave-bqcl krb5kdc[21489](info): commencing operation
krb5kdc: starting...
W20260810 04:45:03.556815 23197 mini_kdc.cc:121] Time spent starting KDC: real 1.129s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786336863388938-23197-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786336863388938-23197-0/krb5kdc/test-user.keytab.
Aug 10 04:45:03 dist-test-slave-bqcl krb5kdc[21489](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786337103, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T04:45:03Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T04:45:03Z Disabled control of system clock
WARNING: no policy specified for kudu/127.22.167.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.167.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.167.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786336863388938-23197-0/krb5kdc/kudu_127.22.167.126.keytab.
Entry for principal kudu/127.22.167.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786336863388938-23197-0/krb5kdc/kudu_127.22.167.126.keytab.
I20260810 04:45:03.711467 23197 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskemv_AM/build/release/bin/kudu
/tmp/dist-test-taskemv_AM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786336863388938-23197-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786336863388938-23197-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786336863388938-23197-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786336863388938-23197-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.167.126:36491
--webserver_interface=127.22.167.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786336863388938-23197-0/krb5kdc/kudu_127.22.167.126.keytab
--principal=kudu/127.22.167.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.167.84:43213
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.167.126:36491
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786336863388938-23197-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786336863388938-23197-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786336863388938-23197-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 04:45:03.854621 21517 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 04:45:03.854965 21517 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 04:45:03.855000 21517 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 04:45:03.856873 21517 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 04:45:03.856941 21517 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 04:45:03.856979 21517 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 04:45:03.857031 21517 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 04:45:03.857071 21517 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 04:45:03.859581 21517 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.167.84:43213
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786336863388938-23197-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786336863388938-23197-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.22.167.126:36491
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786336863388938-23197-0/krb5kdc/kudu_127.22.167.126.keytab
--principal=kudu/127.22.167.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.167.126:36491
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786336863388938-23197-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.167.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21517
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786336863388938-23197-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 03:54:01 UTC on ccycloud.quasar-icijmt.root.comops.site
build id 4577009
I20260810 04:45:03.859977 21517 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 04:45:03.860392 21517 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 04:45:03.868754 21531 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 04:45:03.868784 21534 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 04:45:03.868786 21538 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 04:45:03.870276 21517 server_base.cc:1034] running on GCE node
I20260810 04:45:03.870707 21517 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 04:45:03.871232 21517 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 04:45:03.872437 21517 hybrid_clock.cc:648] HybridClock initialized: now 1786337103872413 us; error 47 us; skew 500 ppm
Aug 10 04:45:03 dist-test-slave-bqcl krb5kdc[21489](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786337103, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.167.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 04:45:03.876869 21517 init.cc:377] Logged in from keytab as kudu/127.22.167.126@KRBTEST.COM (short username kudu)
I20260810 04:45:03.877710 21517 webserver.cc:460] Webserver started at http://127.22.167.126:39479/ using document root <none> and password file <none>
I20260810 04:45:03.878106 21517 fs_manager.cc:359] Metadata directory not provided
I20260810 04:45:03.878291 21517 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 04:45:03.878556 21517 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 04:45:03.880106 21517 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786336863388938-23197-0/minicluster-data/master-0/data/instance:
uuid: "e4a08e9a41264911b0100ad613d923a7"
format_stamp: "Formatted at 2026-08-10 04:45:03 on dist-test-slave-bqcl"
server_key: "c38f8207ad006ee14eaa3032627ec090"
server_key_iv: "592aee5b5f6c2cf56d03f863c01c7325"
server_key_version: "encryptionkey@0"
I20260810 04:45:03.880709 21517 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786336863388938-23197-0/minicluster-data/master-0/wal/instance:
uuid: "e4a08e9a41264911b0100ad613d923a7"
format_stamp: "Formatted at 2026-08-10 04:45:03 on dist-test-slave-bqcl"
server_key: "c38f8207ad006ee14eaa3032627ec090"
server_key_iv: "592aee5b5f6c2cf56d03f863c01c7325"
server_key_version: "encryptionkey@0"
I20260810 04:45:03.883093 21517 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.005s sys 0.000s
I20260810 04:45:03.884392 21553 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 04:45:03.884651 21517 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 04:45:03.884783 21517 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786336863388938-23197-0/minicluster-data/master-0/data,/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786336863388938-23197-0/minicluster-data/master-0/wal
uuid: "e4a08e9a41264911b0100ad613d923a7"
format_stamp: "Formatted at 2026-08-10 04:45:03 on dist-test-slave-bqcl"
server_key: "c38f8207ad006ee14eaa3032627ec090"
server_key_iv: "592aee5b5f6c2cf56d03f863c01c7325"
server_key_version: "encryptionkey@0"
I20260810 04:45:03.884869 21517 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786336863388938-23197-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786336863388938-23197-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786336863388938-23197-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 04:45:03.898729 21517 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 04:45:03.901782 21517 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 04:45:03.901921 21517 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 04:45:03.906251 21517 rpc_server.cc:307] RPC server started. Bound to: 127.22.167.126:36491
I20260810 04:45:03.906291 21643 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.167.126:36491 every 8 connection(s)
I20260810 04:45:03.906610 21517 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786336863388938-23197-0/minicluster-data/master-0/data/info.pb
I20260810 04:45:03.907474 21644 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 04:45:03.908756 23197 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskemv_AM/build/release/bin/kudu as pid 21517
I20260810 04:45:03.908880 23197 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786336863388938-23197-0/minicluster-data/master-0/wal/instance
I20260810 04:45:03.909286 23197 external_mini_cluster.cc:1428] Setting key e9a5a82d872a44cb64801a184854eaba
I20260810 04:45:03.910598 21644 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e4a08e9a41264911b0100ad613d923a7: Bootstrap starting.
I20260810 04:45:03.911168 21644 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e4a08e9a41264911b0100ad613d923a7: Neither blocks nor log segments found. Creating new log.
I20260810 04:45:03.911432 21644 log.cc:824] T 00000000000000000000000000000000 P e4a08e9a41264911b0100ad613d923a7: Log is configured to *not* fsync() on all Append() calls
I20260810 04:45:03.912079 21644 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e4a08e9a41264911b0100ad613d923a7: No bootstrap required, opened a new log
I20260810 04:45:03.913841 21644 raft_consensus.cc:348] T 00000000000000000000000000000000 P e4a08e9a41264911b0100ad613d923a7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e4a08e9a41264911b0100ad613d923a7" member_type: VOTER last_known_addr { host: "127.22.167.126" port: 36491 } }
I20260810 04:45:03.913978 21644 raft_consensus.cc:374] T 00000000000000000000000000000000 P e4a08e9a41264911b0100ad613d923a7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 04:45:03.914011 21644 raft_consensus.cc:729] T 00000000000000000000000000000000 P e4a08e9a41264911b0100ad613d923a7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e4a08e9a41264911b0100ad613d923a7, State: Initialized, Role: FOLLOWER
I20260810 04:45:03.914163 21644 consensus_queue.cc:260] T 00000000000000000000000000000000 P e4a08e9a41264911b0100ad613d923a7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e4a08e9a41264911b0100ad613d923a7" member_type: VOTER last_known_addr { host: "127.22.167.126" port: 36491 } }
I20260810 04:45:03.914232 21644 raft_consensus.cc:388] T 00000000000000000000000000000000 P e4a08e9a41264911b0100ad613d923a7 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 04:45:03.914264 21644 raft_consensus.cc:482] T 00000000000000000000000000000000 P e4a08e9a41264911b0100ad613d923a7 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 04:45:03.914314 21644 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e4a08e9a41264911b0100ad613d923a7 [term 0 FOLLOWER]: Advancing to term 1
I20260810 04:45:03.914820 21644 raft_consensus.cc:504] T 00000000000000000000000000000000 P e4a08e9a41264911b0100ad613d923a7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e4a08e9a41264911b0100ad613d923a7" member_type: VOTER last_known_addr { host: "127.22.167.126" port: 36491 } }
I20260810 04:45:03.914915 21644 leader_election.cc:302] T 00000000000000000000000000000000 P e4a08e9a41264911b0100ad613d923a7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e4a08e9a41264911b0100ad613d923a7; no voters:
I20260810 04:45:03.915072 21644 leader_election.cc:288] T 00000000000000000000000000000000 P e4a08e9a41264911b0100ad613d923a7 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 04:45:03.915110 21655 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e4a08e9a41264911b0100ad613d923a7 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 04:45:03.915226 21655 raft_consensus.cc:686] T 00000000000000000000000000000000 P e4a08e9a41264911b0100ad613d923a7 [term 1 LEADER]: Becoming Leader. State: Replica: e4a08e9a41264911b0100ad613d923a7, State: Running, Role: LEADER
I20260810 04:45:03.915306 21655 consensus_queue.cc:237] T 00000000000000000000000000000000 P e4a08e9a41264911b0100ad613d923a7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e4a08e9a41264911b0100ad613d923a7" member_type: VOTER last_known_addr { host: "127.22.167.126" port: 36491 } }
I20260810 04:45:03.915329 21644 sys_catalog.cc:564] T 00000000000000000000000000000000 P e4a08e9a41264911b0100ad613d923a7 [sys.catalog]: configured and running, proceeding with master startup.
Aug 10 04:45:03 dist-test-slave-bqcl krb5kdc[21489](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786337103, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.167.126@KRBTEST.COM
I20260810 04:45:03.916301 21656 sys_catalog.cc:455] T 00000000000000000000000000000000 P e4a08e9a41264911b0100ad613d923a7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e4a08e9a41264911b0100ad613d923a7. Latest consensus state: current_term: 1 leader_uuid: "e4a08e9a41264911b0100ad613d923a7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e4a08e9a41264911b0100ad613d923a7" member_type: VOTER last_known_addr { host: "127.22.167.126" port: 36491 } } }
I20260810 04:45:03.916386 21656 sys_catalog.cc:458] T 00000000000000000000000000000000 P e4a08e9a41264911b0100ad613d923a7 [sys.catalog]: This master's current role is: LEADER
I20260810 04:45:03.916566 21657 sys_catalog.cc:455] T 00000000000000000000000000000000 P e4a08e9a41264911b0100ad613d923a7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e4a08e9a41264911b0100ad613d923a7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e4a08e9a41264911b0100ad613d923a7" member_type: VOTER last_known_addr { host: "127.22.167.126" port: 36491 } } }
I20260810 04:45:03.916638 21657 sys_catalog.cc:458] T 00000000000000000000000000000000 P e4a08e9a41264911b0100ad613d923a7 [sys.catalog]: This master's current role is: LEADER
I20260810 04:45:03.916800 21665 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 04:45:03.917313 21665 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 04:45:03.918902 21665 catalog_manager.cc:1347] Generated new cluster ID: 43e239a67ce344c499731f35ec4a1eb9
I20260810 04:45:03.918954 21665 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 04:45:03.920272 21651 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:45:03.909949 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54808 (local address 127.22.167.126:36491)
0810 04:45:03.910169 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0810 04:45:03.910178 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 04:45:03.910208 (+ 30us) server_negotiation.cc:407] Connection header received
0810 04:45:03.910595 (+ 387us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:45:03.910601 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:45:03.910648 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:45:03.910799 (+ 151us) server_negotiation.cc:226] Negotiated authn=SASL
0810 04:45:03.911432 (+ 633us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:45:03.912398 (+ 966us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:45:03.913149 (+ 751us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:45:03.913556 (+ 407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:45:03.913652 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:45:03.916194 (+ 2542us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 04:45:03.916199 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 04:45:03.916199 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 04:45:03.916212 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0810 04:45:03.918252 (+ 2040us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 04:45:03.918664 (+ 412us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 04:45:03.918667 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 04:45:03.918668 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 04:45:03.918771 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 04:45:03.918997 (+ 226us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 04:45:03.918998 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 04:45:03.918998 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 04:45:03.919523 (+ 525us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 04:45:03.919751 (+ 228us) server_negotiation.cc:1035] Waiting for connection context
0810 04:45:03.919939 (+ 188us) server_negotiation.cc:299] Negotiation successful
0810 04:45:03.920069 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":141,"thread_start_us":86,"threads_started":1}
I20260810 04:45:03.928846 21665 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 04:45:03.929554 21665 catalog_manager.cc:1504] Loading token signing keys...
I20260810 04:45:03.932801 21665 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e4a08e9a41264911b0100ad613d923a7: Generated new TSK 0
I20260810 04:45:03.933035 21665 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.22.167.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.167.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.167.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786336863388938-23197-0/krb5kdc/kudu_127.22.167.65.keytab.
Entry for principal kudu/127.22.167.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786336863388938-23197-0/krb5kdc/kudu_127.22.167.65.keytab.
I20260810 04:45:03.971103 23197 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskemv_AM/build/release/bin/kudu
/tmp/dist-test-taskemv_AM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786336863388938-23197-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786336863388938-23197-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786336863388938-23197-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786336863388938-23197-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.167.65:0
--local_ip_for_outbound_sockets=127.22.167.65
--webserver_interface=127.22.167.65
--webserver_port=0
--tserver_master_addrs=127.22.167.126:36491
--keytab_file=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786336863388938-23197-0/krb5kdc/kudu_127.22.167.65.keytab
--principal=kudu/127.22.167.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.167.84:43213
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786336863388938-23197-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786336863388938-23197-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786336863388938-23197-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 04:45:04.115926 21686 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 04:45:04.116335 21686 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 04:45:04.116410 21686 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 04:45:04.118594 21686 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 04:45:04.118671 21686 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 04:45:04.118732 21686 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.167.65
I20260810 04:45:04.121177 21686 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.167.84:43213
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786336863388938-23197-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786336863388938-23197-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786336863388938-23197-0/krb5kdc/kudu_127.22.167.65.keytab
--principal=kudu/127.22.167.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.167.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786336863388938-23197-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.22.167.65
--webserver_port=0
--tserver_master_addrs=127.22.167.126:36491
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21686
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.167.65
--log_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786336863388938-23197-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 03:54:01 UTC on ccycloud.quasar-icijmt.root.comops.site
build id 4577009
I20260810 04:45:04.121680 21686 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 04:45:04.122104 21686 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 04:45:04.131896 21696 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 04:45:04.131844 21699 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 04:45:04.131860 21702 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 04:45:04.132452 21686 server_base.cc:1034] running on GCE node
I20260810 04:45:04.132831 21686 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 04:45:04.133231 21686 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 04:45:04.134423 21686 hybrid_clock.cc:648] HybridClock initialized: now 1786337104134395 us; error 59 us; skew 500 ppm
Aug 10 04:45:04 dist-test-slave-bqcl krb5kdc[21489](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786337104, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.167.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 04:45:04.138649 21686 init.cc:377] Logged in from keytab as kudu/127.22.167.65@KRBTEST.COM (short username kudu)
I20260810 04:45:04.139309 21686 webserver.cc:460] Webserver started at http://127.22.167.65:33553/ using document root <none> and password file <none>
I20260810 04:45:04.139580 21686 fs_manager.cc:359] Metadata directory not provided
I20260810 04:45:04.139709 21686 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 04:45:04.139938 21686 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 04:45:04.141700 21686 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786336863388938-23197-0/minicluster-data/ts-0/data/instance:
uuid: "538d5f775d8743ee9d49788304425318"
format_stamp: "Formatted at 2026-08-10 04:45:04 on dist-test-slave-bqcl"
server_key: "c2cbb0e81a9ba15a0b37b48a833196eb"
server_key_iv: "7f7a49b4fecf25813ecf05970aa3b98e"
server_key_version: "encryptionkey@0"
I20260810 04:45:04.142373 21686 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786336863388938-23197-0/minicluster-data/ts-0/wal/instance:
uuid: "538d5f775d8743ee9d49788304425318"
format_stamp: "Formatted at 2026-08-10 04:45:04 on dist-test-slave-bqcl"
server_key: "c2cbb0e81a9ba15a0b37b48a833196eb"
server_key_iv: "7f7a49b4fecf25813ecf05970aa3b98e"
server_key_version: "encryptionkey@0"
I20260810 04:45:04.144775 21686 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.002s sys 0.000s
I20260810 04:45:04.146008 21713 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 04:45:04.146315 21686 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 04:45:04.146464 21686 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786336863388938-23197-0/minicluster-data/ts-0/data,/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786336863388938-23197-0/minicluster-data/ts-0/wal
uuid: "538d5f775d8743ee9d49788304425318"
format_stamp: "Formatted at 2026-08-10 04:45:04 on dist-test-slave-bqcl"
server_key: "c2cbb0e81a9ba15a0b37b48a833196eb"
server_key_iv: "7f7a49b4fecf25813ecf05970aa3b98e"
server_key_version: "encryptionkey@0"
I20260810 04:45:04.146555 21686 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786336863388938-23197-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786336863388938-23197-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786336863388938-23197-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 04:45:04.182204 21686 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 04:45:04.186092 21686 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 04:45:04.186338 21686 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 04:45:04.186676 21686 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 04:45:04.187114 21686 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 04:45:04.187178 21686 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 04:45:04.187227 21686 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 04:45:04.187274 21686 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 04:45:04.197868 21686 rpc_server.cc:307] RPC server started. Bound to: 127.22.167.65:46489
I20260810 04:45:04.197970 21895 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.167.65:46489 every 8 connection(s)
I20260810 04:45:04.198551 21686 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786336863388938-23197-0/minicluster-data/ts-0/data/info.pb
I20260810 04:45:04.200910 23197 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskemv_AM/build/release/bin/kudu as pid 21686
I20260810 04:45:04.201167 23197 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786336863388938-23197-0/minicluster-data/ts-0/wal/instance
I20260810 04:45:04.201563 23197 external_mini_cluster.cc:1428] Setting key e8e19ac230b18b70211d9ea0a91bbcc1
Aug 10 04:45:04 dist-test-slave-bqcl krb5kdc[21489](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786337104, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.167.65@KRBTEST.COM for kudu/127.22.167.126@KRBTEST.COM
I20260810 04:45:04.215390 21651 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:45:04.199725 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.167.65:53883 (local address 127.22.167.126:36491)
0810 04:45:04.199808 (+ 83us) server_negotiation.cc:206] Beginning negotiation
0810 04:45:04.199815 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 04:45:04.200128 (+ 313us) server_negotiation.cc:407] Connection header received
0810 04:45:04.201325 (+ 1197us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:45:04.201327 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:45:04.201351 (+ 24us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:45:04.201430 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0810 04:45:04.203555 (+ 2125us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:45:04.204301 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:45:04.206567 (+ 2266us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:45:04.206896 (+ 329us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:45:04.207007 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:45:04.210263 (+ 3256us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 04:45:04.210264 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 04:45:04.210264 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 04:45:04.210280 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0810 04:45:04.212580 (+ 2300us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 04:45:04.213481 (+ 901us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 04:45:04.213482 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 04:45:04.213482 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 04:45:04.213608 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 04:45:04.214097 (+ 489us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 04:45:04.214098 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 04:45:04.214098 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 04:45:04.214320 (+ 222us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 04:45:04.214399 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0810 04:45:04.215126 (+ 727us) server_negotiation.cc:299] Negotiation successful
0810 04:45:04.215273 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260810 04:45:04.215961 21900 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:45:04.199703 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.126:36491 (local address 127.22.167.65:53883)
0810 04:45:04.199941 (+ 238us) negotiation.cc:107] Waiting for socket to connect
0810 04:45:04.200008 (+ 67us) client_negotiation.cc:174] Beginning negotiation
0810 04:45:04.201075 (+ 1067us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 04:45:04.201597 (+ 522us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 04:45:04.201603 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 04:45:04.202769 (+ 1166us) client_negotiation.cc:189] Negotiated authn=SASL
0810 04:45:04.203411 (+ 642us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:45:04.203420 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:45:04.204450 (+ 1030us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:45:04.204451 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:45:04.206456 (+ 2005us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:45:04.206459 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:45:04.207024 (+ 565us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:45:04.207026 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:45:04.207211 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:45:04.207914 (+ 703us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 04:45:04.207931 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0810 04:45:04.210050 (+ 2119us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 04:45:04.212700 (+ 2650us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 04:45:04.212719 (+ 19us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 04:45:04.212722 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 04:45:04.213330 (+ 608us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 04:45:04.213715 (+ 385us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 04:45:04.213717 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 04:45:04.213717 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 04:45:04.213996 (+ 279us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 04:45:04.214405 (+ 409us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 04:45:04.214426 (+ 21us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 04:45:04.214874 (+ 448us) client_negotiation.cc:769] Sending connection context
0810 04:45:04.215083 (+ 209us) client_negotiation.cc:240] Negotiation successful
0810 04:45:04.215517 (+ 434us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":172,"thread_start_us":118,"threads_started":1}
WARNING: no policy specified for kudu/127.22.167.66@KRBTEST.COM; defaulting to no policy
I20260810 04:45:04.216828 21896 heartbeater.cc:347] Connected to a master server at 127.22.167.126:36491
I20260810 04:45:04.216979 21896 heartbeater.cc:464] Registering TS with master...
I20260810 04:45:04.217314 21896 heartbeater.cc:511] Master 127.22.167.126:36491 requested a full tablet report, sending...
I20260810 04:45:04.218206 21580 ts_manager.cc:194] Registered new tserver with Master: 538d5f775d8743ee9d49788304425318 (127.22.167.65:46489)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.167.66@KRBTEST.COM" created.
I20260810 04:45:04.219329 21580 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.167.65@KRBTEST.COM'} at 127.22.167.65:53883
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.167.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786336863388938-23197-0/krb5kdc/kudu_127.22.167.66.keytab.
Entry for principal kudu/127.22.167.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786336863388938-23197-0/krb5kdc/kudu_127.22.167.66.keytab.
I20260810 04:45:04.239193 23197 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskemv_AM/build/release/bin/kudu
/tmp/dist-test-taskemv_AM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786336863388938-23197-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786336863388938-23197-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786336863388938-23197-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786336863388938-23197-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.167.66:0
--local_ip_for_outbound_sockets=127.22.167.66
--webserver_interface=127.22.167.66
--webserver_port=0
--tserver_master_addrs=127.22.167.126:36491
--keytab_file=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786336863388938-23197-0/krb5kdc/kudu_127.22.167.66.keytab
--principal=kudu/127.22.167.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.167.84:43213
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786336863388938-23197-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786336863388938-23197-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786336863388938-23197-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 04:45:04.371519 21903 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 04:45:04.371729 21903 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 04:45:04.371773 21903 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 04:45:04.373458 21903 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 04:45:04.373524 21903 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 04:45:04.373554 21903 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.167.66
I20260810 04:45:04.375932 21903 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.167.84:43213
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786336863388938-23197-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786336863388938-23197-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786336863388938-23197-0/krb5kdc/kudu_127.22.167.66.keytab
--principal=kudu/127.22.167.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.167.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786336863388938-23197-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.22.167.66
--webserver_port=0
--tserver_master_addrs=127.22.167.126:36491
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21903
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.167.66
--log_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786336863388938-23197-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 03:54:01 UTC on ccycloud.quasar-icijmt.root.comops.site
build id 4577009
I20260810 04:45:04.376358 21903 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 04:45:04.376865 21903 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 04:45:04.384775 21912 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 04:45:04.384755 21914 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 04:45:04.384755 21910 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 04:45:04.385427 21903 server_base.cc:1034] running on GCE node
I20260810 04:45:04.385794 21903 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 04:45:04.386269 21903 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 04:45:04.387467 21903 hybrid_clock.cc:648] HybridClock initialized: now 1786337104387425 us; error 58 us; skew 500 ppm
Aug 10 04:45:04 dist-test-slave-bqcl krb5kdc[21489](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786337104, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.167.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 04:45:04.391573 21903 init.cc:377] Logged in from keytab as kudu/127.22.167.66@KRBTEST.COM (short username kudu)
I20260810 04:45:04.392196 21903 webserver.cc:460] Webserver started at http://127.22.167.66:33085/ using document root <none> and password file <none>
I20260810 04:45:04.392524 21903 fs_manager.cc:359] Metadata directory not provided
I20260810 04:45:04.392611 21903 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 04:45:04.392771 21903 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 04:45:04.394635 21903 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786336863388938-23197-0/minicluster-data/ts-1/data/instance:
uuid: "5044e6aa6adc4262820f968cdec644e0"
format_stamp: "Formatted at 2026-08-10 04:45:04 on dist-test-slave-bqcl"
server_key: "3d0db4a4f46bba81063e3d05e2b03409"
server_key_iv: "466f665b52ba857175030cc5e16286b9"
server_key_version: "encryptionkey@0"
I20260810 04:45:04.395143 21903 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786336863388938-23197-0/minicluster-data/ts-1/wal/instance:
uuid: "5044e6aa6adc4262820f968cdec644e0"
format_stamp: "Formatted at 2026-08-10 04:45:04 on dist-test-slave-bqcl"
server_key: "3d0db4a4f46bba81063e3d05e2b03409"
server_key_iv: "466f665b52ba857175030cc5e16286b9"
server_key_version: "encryptionkey@0"
I20260810 04:45:04.397436 21903 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.003s sys 0.000s
I20260810 04:45:04.398710 21930 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 04:45:04.398950 21903 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260810 04:45:04.399083 21903 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786336863388938-23197-0/minicluster-data/ts-1/data,/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786336863388938-23197-0/minicluster-data/ts-1/wal
uuid: "5044e6aa6adc4262820f968cdec644e0"
format_stamp: "Formatted at 2026-08-10 04:45:04 on dist-test-slave-bqcl"
server_key: "3d0db4a4f46bba81063e3d05e2b03409"
server_key_iv: "466f665b52ba857175030cc5e16286b9"
server_key_version: "encryptionkey@0"
I20260810 04:45:04.399200 21903 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786336863388938-23197-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786336863388938-23197-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786336863388938-23197-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 04:45:04.420907 21903 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 04:45:04.424432 21903 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 04:45:04.424647 21903 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 04:45:04.425012 21903 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 04:45:04.425577 21903 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 04:45:04.425645 21903 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 04:45:04.425683 21903 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 04:45:04.425735 21903 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 04:45:04.436028 21903 rpc_server.cc:307] RPC server started. Bound to: 127.22.167.66:41809
I20260810 04:45:04.436144 22124 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.167.66:41809 every 8 connection(s)
I20260810 04:45:04.436650 21903 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786336863388938-23197-0/minicluster-data/ts-1/data/info.pb
I20260810 04:45:04.437443 23197 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskemv_AM/build/release/bin/kudu as pid 21903
I20260810 04:45:04.437582 23197 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786336863388938-23197-0/minicluster-data/ts-1/wal/instance
I20260810 04:45:04.437897 23197 external_mini_cluster.cc:1428] Setting key 17279e8ede4190ab2c14172fc89a1e23
Aug 10 04:45:04 dist-test-slave-bqcl krb5kdc[21489](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786337104, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.167.66@KRBTEST.COM for kudu/127.22.167.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.22.167.67@KRBTEST.COM; defaulting to no policy
I20260810 04:45:04.454610 21651 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:45:04.437677 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.167.66:55653 (local address 127.22.167.126:36491)
0810 04:45:04.437781 (+ 104us) server_negotiation.cc:206] Beginning negotiation
0810 04:45:04.437783 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 04:45:04.438340 (+ 557us) server_negotiation.cc:407] Connection header received
0810 04:45:04.439368 (+ 1028us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:45:04.439369 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:45:04.439389 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:45:04.439454 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0810 04:45:04.441886 (+ 2432us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:45:04.442681 (+ 795us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:45:04.445336 (+ 2655us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:45:04.445636 (+ 300us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:45:04.445711 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:45:04.449976 (+ 4265us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 04:45:04.449978 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 04:45:04.449978 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 04:45:04.449987 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0810 04:45:04.452354 (+ 2367us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 04:45:04.453025 (+ 671us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 04:45:04.453035 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 04:45:04.453035 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 04:45:04.453140 (+ 105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 04:45:04.453556 (+ 416us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 04:45:04.453557 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 04:45:04.453557 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 04:45:04.453774 (+ 217us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 04:45:04.453884 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0810 04:45:04.454323 (+ 439us) server_negotiation.cc:299] Negotiation successful
0810 04:45:04.454458 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260810 04:45:04.454893 22127 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:45:04.437676 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.126:36491 (local address 127.22.167.66:55653)
0810 04:45:04.438056 (+ 380us) negotiation.cc:107] Waiting for socket to connect
0810 04:45:04.438208 (+ 152us) client_negotiation.cc:174] Beginning negotiation
0810 04:45:04.439214 (+ 1006us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 04:45:04.439644 (+ 430us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 04:45:04.439658 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 04:45:04.441067 (+ 1409us) client_negotiation.cc:189] Negotiated authn=SASL
0810 04:45:04.441734 (+ 667us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:45:04.441744 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:45:04.442838 (+ 1094us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:45:04.442840 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:45:04.445168 (+ 2328us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:45:04.445180 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:45:04.445748 (+ 568us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:45:04.445749 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:45:04.446041 (+ 292us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:45:04.447018 (+ 977us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 04:45:04.447039 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0810 04:45:04.449846 (+ 2807us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 04:45:04.452480 (+ 2634us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 04:45:04.452495 (+ 15us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 04:45:04.452498 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 04:45:04.452926 (+ 428us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 04:45:04.453241 (+ 315us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 04:45:04.453242 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 04:45:04.453242 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 04:45:04.453471 (+ 229us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 04:45:04.453864 (+ 393us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 04:45:04.453881 (+ 17us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 04:45:04.454084 (+ 203us) client_negotiation.cc:769] Sending connection context
0810 04:45:04.454290 (+ 206us) client_negotiation.cc:240] Negotiation successful
0810 04:45:04.454561 (+ 271us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":199,"thread_start_us":133,"threads_started":1}
I20260810 04:45:04.455567 22125 heartbeater.cc:347] Connected to a master server at 127.22.167.126:36491
I20260810 04:45:04.455753 22125 heartbeater.cc:464] Registering TS with master...
I20260810 04:45:04.456060 22125 heartbeater.cc:511] Master 127.22.167.126:36491 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.167.67@KRBTEST.COM" created.
I20260810 04:45:04.456607 21580 ts_manager.cc:194] Registered new tserver with Master: 5044e6aa6adc4262820f968cdec644e0 (127.22.167.66:41809)
I20260810 04:45:04.457163 21580 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.167.66@KRBTEST.COM'} at 127.22.167.66:55653
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.167.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786336863388938-23197-0/krb5kdc/kudu_127.22.167.67.keytab.
Entry for principal kudu/127.22.167.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786336863388938-23197-0/krb5kdc/kudu_127.22.167.67.keytab.
I20260810 04:45:04.476626 23197 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskemv_AM/build/release/bin/kudu
/tmp/dist-test-taskemv_AM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786336863388938-23197-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786336863388938-23197-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786336863388938-23197-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786336863388938-23197-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.167.67:0
--local_ip_for_outbound_sockets=127.22.167.67
--webserver_interface=127.22.167.67
--webserver_port=0
--tserver_master_addrs=127.22.167.126:36491
--keytab_file=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786336863388938-23197-0/krb5kdc/kudu_127.22.167.67.keytab
--principal=kudu/127.22.167.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.167.84:43213
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786336863388938-23197-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786336863388938-23197-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786336863388938-23197-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 04:45:04.622546 22134 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 04:45:04.622999 22134 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 04:45:04.623044 22134 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 04:45:04.625033 22134 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 04:45:04.625106 22134 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 04:45:04.625144 22134 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.167.67
I20260810 04:45:04.627628 22134 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.167.84:43213
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786336863388938-23197-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786336863388938-23197-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786336863388938-23197-0/krb5kdc/kudu_127.22.167.67.keytab
--principal=kudu/127.22.167.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.167.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786336863388938-23197-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.22.167.67
--webserver_port=0
--tserver_master_addrs=127.22.167.126:36491
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22134
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.167.67
--log_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786336863388938-23197-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 03:54:01 UTC on ccycloud.quasar-icijmt.root.comops.site
build id 4577009
I20260810 04:45:04.627946 22134 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 04:45:04.628327 22134 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 04:45:04.636654 22143 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 04:45:04.636667 22142 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 04:45:04.636667 22153 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 04:45:04.637022 22134 server_base.cc:1034] running on GCE node
I20260810 04:45:04.637552 22134 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 04:45:04.637945 22134 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 04:45:04.639212 22134 hybrid_clock.cc:648] HybridClock initialized: now 1786337104639188 us; error 36 us; skew 500 ppm
Aug 10 04:45:04 dist-test-slave-bqcl krb5kdc[21489](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786337104, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.167.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 04:45:04.648234 22134 init.cc:377] Logged in from keytab as kudu/127.22.167.67@KRBTEST.COM (short username kudu)
I20260810 04:45:04.649010 22134 webserver.cc:460] Webserver started at http://127.22.167.67:33391/ using document root <none> and password file <none>
I20260810 04:45:04.649432 22134 fs_manager.cc:359] Metadata directory not provided
I20260810 04:45:04.649593 22134 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 04:45:04.649814 22134 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 04:45:04.651551 22134 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786336863388938-23197-0/minicluster-data/ts-2/data/instance:
uuid: "e0665795199545e38e85b7e2365d12b8"
format_stamp: "Formatted at 2026-08-10 04:45:04 on dist-test-slave-bqcl"
server_key: "a263e6a80fbef3f832e4b93f024373ad"
server_key_iv: "d18dc6d718d9eb2e13a568857b46b08a"
server_key_version: "encryptionkey@0"
I20260810 04:45:04.652185 22134 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786336863388938-23197-0/minicluster-data/ts-2/wal/instance:
uuid: "e0665795199545e38e85b7e2365d12b8"
format_stamp: "Formatted at 2026-08-10 04:45:04 on dist-test-slave-bqcl"
server_key: "a263e6a80fbef3f832e4b93f024373ad"
server_key_iv: "d18dc6d718d9eb2e13a568857b46b08a"
server_key_version: "encryptionkey@0"
I20260810 04:45:04.654749 22134 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.001s sys 0.004s
I20260810 04:45:04.655994 22162 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 04:45:04.656268 22134 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 04:45:04.656416 22134 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786336863388938-23197-0/minicluster-data/ts-2/data,/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786336863388938-23197-0/minicluster-data/ts-2/wal
uuid: "e0665795199545e38e85b7e2365d12b8"
format_stamp: "Formatted at 2026-08-10 04:45:04 on dist-test-slave-bqcl"
server_key: "a263e6a80fbef3f832e4b93f024373ad"
server_key_iv: "d18dc6d718d9eb2e13a568857b46b08a"
server_key_version: "encryptionkey@0"
I20260810 04:45:04.656534 22134 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786336863388938-23197-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786336863388938-23197-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786336863388938-23197-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 04:45:04.689604 22134 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 04:45:04.693521 22134 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 04:45:04.693747 22134 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 04:45:04.694175 22134 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 04:45:04.694661 22134 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 04:45:04.694721 22134 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 04:45:04.694790 22134 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 04:45:04.694850 22134 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 04:45:04.712137 22134 rpc_server.cc:307] RPC server started. Bound to: 127.22.167.67:33019
I20260810 04:45:04.712209 22349 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.167.67:33019 every 8 connection(s)
I20260810 04:45:04.712893 22134 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786336863388938-23197-0/minicluster-data/ts-2/data/info.pb
I20260810 04:45:04.715778 23197 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskemv_AM/build/release/bin/kudu as pid 22134
I20260810 04:45:04.716018 23197 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786336863388938-23197-0/minicluster-data/ts-2/wal/instance
I20260810 04:45:04.716410 23197 external_mini_cluster.cc:1428] Setting key 8849cc822594d9d218ce931528695987
Aug 10 04:45:04 dist-test-slave-bqcl krb5kdc[21489](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786337104, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.167.67@KRBTEST.COM for kudu/127.22.167.126@KRBTEST.COM
I20260810 04:45:04.732560 21651 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:45:04.716198 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.167.67:40333 (local address 127.22.167.126:36491)
0810 04:45:04.716275 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0810 04:45:04.716277 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 04:45:04.716297 (+ 20us) server_negotiation.cc:407] Connection header received
0810 04:45:04.718923 (+ 2626us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:45:04.718924 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:45:04.718966 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:45:04.719041 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0810 04:45:04.721003 (+ 1962us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:45:04.722081 (+ 1078us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:45:04.723780 (+ 1699us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:45:04.724053 (+ 273us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:45:04.724133 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:45:04.727622 (+ 3489us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 04:45:04.727624 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 04:45:04.727625 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 04:45:04.727647 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0810 04:45:04.730244 (+ 2597us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 04:45:04.730987 (+ 743us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 04:45:04.730988 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 04:45:04.730989 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 04:45:04.731100 (+ 111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 04:45:04.731533 (+ 433us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 04:45:04.731535 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 04:45:04.731535 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 04:45:04.731707 (+ 172us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 04:45:04.731805 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0810 04:45:04.732241 (+ 436us) server_negotiation.cc:299] Negotiation successful
0810 04:45:04.732380 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260810 04:45:04.732690 22352 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:45:04.715667 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.126:36491 (local address 127.22.167.67:40333)
0810 04:45:04.715990 (+ 323us) negotiation.cc:107] Waiting for socket to connect
0810 04:45:04.716109 (+ 119us) client_negotiation.cc:174] Beginning negotiation
0810 04:45:04.718655 (+ 2546us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 04:45:04.719438 (+ 783us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 04:45:04.719445 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 04:45:04.720150 (+ 705us) client_negotiation.cc:189] Negotiated authn=SASL
0810 04:45:04.720802 (+ 652us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:45:04.720809 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:45:04.722311 (+ 1502us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:45:04.722312 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:45:04.723682 (+ 1370us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:45:04.723691 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:45:04.724156 (+ 465us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:45:04.724157 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:45:04.724348 (+ 191us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:45:04.725174 (+ 826us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 04:45:04.725192 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0810 04:45:04.727466 (+ 2274us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 04:45:04.730333 (+ 2867us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 04:45:04.730339 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 04:45:04.730352 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0810 04:45:04.730876 (+ 524us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 04:45:04.731198 (+ 322us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 04:45:04.731198 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 04:45:04.731199 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 04:45:04.731365 (+ 166us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 04:45:04.731789 (+ 424us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 04:45:04.731801 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 04:45:04.731986 (+ 185us) client_negotiation.cc:769] Sending connection context
0810 04:45:04.732158 (+ 172us) client_negotiation.cc:240] Negotiation successful
0810 04:45:04.732363 (+ 205us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":205,"thread_start_us":135,"threads_started":1}
I20260810 04:45:04.733520 22350 heartbeater.cc:347] Connected to a master server at 127.22.167.126:36491
I20260810 04:45:04.733707 22350 heartbeater.cc:464] Registering TS with master...
I20260810 04:45:04.734172 22350 heartbeater.cc:511] Master 127.22.167.126:36491 requested a full tablet report, sending...
I20260810 04:45:04.734876 21580 ts_manager.cc:194] Registered new tserver with Master: e0665795199545e38e85b7e2365d12b8 (127.22.167.67:33019)
I20260810 04:45:04.735471 21580 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.167.67@KRBTEST.COM'} at 127.22.167.67:40333
I20260810 04:45:04.742146 23197 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260810 04:45:04.753593 21651 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:45:04.744139 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54812 (local address 127.22.167.126:36491)
0810 04:45:04.744214 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0810 04:45:04.744216 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 04:45:04.744361 (+ 145us) server_negotiation.cc:407] Connection header received
0810 04:45:04.744440 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:45:04.744441 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:45:04.744451 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:45:04.744523 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0810 04:45:04.745434 (+ 911us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:45:04.746171 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:45:04.747170 (+ 999us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:45:04.747424 (+ 254us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:45:04.747496 (+ 72us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:45:04.749477 (+ 1981us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 04:45:04.749478 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 04:45:04.749478 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 04:45:04.749492 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0810 04:45:04.751862 (+ 2370us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 04:45:04.752529 (+ 667us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 04:45:04.752530 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 04:45:04.752530 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 04:45:04.752628 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 04:45:04.752936 (+ 308us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 04:45:04.752937 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 04:45:04.752944 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0810 04:45:04.753137 (+ 193us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 04:45:04.753278 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0810 04:45:04.753422 (+ 144us) server_negotiation.cc:299] Negotiation successful
0810 04:45:04.753508 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260810 04:45:04.755774 21580 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54812:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 04:45:04.758482 21580 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 04:45:04.768460 22377 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:45:04.762015 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.67:33019 (local address 127.0.0.1:58640)
0810 04:45:04.762722 (+ 707us) negotiation.cc:107] Waiting for socket to connect
0810 04:45:04.762733 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0810 04:45:04.762812 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 04:45:04.763391 (+ 579us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 04:45:04.763392 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 04:45:04.763394 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 04:45:04.763483 (+ 89us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:45:04.763485 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:45:04.765470 (+ 1985us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:45:04.765471 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:45:04.766741 (+ 1270us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:45:04.766743 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:45:04.767917 (+ 1174us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:45:04.767919 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:45:04.768079 (+ 160us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:45:04.768114 (+ 35us) client_negotiation.cc:769] Sending connection context
0810 04:45:04.768224 (+ 110us) client_negotiation.cc:240] Negotiation successful
0810 04:45:04.768331 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":668,"mutex_wait_us":2,"thread_start_us":194,"threads_started":1}
I20260810 04:45:04.768478 22379 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:45:04.762425 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58640 (local address 127.22.167.67:33019)
0810 04:45:04.763051 (+ 626us) server_negotiation.cc:206] Beginning negotiation
0810 04:45:04.763056 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 04:45:04.763066 (+ 10us) server_negotiation.cc:407] Connection header received
0810 04:45:04.763165 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:45:04.763170 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:45:04.763213 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:45:04.763395 (+ 182us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 04:45:04.763618 (+ 223us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:45:04.765308 (+ 1690us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:45:04.766841 (+ 1533us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:45:04.767812 (+ 971us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:45:04.767900 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:45:04.768052 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0810 04:45:04.768252 (+ 200us) server_negotiation.cc:299] Negotiation successful
0810 04:45:04.768330 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":569,"thread_start_us":138,"threads_started":1}
I20260810 04:45:04.769173 22374 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:45:04.762039 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.66:41809 (local address 127.0.0.1:56264)
0810 04:45:04.762943 (+ 904us) negotiation.cc:107] Waiting for socket to connect
0810 04:45:04.762964 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0810 04:45:04.763046 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 04:45:04.763566 (+ 520us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 04:45:04.763567 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 04:45:04.763569 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 04:45:04.763629 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:45:04.763631 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:45:04.765023 (+ 1392us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:45:04.765024 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:45:04.767098 (+ 2074us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:45:04.767101 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:45:04.768788 (+ 1687us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:45:04.768789 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:45:04.768885 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:45:04.768891 (+ 6us) client_negotiation.cc:769] Sending connection context
0810 04:45:04.768997 (+ 106us) client_negotiation.cc:240] Negotiation successful
0810 04:45:04.769073 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":875,"mutex_wait_us":37,"thread_start_us":152,"threads_started":1}
I20260810 04:45:04.769407 22369 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:45:04.762005 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.65:46489 (local address 127.0.0.1:36938)
0810 04:45:04.762352 (+ 347us) negotiation.cc:107] Waiting for socket to connect
0810 04:45:04.762433 (+ 81us) client_negotiation.cc:174] Beginning negotiation
0810 04:45:04.762549 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 04:45:04.763110 (+ 561us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 04:45:04.763123 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 04:45:04.763126 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 04:45:04.763309 (+ 183us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:45:04.763311 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:45:04.765459 (+ 2148us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:45:04.765460 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:45:04.767289 (+ 1829us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:45:04.767292 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:45:04.769029 (+ 1737us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:45:04.769031 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:45:04.769106 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:45:04.769110 (+ 4us) client_negotiation.cc:769] Sending connection context
0810 04:45:04.769211 (+ 101us) client_negotiation.cc:240] Negotiation successful
0810 04:45:04.769283 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":324,"mutex_wait_us":59,"thread_start_us":110,"threads_started":1}
I20260810 04:45:04.769613 22382 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:45:04.762454 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56264 (local address 127.22.167.66:41809)
0810 04:45:04.762830 (+ 376us) server_negotiation.cc:206] Beginning negotiation
0810 04:45:04.762834 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 04:45:04.763172 (+ 338us) server_negotiation.cc:407] Connection header received
0810 04:45:04.763260 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:45:04.763264 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:45:04.763289 (+ 25us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:45:04.763408 (+ 119us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 04:45:04.763734 (+ 326us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:45:04.764836 (+ 1102us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:45:04.767233 (+ 2397us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:45:04.768484 (+ 1251us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:45:04.768610 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:45:04.768730 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0810 04:45:04.769417 (+ 687us) server_negotiation.cc:299] Negotiation successful
0810 04:45:04.769477 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":322,"thread_start_us":145,"threads_started":1}
I20260810 04:45:04.769838 22378 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:45:04.762283 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36938 (local address 127.22.167.65:46489)
0810 04:45:04.762803 (+ 520us) server_negotiation.cc:206] Beginning negotiation
0810 04:45:04.762808 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 04:45:04.762831 (+ 23us) server_negotiation.cc:407] Connection header received
0810 04:45:04.762909 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:45:04.762913 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:45:04.762944 (+ 31us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:45:04.763101 (+ 157us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 04:45:04.763485 (+ 384us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:45:04.765309 (+ 1824us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:45:04.767429 (+ 2120us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:45:04.768890 (+ 1461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:45:04.769016 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:45:04.769096 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0810 04:45:04.769389 (+ 293us) server_negotiation.cc:299] Negotiation successful
0810 04:45:04.769693 (+ 304us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":449,"thread_start_us":105,"threads_started":1}
I20260810 04:45:04.769882 22217 tablet_service.cc:1467] Processing CreateTablet for tablet 76677e9e7f5540e8a4ed49fa19e1a918 (DEFAULT_TABLE table=test-table [id=be4121cc12b54615974ac33835460140]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 04:45:04.770627 22217 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 76677e9e7f5540e8a4ed49fa19e1a918. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 04:45:04.770869 22026 tablet_service.cc:1467] Processing CreateTablet for tablet 76677e9e7f5540e8a4ed49fa19e1a918 (DEFAULT_TABLE table=test-table [id=be4121cc12b54615974ac33835460140]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 04:45:04.771409 22026 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 76677e9e7f5540e8a4ed49fa19e1a918. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 04:45:04.775043 21795 tablet_service.cc:1467] Processing CreateTablet for tablet 76677e9e7f5540e8a4ed49fa19e1a918 (DEFAULT_TABLE table=test-table [id=be4121cc12b54615974ac33835460140]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 04:45:04.775337 22384 tablet_bootstrap.cc:492] T 76677e9e7f5540e8a4ed49fa19e1a918 P e0665795199545e38e85b7e2365d12b8: Bootstrap starting.
I20260810 04:45:04.775709 21795 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 76677e9e7f5540e8a4ed49fa19e1a918. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 04:45:04.775954 22385 tablet_bootstrap.cc:492] T 76677e9e7f5540e8a4ed49fa19e1a918 P 5044e6aa6adc4262820f968cdec644e0: Bootstrap starting.
I20260810 04:45:04.776688 22384 tablet_bootstrap.cc:654] T 76677e9e7f5540e8a4ed49fa19e1a918 P e0665795199545e38e85b7e2365d12b8: Neither blocks nor log segments found. Creating new log.
I20260810 04:45:04.777144 22385 tablet_bootstrap.cc:654] T 76677e9e7f5540e8a4ed49fa19e1a918 P 5044e6aa6adc4262820f968cdec644e0: Neither blocks nor log segments found. Creating new log.
I20260810 04:45:04.777499 22384 log.cc:824] T 76677e9e7f5540e8a4ed49fa19e1a918 P e0665795199545e38e85b7e2365d12b8: Log is configured to *not* fsync() on all Append() calls
I20260810 04:45:04.778187 22385 log.cc:824] T 76677e9e7f5540e8a4ed49fa19e1a918 P 5044e6aa6adc4262820f968cdec644e0: Log is configured to *not* fsync() on all Append() calls
I20260810 04:45:04.778986 22384 tablet_bootstrap.cc:492] T 76677e9e7f5540e8a4ed49fa19e1a918 P e0665795199545e38e85b7e2365d12b8: No bootstrap required, opened a new log
I20260810 04:45:04.779120 22384 ts_tablet_manager.cc:1397] T 76677e9e7f5540e8a4ed49fa19e1a918 P e0665795199545e38e85b7e2365d12b8: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20260810 04:45:04.779497 22385 tablet_bootstrap.cc:492] T 76677e9e7f5540e8a4ed49fa19e1a918 P 5044e6aa6adc4262820f968cdec644e0: No bootstrap required, opened a new log
I20260810 04:45:04.779632 22385 ts_tablet_manager.cc:1397] T 76677e9e7f5540e8a4ed49fa19e1a918 P 5044e6aa6adc4262820f968cdec644e0: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20260810 04:45:04.781844 22388 tablet_bootstrap.cc:492] T 76677e9e7f5540e8a4ed49fa19e1a918 P 538d5f775d8743ee9d49788304425318: Bootstrap starting.
I20260810 04:45:04.781600 22384 raft_consensus.cc:348] T 76677e9e7f5540e8a4ed49fa19e1a918 P e0665795199545e38e85b7e2365d12b8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "538d5f775d8743ee9d49788304425318" member_type: VOTER last_known_addr { host: "127.22.167.65" port: 46489 } } peers { permanent_uuid: "e0665795199545e38e85b7e2365d12b8" member_type: VOTER last_known_addr { host: "127.22.167.67" port: 33019 } } peers { permanent_uuid: "5044e6aa6adc4262820f968cdec644e0" member_type: VOTER last_known_addr { host: "127.22.167.66" port: 41809 } }
I20260810 04:45:04.782243 22384 raft_consensus.cc:374] T 76677e9e7f5540e8a4ed49fa19e1a918 P e0665795199545e38e85b7e2365d12b8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 04:45:04.782334 22384 raft_consensus.cc:729] T 76677e9e7f5540e8a4ed49fa19e1a918 P e0665795199545e38e85b7e2365d12b8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e0665795199545e38e85b7e2365d12b8, State: Initialized, Role: FOLLOWER
I20260810 04:45:04.782573 22384 consensus_queue.cc:260] T 76677e9e7f5540e8a4ed49fa19e1a918 P e0665795199545e38e85b7e2365d12b8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "538d5f775d8743ee9d49788304425318" member_type: VOTER last_known_addr { host: "127.22.167.65" port: 46489 } } peers { permanent_uuid: "e0665795199545e38e85b7e2365d12b8" member_type: VOTER last_known_addr { host: "127.22.167.67" port: 33019 } } peers { permanent_uuid: "5044e6aa6adc4262820f968cdec644e0" member_type: VOTER last_known_addr { host: "127.22.167.66" port: 41809 } }
I20260810 04:45:04.783059 22384 ts_tablet_manager.cc:1428] T 76677e9e7f5540e8a4ed49fa19e1a918 P e0665795199545e38e85b7e2365d12b8: Time spent starting tablet: real 0.004s user 0.004s sys 0.000s
I20260810 04:45:04.783150 22388 tablet_bootstrap.cc:654] T 76677e9e7f5540e8a4ed49fa19e1a918 P 538d5f775d8743ee9d49788304425318: Neither blocks nor log segments found. Creating new log.
I20260810 04:45:04.783303 22350 heartbeater.cc:503] Master 127.22.167.126:36491 was elected leader, sending a full tablet report...
I20260810 04:45:04.783304 22385 raft_consensus.cc:348] T 76677e9e7f5540e8a4ed49fa19e1a918 P 5044e6aa6adc4262820f968cdec644e0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "538d5f775d8743ee9d49788304425318" member_type: VOTER last_known_addr { host: "127.22.167.65" port: 46489 } } peers { permanent_uuid: "e0665795199545e38e85b7e2365d12b8" member_type: VOTER last_known_addr { host: "127.22.167.67" port: 33019 } } peers { permanent_uuid: "5044e6aa6adc4262820f968cdec644e0" member_type: VOTER last_known_addr { host: "127.22.167.66" port: 41809 } }
I20260810 04:45:04.783595 22385 raft_consensus.cc:374] T 76677e9e7f5540e8a4ed49fa19e1a918 P 5044e6aa6adc4262820f968cdec644e0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 04:45:04.783677 22385 raft_consensus.cc:729] T 76677e9e7f5540e8a4ed49fa19e1a918 P 5044e6aa6adc4262820f968cdec644e0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5044e6aa6adc4262820f968cdec644e0, State: Initialized, Role: FOLLOWER
I20260810 04:45:04.783761 22388 log.cc:824] T 76677e9e7f5540e8a4ed49fa19e1a918 P 538d5f775d8743ee9d49788304425318: Log is configured to *not* fsync() on all Append() calls
I20260810 04:45:04.783890 22385 consensus_queue.cc:260] T 76677e9e7f5540e8a4ed49fa19e1a918 P 5044e6aa6adc4262820f968cdec644e0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "538d5f775d8743ee9d49788304425318" member_type: VOTER last_known_addr { host: "127.22.167.65" port: 46489 } } peers { permanent_uuid: "e0665795199545e38e85b7e2365d12b8" member_type: VOTER last_known_addr { host: "127.22.167.67" port: 33019 } } peers { permanent_uuid: "5044e6aa6adc4262820f968cdec644e0" member_type: VOTER last_known_addr { host: "127.22.167.66" port: 41809 } }
I20260810 04:45:04.784361 22385 ts_tablet_manager.cc:1428] T 76677e9e7f5540e8a4ed49fa19e1a918 P 5044e6aa6adc4262820f968cdec644e0: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260810 04:45:04.784551 22125 heartbeater.cc:503] Master 127.22.167.126:36491 was elected leader, sending a full tablet report...
I20260810 04:45:04.785820 22388 tablet_bootstrap.cc:492] T 76677e9e7f5540e8a4ed49fa19e1a918 P 538d5f775d8743ee9d49788304425318: No bootstrap required, opened a new log
I20260810 04:45:04.785917 22388 ts_tablet_manager.cc:1397] T 76677e9e7f5540e8a4ed49fa19e1a918 P 538d5f775d8743ee9d49788304425318: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20260810 04:45:04.788902 22388 raft_consensus.cc:348] T 76677e9e7f5540e8a4ed49fa19e1a918 P 538d5f775d8743ee9d49788304425318 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "538d5f775d8743ee9d49788304425318" member_type: VOTER last_known_addr { host: "127.22.167.65" port: 46489 } } peers { permanent_uuid: "e0665795199545e38e85b7e2365d12b8" member_type: VOTER last_known_addr { host: "127.22.167.67" port: 33019 } } peers { permanent_uuid: "5044e6aa6adc4262820f968cdec644e0" member_type: VOTER last_known_addr { host: "127.22.167.66" port: 41809 } }
I20260810 04:45:04.789125 22388 raft_consensus.cc:374] T 76677e9e7f5540e8a4ed49fa19e1a918 P 538d5f775d8743ee9d49788304425318 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 04:45:04.789208 22388 raft_consensus.cc:729] T 76677e9e7f5540e8a4ed49fa19e1a918 P 538d5f775d8743ee9d49788304425318 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 538d5f775d8743ee9d49788304425318, State: Initialized, Role: FOLLOWER
I20260810 04:45:04.789443 22388 consensus_queue.cc:260] T 76677e9e7f5540e8a4ed49fa19e1a918 P 538d5f775d8743ee9d49788304425318 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "538d5f775d8743ee9d49788304425318" member_type: VOTER last_known_addr { host: "127.22.167.65" port: 46489 } } peers { permanent_uuid: "e0665795199545e38e85b7e2365d12b8" member_type: VOTER last_known_addr { host: "127.22.167.67" port: 33019 } } peers { permanent_uuid: "5044e6aa6adc4262820f968cdec644e0" member_type: VOTER last_known_addr { host: "127.22.167.66" port: 41809 } }
I20260810 04:45:04.789865 22388 ts_tablet_manager.cc:1428] T 76677e9e7f5540e8a4ed49fa19e1a918 P 538d5f775d8743ee9d49788304425318: Time spent starting tablet: real 0.004s user 0.001s sys 0.004s
I20260810 04:45:04.789942 21896 heartbeater.cc:503] Master 127.22.167.126:36491 was elected leader, sending a full tablet report...
I20260810 04:45:04.815400 22391 raft_consensus.cc:482] T 76677e9e7f5540e8a4ed49fa19e1a918 P 5044e6aa6adc4262820f968cdec644e0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 04:45:04.815544 22391 raft_consensus.cc:504] T 76677e9e7f5540e8a4ed49fa19e1a918 P 5044e6aa6adc4262820f968cdec644e0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "538d5f775d8743ee9d49788304425318" member_type: VOTER last_known_addr { host: "127.22.167.65" port: 46489 } } peers { permanent_uuid: "e0665795199545e38e85b7e2365d12b8" member_type: VOTER last_known_addr { host: "127.22.167.67" port: 33019 } } peers { permanent_uuid: "5044e6aa6adc4262820f968cdec644e0" member_type: VOTER last_known_addr { host: "127.22.167.66" port: 41809 } }
I20260810 04:45:04.816095 22391 leader_election.cc:288] T 76677e9e7f5540e8a4ed49fa19e1a918 P 5044e6aa6adc4262820f968cdec644e0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 538d5f775d8743ee9d49788304425318 (127.22.167.65:46489), e0665795199545e38e85b7e2365d12b8 (127.22.167.67:33019)
I20260810 04:45:04.820350 22379 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:45:04.816441 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.167.66:45937 (local address 127.22.167.67:33019)
0810 04:45:04.816502 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0810 04:45:04.816504 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 04:45:04.816512 (+ 8us) server_negotiation.cc:407] Connection header received
0810 04:45:04.816550 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:45:04.816551 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:45:04.816560 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:45:04.816672 (+ 112us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 04:45:04.816873 (+ 201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:45:04.817681 (+ 808us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:45:04.819203 (+ 1522us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:45:04.819776 (+ 573us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:45:04.819896 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:45:04.819926 (+ 30us) server_negotiation.cc:1035] Waiting for connection context
0810 04:45:04.820155 (+ 229us) server_negotiation.cc:299] Negotiation successful
0810 04:45:04.820222 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260810 04:45:04.820406 22378 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:45:04.816363 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.167.66:39489 (local address 127.22.167.65:46489)
0810 04:45:04.816441 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0810 04:45:04.816444 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 04:45:04.816720 (+ 276us) server_negotiation.cc:407] Connection header received
0810 04:45:04.816802 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:45:04.816804 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:45:04.816814 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:45:04.816897 (+ 83us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 04:45:04.817190 (+ 293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:45:04.817887 (+ 697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:45:04.819289 (+ 1402us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:45:04.819864 (+ 575us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:45:04.820012 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:45:04.820022 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0810 04:45:04.820195 (+ 173us) server_negotiation.cc:299] Negotiation successful
0810 04:45:04.820279 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260810 04:45:04.820420 22127 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:45:04.816248 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.67:33019 (local address 127.22.167.66:45937)
0810 04:45:04.816305 (+ 57us) negotiation.cc:107] Waiting for socket to connect
0810 04:45:04.816322 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0810 04:45:04.816392 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 04:45:04.816684 (+ 292us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 04:45:04.816695 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 04:45:04.816697 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 04:45:04.816789 (+ 92us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:45:04.816791 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:45:04.817796 (+ 1005us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:45:04.817796 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:45:04.819057 (+ 1261us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:45:04.819067 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:45:04.819911 (+ 844us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:45:04.819912 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:45:04.820037 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:45:04.820041 (+ 4us) client_negotiation.cc:769] Sending connection context
0810 04:45:04.820111 (+ 70us) client_negotiation.cc:240] Negotiation successful
0810 04:45:04.820174 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":36,"mutex_wait_us":82}
I20260810 04:45:04.820463 22398 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:45:04.816248 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.65:46489 (local address 127.22.167.66:39489)
0810 04:45:04.816607 (+ 359us) negotiation.cc:107] Waiting for socket to connect
0810 04:45:04.816616 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0810 04:45:04.816702 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 04:45:04.816950 (+ 248us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 04:45:04.816951 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 04:45:04.816952 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 04:45:04.817086 (+ 134us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:45:04.817088 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:45:04.818011 (+ 923us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:45:04.818012 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:45:04.819178 (+ 1166us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:45:04.819189 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:45:04.820000 (+ 811us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:45:04.820001 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:45:04.820110 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:45:04.820113 (+ 3us) client_negotiation.cc:769] Sending connection context
0810 04:45:04.820187 (+ 74us) client_negotiation.cc:240] Negotiation successful
0810 04:45:04.820267 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":326,"mutex_wait_us":71,"thread_start_us":184,"threads_started":1}
I20260810 04:45:04.821079 22270 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "76677e9e7f5540e8a4ed49fa19e1a918" candidate_uuid: "5044e6aa6adc4262820f968cdec644e0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e0665795199545e38e85b7e2365d12b8" is_pre_election: true
I20260810 04:45:04.821132 21826 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "76677e9e7f5540e8a4ed49fa19e1a918" candidate_uuid: "5044e6aa6adc4262820f968cdec644e0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "538d5f775d8743ee9d49788304425318" is_pre_election: true
I20260810 04:45:04.821384 22270 raft_consensus.cc:2454] T 76677e9e7f5540e8a4ed49fa19e1a918 P e0665795199545e38e85b7e2365d12b8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5044e6aa6adc4262820f968cdec644e0 in term 0.
I20260810 04:45:04.821456 21826 raft_consensus.cc:2454] T 76677e9e7f5540e8a4ed49fa19e1a918 P 538d5f775d8743ee9d49788304425318 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5044e6aa6adc4262820f968cdec644e0 in term 0.
I20260810 04:45:04.821810 21932 leader_election.cc:302] T 76677e9e7f5540e8a4ed49fa19e1a918 P 5044e6aa6adc4262820f968cdec644e0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5044e6aa6adc4262820f968cdec644e0, 538d5f775d8743ee9d49788304425318; no voters:
I20260810 04:45:04.822054 22391 raft_consensus.cc:2781] T 76677e9e7f5540e8a4ed49fa19e1a918 P 5044e6aa6adc4262820f968cdec644e0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 04:45:04.822168 22391 raft_consensus.cc:482] T 76677e9e7f5540e8a4ed49fa19e1a918 P 5044e6aa6adc4262820f968cdec644e0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 04:45:04.822209 22391 raft_consensus.cc:3037] T 76677e9e7f5540e8a4ed49fa19e1a918 P 5044e6aa6adc4262820f968cdec644e0 [term 0 FOLLOWER]: Advancing to term 1
I20260810 04:45:04.823276 22391 raft_consensus.cc:504] T 76677e9e7f5540e8a4ed49fa19e1a918 P 5044e6aa6adc4262820f968cdec644e0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "538d5f775d8743ee9d49788304425318" member_type: VOTER last_known_addr { host: "127.22.167.65" port: 46489 } } peers { permanent_uuid: "e0665795199545e38e85b7e2365d12b8" member_type: VOTER last_known_addr { host: "127.22.167.67" port: 33019 } } peers { permanent_uuid: "5044e6aa6adc4262820f968cdec644e0" member_type: VOTER last_known_addr { host: "127.22.167.66" port: 41809 } }
I20260810 04:45:04.823468 22391 leader_election.cc:288] T 76677e9e7f5540e8a4ed49fa19e1a918 P 5044e6aa6adc4262820f968cdec644e0 [CANDIDATE]: Term 1 election: Requested vote from peers 538d5f775d8743ee9d49788304425318 (127.22.167.65:46489), e0665795199545e38e85b7e2365d12b8 (127.22.167.67:33019)
I20260810 04:45:04.823681 22270 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "76677e9e7f5540e8a4ed49fa19e1a918" candidate_uuid: "5044e6aa6adc4262820f968cdec644e0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e0665795199545e38e85b7e2365d12b8"
I20260810 04:45:04.823698 21826 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "76677e9e7f5540e8a4ed49fa19e1a918" candidate_uuid: "5044e6aa6adc4262820f968cdec644e0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "538d5f775d8743ee9d49788304425318"
I20260810 04:45:04.823792 21826 raft_consensus.cc:3037] T 76677e9e7f5540e8a4ed49fa19e1a918 P 538d5f775d8743ee9d49788304425318 [term 0 FOLLOWER]: Advancing to term 1
I20260810 04:45:04.823789 22270 raft_consensus.cc:3037] T 76677e9e7f5540e8a4ed49fa19e1a918 P e0665795199545e38e85b7e2365d12b8 [term 0 FOLLOWER]: Advancing to term 1
I20260810 04:45:04.824913 21826 raft_consensus.cc:2454] T 76677e9e7f5540e8a4ed49fa19e1a918 P 538d5f775d8743ee9d49788304425318 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5044e6aa6adc4262820f968cdec644e0 in term 1.
I20260810 04:45:04.824959 22270 raft_consensus.cc:2454] T 76677e9e7f5540e8a4ed49fa19e1a918 P e0665795199545e38e85b7e2365d12b8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5044e6aa6adc4262820f968cdec644e0 in term 1.
I20260810 04:45:04.825146 21932 leader_election.cc:302] T 76677e9e7f5540e8a4ed49fa19e1a918 P 5044e6aa6adc4262820f968cdec644e0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5044e6aa6adc4262820f968cdec644e0, 538d5f775d8743ee9d49788304425318; no voters:
I20260810 04:45:04.825276 22391 raft_consensus.cc:2781] T 76677e9e7f5540e8a4ed49fa19e1a918 P 5044e6aa6adc4262820f968cdec644e0 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 04:45:04.825497 22391 raft_consensus.cc:686] T 76677e9e7f5540e8a4ed49fa19e1a918 P 5044e6aa6adc4262820f968cdec644e0 [term 1 LEADER]: Becoming Leader. State: Replica: 5044e6aa6adc4262820f968cdec644e0, State: Running, Role: LEADER
I20260810 04:45:04.825667 22391 consensus_queue.cc:237] T 76677e9e7f5540e8a4ed49fa19e1a918 P 5044e6aa6adc4262820f968cdec644e0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "538d5f775d8743ee9d49788304425318" member_type: VOTER last_known_addr { host: "127.22.167.65" port: 46489 } } peers { permanent_uuid: "e0665795199545e38e85b7e2365d12b8" member_type: VOTER last_known_addr { host: "127.22.167.67" port: 33019 } } peers { permanent_uuid: "5044e6aa6adc4262820f968cdec644e0" member_type: VOTER last_known_addr { host: "127.22.167.66" port: 41809 } }
I20260810 04:45:04.827114 21579 catalog_manager.cc:5495] T 76677e9e7f5540e8a4ed49fa19e1a918 P 5044e6aa6adc4262820f968cdec644e0 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5044e6aa6adc4262820f968cdec644e0 (127.22.167.66). New cstate: current_term: 1 leader_uuid: "5044e6aa6adc4262820f968cdec644e0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "538d5f775d8743ee9d49788304425318" member_type: VOTER last_known_addr { host: "127.22.167.65" port: 46489 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e0665795199545e38e85b7e2365d12b8" member_type: VOTER last_known_addr { host: "127.22.167.67" port: 33019 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5044e6aa6adc4262820f968cdec644e0" member_type: VOTER last_known_addr { host: "127.22.167.66" port: 41809 } health_report { overall_health: HEALTHY } } }
I20260810 04:45:04.842563 22382 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:45:04.839317 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56276 (local address 127.22.167.66:41809)
0810 04:45:04.839403 (+ 86us) server_negotiation.cc:206] Beginning negotiation
0810 04:45:04.839405 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 04:45:04.839420 (+ 15us) server_negotiation.cc:407] Connection header received
0810 04:45:04.839516 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:45:04.839517 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:45:04.839542 (+ 25us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:45:04.839637 (+ 95us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 04:45:04.839798 (+ 161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:45:04.840463 (+ 665us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:45:04.841482 (+ 1019us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:45:04.841764 (+ 282us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:45:04.841828 (+ 64us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:45:04.842034 (+ 206us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 04:45:04.842212 (+ 178us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 04:45:04.842362 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0810 04:45:04.842410 (+ 48us) server_negotiation.cc:299] Negotiation successful
0810 04:45:04.842440 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260810 04:45:04.845201 21826 raft_consensus.cc:1261] T 76677e9e7f5540e8a4ed49fa19e1a918 P 538d5f775d8743ee9d49788304425318 [term 1 FOLLOWER]: Refusing update from remote peer 5044e6aa6adc4262820f968cdec644e0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 04:45:04.845207 22270 raft_consensus.cc:1261] T 76677e9e7f5540e8a4ed49fa19e1a918 P e0665795199545e38e85b7e2365d12b8 [term 1 FOLLOWER]: Refusing update from remote peer 5044e6aa6adc4262820f968cdec644e0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 04:45:04.845781 22391 consensus_queue.cc:1035] T 76677e9e7f5540e8a4ed49fa19e1a918 P 5044e6aa6adc4262820f968cdec644e0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "538d5f775d8743ee9d49788304425318" member_type: VOTER last_known_addr { host: "127.22.167.65" port: 46489 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 04:45:04.846014 22399 consensus_queue.cc:1035] T 76677e9e7f5540e8a4ed49fa19e1a918 P 5044e6aa6adc4262820f968cdec644e0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e0665795199545e38e85b7e2365d12b8" member_type: VOTER last_known_addr { host: "127.22.167.67" port: 33019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 04:45:04.851817 22405 mvcc.cc:205] Tried to move back new op lower bound from 7316836781442326528 to 7316836781367013376. Current Snapshot: MvccSnapshot[applied={T|T < 7316836781442326528 or (T in {7316836781442326528})}]
I20260810 04:45:04.854080 21580 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54812:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 04:45:04.854257 21580 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54812:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 04:45:04.855430 21580 catalog_manager.cc:5782] T 00000000000000000000000000000000 P e4a08e9a41264911b0100ad613d923a7: Sending DeleteTablet for 3 replicas of tablet 76677e9e7f5540e8a4ed49fa19e1a918
I20260810 04:45:04.855911 22217 tablet_service.cc:1514] Processing DeleteTablet for tablet 76677e9e7f5540e8a4ed49fa19e1a918 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 04:45:04 UTC) from {username='kudu', principal='kudu/127.22.167.126@KRBTEST.COM'} at 127.0.0.1:58640
I20260810 04:45:04.855911 21795 tablet_service.cc:1514] Processing DeleteTablet for tablet 76677e9e7f5540e8a4ed49fa19e1a918 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 04:45:04 UTC) from {username='kudu', principal='kudu/127.22.167.126@KRBTEST.COM'} at 127.0.0.1:36938
I20260810 04:45:04.856192 22419 tablet_replica.cc:330] T 76677e9e7f5540e8a4ed49fa19e1a918 P 538d5f775d8743ee9d49788304425318: stopping tablet replica
I20260810 04:45:04.856232 23197 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskemv_AM/build/release/bin/kudu with pid 21686
I20260810 04:45:04.856367 22419 raft_consensus.cc:2229] T 76677e9e7f5540e8a4ed49fa19e1a918 P 538d5f775d8743ee9d49788304425318 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 04:45:04.871408 22422 tablet_replica.cc:330] T 76677e9e7f5540e8a4ed49fa19e1a918 P e0665795199545e38e85b7e2365d12b8: stopping tablet replica
I20260810 04:45:04.871721 22422 raft_consensus.cc:2229] T 76677e9e7f5540e8a4ed49fa19e1a918 P e0665795199545e38e85b7e2365d12b8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 04:45:04.871984 22422 raft_consensus.cc:2258] T 76677e9e7f5540e8a4ed49fa19e1a918 P e0665795199545e38e85b7e2365d12b8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 04:45:04.883998 22026 tablet_service.cc:1514] Processing DeleteTablet for tablet 76677e9e7f5540e8a4ed49fa19e1a918 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 04:45:04 UTC) from {username='kudu', principal='kudu/127.22.167.126@KRBTEST.COM'} at 127.0.0.1:56264
I20260810 04:45:04.884394 22422 ts_tablet_manager.cc:1905] T 76677e9e7f5540e8a4ed49fa19e1a918 P e0665795199545e38e85b7e2365d12b8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 04:45:04.884603 22423 tablet_replica.cc:330] T 76677e9e7f5540e8a4ed49fa19e1a918 P 5044e6aa6adc4262820f968cdec644e0: stopping tablet replica
I20260810 04:45:04.884857 22423 raft_consensus.cc:2229] T 76677e9e7f5540e8a4ed49fa19e1a918 P 5044e6aa6adc4262820f968cdec644e0 [term 1 LEADER]: Raft consensus shutting down.
I20260810 04:45:04.885218 22423 raft_consensus.cc:2258] T 76677e9e7f5540e8a4ed49fa19e1a918 P 5044e6aa6adc4262820f968cdec644e0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 04:45:04.886204 22423 ts_tablet_manager.cc:1905] T 76677e9e7f5540e8a4ed49fa19e1a918 P 5044e6aa6adc4262820f968cdec644e0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 04:45:04.887422 22422 ts_tablet_manager.cc:1918] T 76677e9e7f5540e8a4ed49fa19e1a918 P e0665795199545e38e85b7e2365d12b8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 04:45:04.887580 22422 log.cc:1196] T 76677e9e7f5540e8a4ed49fa19e1a918 P e0665795199545e38e85b7e2365d12b8: Deleting WAL directory at /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786336863388938-23197-0/minicluster-data/ts-2/wal/wals/76677e9e7f5540e8a4ed49fa19e1a918
I20260810 04:45:04.888084 22422 ts_tablet_manager.cc:1939] T 76677e9e7f5540e8a4ed49fa19e1a918 P e0665795199545e38e85b7e2365d12b8: Deleting consensus metadata
I20260810 04:45:04.888803 21559 catalog_manager.cc:4841] TS e0665795199545e38e85b7e2365d12b8 (127.22.167.67:33019): tablet 76677e9e7f5540e8a4ed49fa19e1a918 (table test-table [id=be4121cc12b54615974ac33835460140]) successfully deleted
I20260810 04:45:04.889683 22423 ts_tablet_manager.cc:1918] T 76677e9e7f5540e8a4ed49fa19e1a918 P 5044e6aa6adc4262820f968cdec644e0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 04:45:04.889861 22423 log.cc:1196] T 76677e9e7f5540e8a4ed49fa19e1a918 P 5044e6aa6adc4262820f968cdec644e0: Deleting WAL directory at /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786336863388938-23197-0/minicluster-data/ts-1/wal/wals/76677e9e7f5540e8a4ed49fa19e1a918
I20260810 04:45:04.890340 22423 ts_tablet_manager.cc:1939] T 76677e9e7f5540e8a4ed49fa19e1a918 P 5044e6aa6adc4262820f968cdec644e0: Deleting consensus metadata
I20260810 04:45:04.891036 21557 catalog_manager.cc:4841] TS 5044e6aa6adc4262820f968cdec644e0 (127.22.167.66:41809): tablet 76677e9e7f5540e8a4ed49fa19e1a918 (table test-table [id=be4121cc12b54615974ac33835460140]) successfully deleted
W20260810 04:45:04.892688 21556 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.22.167.65:46489 (error 108)
I20260810 04:45:04.893095 23197 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskemv_AM/build/release/bin/kudu with pid 21903
I20260810 04:45:04.896162 22374 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:45:04.893250 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.65:46489 (local address 127.0.0.1:36946)
0810 04:45:04.896025 (+ 2775us) negotiation.cc:107] Waiting for socket to connect
0810 04:45:04.896114 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.167.65:46489: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":2738}
W20260810 04:45:04.896461 21556 catalog_manager.cc:4584] TS 538d5f775d8743ee9d49788304425318 (127.22.167.65:46489): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 76677e9e7f5540e8a4ed49fa19e1a918: Network error: Client connection negotiation failed: client connection to 127.22.167.65:46489: connect: Connection refused (error 111)
I20260810 04:45:04.903822 23197 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskemv_AM/build/release/bin/kudu with pid 22134
I20260810 04:45:04.919250 23197 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskemv_AM/build/release/bin/kudu with pid 21517
2026-08-10T04:45:04Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/2 (2526 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/3
2026-08-10T04:45:04Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T04:45:04Z Disabled control of system clock
I20260810 04:45:04.966519 23197 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskemv_AM/build/release/bin/kudu
/tmp/dist-test-taskemv_AM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786336863388938-23197-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786336863388938-23197-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786336863388938-23197-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786336863388938-23197-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.167.126:37355
--webserver_interface=127.22.167.126
--webserver_port=0
--builtin_ntp_servers=127.22.167.84:35229
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.167.126:37355
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260810 04:45:05.111116 22430 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 04:45:05.111493 22430 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 04:45:05.111527 22430 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 04:45:05.113461 22430 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 04:45:05.113538 22430 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 04:45:05.113555 22430 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 04:45:05.113607 22430 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 04:45:05.113648 22430 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 04:45:05.115947 22430 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.167.84:35229
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786336863388938-23197-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786336863388938-23197-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.22.167.126:37355
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.167.126:37355
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786336863388938-23197-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.167.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22430
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786336863388938-23197-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 03:54:01 UTC on ccycloud.quasar-icijmt.root.comops.site
build id 4577009
I20260810 04:45:05.116379 22430 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 04:45:05.116940 22430 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 04:45:05.124675 22441 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 04:45:05.124717 22439 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 04:45:05.124717 22443 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 04:45:05.125259 22430 server_base.cc:1034] running on GCE node
I20260810 04:45:05.125711 22430 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 04:45:05.126282 22430 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 04:45:05.127456 22430 hybrid_clock.cc:648] HybridClock initialized: now 1786337105127431 us; error 44 us; skew 500 ppm
I20260810 04:45:05.129380 22430 webserver.cc:460] Webserver started at http://127.22.167.126:36737/ using document root <none> and password file <none>
I20260810 04:45:05.129778 22430 fs_manager.cc:359] Metadata directory not provided
I20260810 04:45:05.129878 22430 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 04:45:05.130117 22430 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 04:45:05.131891 22430 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786336863388938-23197-0/minicluster-data/master-0/data/instance:
uuid: "18dabf510a4b471bae892f37eef36188"
format_stamp: "Formatted at 2026-08-10 04:45:05 on dist-test-slave-bqcl"
server_key: "1b1ff4f008960b8fe23f357cb15d35f9"
server_key_iv: "f688ba37288d245e4328ab588cb570c1"
server_key_version: "encryptionkey@0"
I20260810 04:45:05.132454 22430 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786336863388938-23197-0/minicluster-data/master-0/wal/instance:
uuid: "18dabf510a4b471bae892f37eef36188"
format_stamp: "Formatted at 2026-08-10 04:45:05 on dist-test-slave-bqcl"
server_key: "1b1ff4f008960b8fe23f357cb15d35f9"
server_key_iv: "f688ba37288d245e4328ab588cb570c1"
server_key_version: "encryptionkey@0"
I20260810 04:45:05.134671 22430 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.004s sys 0.000s
I20260810 04:45:05.135845 22454 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 04:45:05.136085 22430 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 04:45:05.136233 22430 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786336863388938-23197-0/minicluster-data/master-0/data,/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786336863388938-23197-0/minicluster-data/master-0/wal
uuid: "18dabf510a4b471bae892f37eef36188"
format_stamp: "Formatted at 2026-08-10 04:45:05 on dist-test-slave-bqcl"
server_key: "1b1ff4f008960b8fe23f357cb15d35f9"
server_key_iv: "f688ba37288d245e4328ab588cb570c1"
server_key_version: "encryptionkey@0"
I20260810 04:45:05.136346 22430 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786336863388938-23197-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786336863388938-23197-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786336863388938-23197-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 04:45:05.147094 22430 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 04:45:05.147532 22430 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 04:45:05.147759 22430 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 04:45:05.154340 22430 rpc_server.cc:307] RPC server started. Bound to: 127.22.167.126:37355
I20260810 04:45:05.154385 22551 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.167.126:37355 every 8 connection(s)
I20260810 04:45:05.154923 22430 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786336863388938-23197-0/minicluster-data/master-0/data/info.pb
I20260810 04:45:05.156546 22554 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 04:45:05.160349 22554 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 18dabf510a4b471bae892f37eef36188: Bootstrap starting.
I20260810 04:45:05.161360 22554 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 18dabf510a4b471bae892f37eef36188: Neither blocks nor log segments found. Creating new log.
I20260810 04:45:05.161852 22554 log.cc:824] T 00000000000000000000000000000000 P 18dabf510a4b471bae892f37eef36188: Log is configured to *not* fsync() on all Append() calls
I20260810 04:45:05.162870 22554 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 18dabf510a4b471bae892f37eef36188: No bootstrap required, opened a new log
I20260810 04:45:05.164430 23197 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskemv_AM/build/release/bin/kudu as pid 22430
I20260810 04:45:05.164624 23197 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786336863388938-23197-0/minicluster-data/master-0/wal/instance
I20260810 04:45:05.164999 23197 external_mini_cluster.cc:1428] Setting key 3135deda22bc21a5c8151f569b771fd3
I20260810 04:45:05.166441 22554 raft_consensus.cc:348] T 00000000000000000000000000000000 P 18dabf510a4b471bae892f37eef36188 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "18dabf510a4b471bae892f37eef36188" member_type: VOTER last_known_addr { host: "127.22.167.126" port: 37355 } }
I20260810 04:45:05.166723 22554 raft_consensus.cc:374] T 00000000000000000000000000000000 P 18dabf510a4b471bae892f37eef36188 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 04:45:05.166817 22554 raft_consensus.cc:729] T 00000000000000000000000000000000 P 18dabf510a4b471bae892f37eef36188 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 18dabf510a4b471bae892f37eef36188, State: Initialized, Role: FOLLOWER
I20260810 04:45:05.167057 22554 consensus_queue.cc:260] T 00000000000000000000000000000000 P 18dabf510a4b471bae892f37eef36188 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "18dabf510a4b471bae892f37eef36188" member_type: VOTER last_known_addr { host: "127.22.167.126" port: 37355 } }
I20260810 04:45:05.167182 22554 raft_consensus.cc:388] T 00000000000000000000000000000000 P 18dabf510a4b471bae892f37eef36188 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 04:45:05.167294 22554 raft_consensus.cc:482] T 00000000000000000000000000000000 P 18dabf510a4b471bae892f37eef36188 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 04:45:05.167382 22554 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 18dabf510a4b471bae892f37eef36188 [term 0 FOLLOWER]: Advancing to term 1
I20260810 04:45:05.168196 22554 raft_consensus.cc:504] T 00000000000000000000000000000000 P 18dabf510a4b471bae892f37eef36188 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "18dabf510a4b471bae892f37eef36188" member_type: VOTER last_known_addr { host: "127.22.167.126" port: 37355 } }
I20260810 04:45:05.168351 22554 leader_election.cc:302] T 00000000000000000000000000000000 P 18dabf510a4b471bae892f37eef36188 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 18dabf510a4b471bae892f37eef36188; no voters:
I20260810 04:45:05.168602 22554 leader_election.cc:288] T 00000000000000000000000000000000 P 18dabf510a4b471bae892f37eef36188 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 04:45:05.168665 22561 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 18dabf510a4b471bae892f37eef36188 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 04:45:05.168869 22561 raft_consensus.cc:686] T 00000000000000000000000000000000 P 18dabf510a4b471bae892f37eef36188 [term 1 LEADER]: Becoming Leader. State: Replica: 18dabf510a4b471bae892f37eef36188, State: Running, Role: LEADER
I20260810 04:45:05.169062 22554 sys_catalog.cc:564] T 00000000000000000000000000000000 P 18dabf510a4b471bae892f37eef36188 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 04:45:05.169010 22561 consensus_queue.cc:237] T 00000000000000000000000000000000 P 18dabf510a4b471bae892f37eef36188 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "18dabf510a4b471bae892f37eef36188" member_type: VOTER last_known_addr { host: "127.22.167.126" port: 37355 } }
I20260810 04:45:05.169586 22561 sys_catalog.cc:455] T 00000000000000000000000000000000 P 18dabf510a4b471bae892f37eef36188 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 18dabf510a4b471bae892f37eef36188. Latest consensus state: current_term: 1 leader_uuid: "18dabf510a4b471bae892f37eef36188" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "18dabf510a4b471bae892f37eef36188" member_type: VOTER last_known_addr { host: "127.22.167.126" port: 37355 } } }
I20260810 04:45:05.169734 22561 sys_catalog.cc:458] T 00000000000000000000000000000000 P 18dabf510a4b471bae892f37eef36188 [sys.catalog]: This master's current role is: LEADER
I20260810 04:45:05.170282 22568 sys_catalog.cc:455] T 00000000000000000000000000000000 P 18dabf510a4b471bae892f37eef36188 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "18dabf510a4b471bae892f37eef36188" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "18dabf510a4b471bae892f37eef36188" member_type: VOTER last_known_addr { host: "127.22.167.126" port: 37355 } } }
I20260810 04:45:05.170383 22568 sys_catalog.cc:458] T 00000000000000000000000000000000 P 18dabf510a4b471bae892f37eef36188 [sys.catalog]: This master's current role is: LEADER
I20260810 04:45:05.171118 22573 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 04:45:05.172001 22573 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 04:45:05.175004 22573 catalog_manager.cc:1347] Generated new cluster ID: 96ff57fde2054615898ddec46c4d5285
I20260810 04:45:05.175096 22573 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 04:45:05.175374 22558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:45:05.165835 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60610 (local address 127.22.167.126:37355)
0810 04:45:05.166105 (+ 270us) server_negotiation.cc:206] Beginning negotiation
0810 04:45:05.166154 (+ 49us) server_negotiation.cc:399] Waiting for connection header
0810 04:45:05.166213 (+ 59us) server_negotiation.cc:407] Connection header received
0810 04:45:05.166374 (+ 161us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:45:05.166380 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:45:05.166463 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:45:05.166628 (+ 165us) server_negotiation.cc:226] Negotiated authn=SASL
0810 04:45:05.167037 (+ 409us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:45:05.169473 (+ 2436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:45:05.171297 (+ 1824us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:45:05.172570 (+ 1273us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:45:05.172786 (+ 216us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:45:05.173927 (+ 1141us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 04:45:05.173942 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 04:45:05.173943 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 04:45:05.174322 (+ 379us) server_negotiation.cc:922] calling sasl_server_start()
0810 04:45:05.174390 (+ 68us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 04:45:05.174413 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 04:45:05.174543 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0810 04:45:05.174678 (+ 135us) server_negotiation.cc:299] Negotiation successful
0810 04:45:05.174820 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":168,"thread_start_us":103,"threads_started":1}
I20260810 04:45:05.192610 22573 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 04:45:05.193418 22573 catalog_manager.cc:1504] Loading token signing keys...
I20260810 04:45:05.199581 22573 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 18dabf510a4b471bae892f37eef36188: Generated new TSK 0
I20260810 04:45:05.199836 22573 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 04:45:05.207683 23197 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskemv_AM/build/release/bin/kudu
/tmp/dist-test-taskemv_AM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786336863388938-23197-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786336863388938-23197-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786336863388938-23197-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786336863388938-23197-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.167.65:0
--local_ip_for_outbound_sockets=127.22.167.65
--webserver_interface=127.22.167.65
--webserver_port=0
--tserver_master_addrs=127.22.167.126:37355
--builtin_ntp_servers=127.22.167.84:35229
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260810 04:45:05.310889 22592 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 04:45:05.311100 22592 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 04:45:05.311131 22592 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 04:45:05.312259 22592 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 04:45:05.312299 22592 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 04:45:05.312322 22592 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.167.65
I20260810 04:45:05.313619 22592 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.167.84:35229
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786336863388938-23197-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786336863388938-23197-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.167.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786336863388938-23197-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.22.167.65
--webserver_port=0
--tserver_master_addrs=127.22.167.126:37355
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22592
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.167.65
--log_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786336863388938-23197-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 03:54:01 UTC on ccycloud.quasar-icijmt.root.comops.site
build id 4577009
I20260810 04:45:05.313841 22592 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 04:45:05.314143 22592 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 04:45:05.318813 22607 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 04:45:05.318873 22606 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 04:45:05.318905 22614 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 04:45:05.319594 22592 server_base.cc:1034] running on GCE node
I20260810 04:45:05.319802 22592 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 04:45:05.320032 22592 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 04:45:05.321178 22592 hybrid_clock.cc:648] HybridClock initialized: now 1786337105321164 us; error 31 us; skew 500 ppm
I20260810 04:45:05.322441 22592 webserver.cc:460] Webserver started at http://127.22.167.65:37819/ using document root <none> and password file <none>
I20260810 04:45:05.322671 22592 fs_manager.cc:359] Metadata directory not provided
I20260810 04:45:05.322721 22592 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 04:45:05.322850 22592 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 04:45:05.323863 22592 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786336863388938-23197-0/minicluster-data/ts-0/data/instance:
uuid: "672ea98aeed649adbe8fe8dd2d5f6326"
format_stamp: "Formatted at 2026-08-10 04:45:05 on dist-test-slave-bqcl"
server_key: "519a6d363aaa6f407c4418664e27cd8a"
server_key_iv: "acac502401c1d83e7f1a09c62d8dd7ab"
server_key_version: "encryptionkey@0"
I20260810 04:45:05.324193 22592 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786336863388938-23197-0/minicluster-data/ts-0/wal/instance:
uuid: "672ea98aeed649adbe8fe8dd2d5f6326"
format_stamp: "Formatted at 2026-08-10 04:45:05 on dist-test-slave-bqcl"
server_key: "519a6d363aaa6f407c4418664e27cd8a"
server_key_iv: "acac502401c1d83e7f1a09c62d8dd7ab"
server_key_version: "encryptionkey@0"
I20260810 04:45:05.325428 22592 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260810 04:45:05.326115 22624 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 04:45:05.326326 22592 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260810 04:45:05.326421 22592 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786336863388938-23197-0/minicluster-data/ts-0/data,/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786336863388938-23197-0/minicluster-data/ts-0/wal
uuid: "672ea98aeed649adbe8fe8dd2d5f6326"
format_stamp: "Formatted at 2026-08-10 04:45:05 on dist-test-slave-bqcl"
server_key: "519a6d363aaa6f407c4418664e27cd8a"
server_key_iv: "acac502401c1d83e7f1a09c62d8dd7ab"
server_key_version: "encryptionkey@0"
I20260810 04:45:05.326489 22592 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786336863388938-23197-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786336863388938-23197-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786336863388938-23197-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 04:45:05.337683 22592 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 04:45:05.337940 22592 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 04:45:05.338040 22592 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 04:45:05.338260 22592 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 04:45:05.338560 22592 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 04:45:05.338591 22592 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 04:45:05.338619 22592 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 04:45:05.338636 22592 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 04:45:05.344257 22592 rpc_server.cc:307] RPC server started. Bound to: 127.22.167.65:33403
I20260810 04:45:05.344324 22831 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.167.65:33403 every 8 connection(s)
I20260810 04:45:05.344624 22592 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786336863388938-23197-0/minicluster-data/ts-0/data/info.pb
I20260810 04:45:05.346269 23197 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskemv_AM/build/release/bin/kudu as pid 22592
I20260810 04:45:05.346350 23197 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786336863388938-23197-0/minicluster-data/ts-0/wal/instance
I20260810 04:45:05.346518 23197 external_mini_cluster.cc:1428] Setting key 7bb0471c1080456a566e324c640de7a0
I20260810 04:45:05.347394 23197 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskemv_AM/build/release/bin/kudu
/tmp/dist-test-taskemv_AM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786336863388938-23197-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786336863388938-23197-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786336863388938-23197-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786336863388938-23197-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.167.66:0
--local_ip_for_outbound_sockets=127.22.167.66
--webserver_interface=127.22.167.66
--webserver_port=0
--tserver_master_addrs=127.22.167.126:37355
--builtin_ntp_servers=127.22.167.84:35229
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260810 04:45:05.349912 22558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:45:05.345317 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.167.65:56657 (local address 127.22.167.126:37355)
0810 04:45:05.345374 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0810 04:45:05.345376 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 04:45:05.345605 (+ 229us) server_negotiation.cc:407] Connection header received
0810 04:45:05.346098 (+ 493us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:45:05.346100 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:45:05.346114 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:45:05.346181 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0810 04:45:05.346906 (+ 725us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:45:05.347403 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:45:05.348444 (+ 1041us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:45:05.348657 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:45:05.348719 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:45:05.349668 (+ 949us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 04:45:05.349670 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 04:45:05.349670 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 04:45:05.349674 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0810 04:45:05.349691 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 04:45:05.349695 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 04:45:05.349747 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0810 04:45:05.349830 (+ 83us) server_negotiation.cc:299] Negotiation successful
0810 04:45:05.349854 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260810 04:45:05.350191 22836 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:45:05.345302 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.126:37355 (local address 127.22.167.65:56657)
0810 04:45:05.345492 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0810 04:45:05.345526 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0810 04:45:05.346003 (+ 477us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 04:45:05.346261 (+ 258us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 04:45:05.346265 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 04:45:05.346273 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0810 04:45:05.346821 (+ 548us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:45:05.346827 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:45:05.347573 (+ 746us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:45:05.347573 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:45:05.348363 (+ 790us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:45:05.348366 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:45:05.348796 (+ 430us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:45:05.348797 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:45:05.348969 (+ 172us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:45:05.349512 (+ 543us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 04:45:05.349513 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0810 04:45:05.349552 (+ 39us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 04:45:05.349581 (+ 29us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 04:45:05.349757 (+ 176us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 04:45:05.349761 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 04:45:05.349765 (+ 4us) client_negotiation.cc:769] Sending connection context
0810 04:45:05.349833 (+ 68us) client_negotiation.cc:240] Negotiation successful
0810 04:45:05.349947 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":90,"threads_started":1}
I20260810 04:45:05.350699 22832 heartbeater.cc:347] Connected to a master server at 127.22.167.126:37355
I20260810 04:45:05.350790 22832 heartbeater.cc:464] Registering TS with master...
I20260810 04:45:05.350962 22832 heartbeater.cc:511] Master 127.22.167.126:37355 requested a full tablet report, sending...
I20260810 04:45:05.351438 22485 ts_manager.cc:194] Registered new tserver with Master: 672ea98aeed649adbe8fe8dd2d5f6326 (127.22.167.65:33403)
I20260810 04:45:05.352030 22485 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.22.167.65:56657
W20260810 04:45:05.428030 22837 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 04:45:05.428234 22837 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 04:45:05.428262 22837 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 04:45:05.429376 22837 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 04:45:05.429414 22837 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 04:45:05.429446 22837 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.167.66
I20260810 04:45:05.430817 22837 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.167.84:35229
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786336863388938-23197-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786336863388938-23197-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.167.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786336863388938-23197-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.22.167.66
--webserver_port=0
--tserver_master_addrs=127.22.167.126:37355
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22837
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.167.66
--log_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786336863388938-23197-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 03:54:01 UTC on ccycloud.quasar-icijmt.root.comops.site
build id 4577009
I20260810 04:45:05.431034 22837 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 04:45:05.431337 22837 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 04:45:05.436110 22843 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 04:45:05.436103 22850 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 04:45:05.436100 22846 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 04:45:05.436245 22837 server_base.cc:1034] running on GCE node
I20260810 04:45:05.436496 22837 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 04:45:05.436717 22837 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 04:45:05.437865 22837 hybrid_clock.cc:648] HybridClock initialized: now 1786337105437844 us; error 31 us; skew 500 ppm
I20260810 04:45:05.439267 22837 webserver.cc:460] Webserver started at http://127.22.167.66:45605/ using document root <none> and password file <none>
I20260810 04:45:05.439447 22837 fs_manager.cc:359] Metadata directory not provided
I20260810 04:45:05.439487 22837 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 04:45:05.439581 22837 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 04:45:05.440518 22837 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786336863388938-23197-0/minicluster-data/ts-1/data/instance:
uuid: "0c59408e485e4e46ac4849c67b9ea749"
format_stamp: "Formatted at 2026-08-10 04:45:05 on dist-test-slave-bqcl"
server_key: "733124a52279549b6f1c3d038fdc66a1"
server_key_iv: "c39099351bdaed4077517aedb03754e2"
server_key_version: "encryptionkey@0"
I20260810 04:45:05.440851 22837 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786336863388938-23197-0/minicluster-data/ts-1/wal/instance:
uuid: "0c59408e485e4e46ac4849c67b9ea749"
format_stamp: "Formatted at 2026-08-10 04:45:05 on dist-test-slave-bqcl"
server_key: "733124a52279549b6f1c3d038fdc66a1"
server_key_iv: "c39099351bdaed4077517aedb03754e2"
server_key_version: "encryptionkey@0"
I20260810 04:45:05.441992 22837 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260810 04:45:05.442692 22861 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 04:45:05.442839 22837 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260810 04:45:05.442912 22837 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786336863388938-23197-0/minicluster-data/ts-1/data,/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786336863388938-23197-0/minicluster-data/ts-1/wal
uuid: "0c59408e485e4e46ac4849c67b9ea749"
format_stamp: "Formatted at 2026-08-10 04:45:05 on dist-test-slave-bqcl"
server_key: "733124a52279549b6f1c3d038fdc66a1"
server_key_iv: "c39099351bdaed4077517aedb03754e2"
server_key_version: "encryptionkey@0"
I20260810 04:45:05.442981 22837 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786336863388938-23197-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786336863388938-23197-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786336863388938-23197-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 04:45:05.451172 22837 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 04:45:05.451404 22837 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 04:45:05.451521 22837 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 04:45:05.451721 22837 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 04:45:05.452005 22837 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 04:45:05.452036 22837 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 04:45:05.452056 22837 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 04:45:05.452080 22837 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 04:45:05.457305 22837 rpc_server.cc:307] RPC server started. Bound to: 127.22.167.66:45977
I20260810 04:45:05.457376 23035 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.167.66:45977 every 8 connection(s)
I20260810 04:45:05.457623 22837 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786336863388938-23197-0/minicluster-data/ts-1/data/info.pb
I20260810 04:45:05.462250 22558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:45:05.458298 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.167.66:43995 (local address 127.22.167.126:37355)
0810 04:45:05.458346 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0810 04:45:05.458348 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 04:45:05.458582 (+ 234us) server_negotiation.cc:407] Connection header received
0810 04:45:05.459007 (+ 425us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:45:05.459008 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:45:05.459019 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:45:05.459079 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0810 04:45:05.459710 (+ 631us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:45:05.460206 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:45:05.461104 (+ 898us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:45:05.461302 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:45:05.461366 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:45:05.461995 (+ 629us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 04:45:05.461997 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 04:45:05.461997 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 04:45:05.462001 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0810 04:45:05.462016 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 04:45:05.462019 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 04:45:05.462064 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0810 04:45:05.462150 (+ 86us) server_negotiation.cc:299] Negotiation successful
0810 04:45:05.462178 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260810 04:45:05.462409 23039 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:45:05.458301 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.126:37355 (local address 127.22.167.66:43995)
0810 04:45:05.458471 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0810 04:45:05.458503 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0810 04:45:05.458928 (+ 425us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 04:45:05.459130 (+ 202us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 04:45:05.459133 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 04:45:05.459141 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0810 04:45:05.459616 (+ 475us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:45:05.459620 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:45:05.460282 (+ 662us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:45:05.460282 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:45:05.461022 (+ 740us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:45:05.461024 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:45:05.461371 (+ 347us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:45:05.461371 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:45:05.461470 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:45:05.461876 (+ 406us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 04:45:05.461876 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0810 04:45:05.461902 (+ 26us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 04:45:05.461921 (+ 19us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 04:45:05.462069 (+ 148us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 04:45:05.462071 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 04:45:05.462073 (+ 2us) client_negotiation.cc:769] Sending connection context
0810 04:45:05.462116 (+ 43us) client_negotiation.cc:240] Negotiation successful
0810 04:45:05.462237 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":87,"threads_started":1}
I20260810 04:45:05.462824 23036 heartbeater.cc:347] Connected to a master server at 127.22.167.126:37355
I20260810 04:45:05.462924 23036 heartbeater.cc:464] Registering TS with master...
I20260810 04:45:05.463081 23036 heartbeater.cc:511] Master 127.22.167.126:37355 requested a full tablet report, sending...
I20260810 04:45:05.463066 23197 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskemv_AM/build/release/bin/kudu as pid 22837
I20260810 04:45:05.463145 23197 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786336863388938-23197-0/minicluster-data/ts-1/wal/instance
I20260810 04:45:05.463348 23197 external_mini_cluster.cc:1428] Setting key 591b0e8f08537eb145361729a5f64c8b
I20260810 04:45:05.463429 22485 ts_manager.cc:194] Registered new tserver with Master: 0c59408e485e4e46ac4849c67b9ea749 (127.22.167.66:45977)
I20260810 04:45:05.463881 22485 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.22.167.66:43995
I20260810 04:45:05.464334 23197 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskemv_AM/build/release/bin/kudu
/tmp/dist-test-taskemv_AM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786336863388938-23197-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786336863388938-23197-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786336863388938-23197-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786336863388938-23197-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.167.67:0
--local_ip_for_outbound_sockets=127.22.167.67
--webserver_interface=127.22.167.67
--webserver_port=0
--tserver_master_addrs=127.22.167.126:37355
--builtin_ntp_servers=127.22.167.84:35229
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260810 04:45:05.546636 23040 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 04:45:05.546830 23040 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 04:45:05.546854 23040 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 04:45:05.547914 23040 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 04:45:05.547955 23040 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 04:45:05.547986 23040 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.167.67
I20260810 04:45:05.549265 23040 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.167.84:35229
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786336863388938-23197-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786336863388938-23197-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.167.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786336863388938-23197-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.22.167.67
--webserver_port=0
--tserver_master_addrs=127.22.167.126:37355
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23040
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.167.67
--log_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786336863388938-23197-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 03:54:01 UTC on ccycloud.quasar-icijmt.root.comops.site
build id 4577009
I20260810 04:45:05.549443 23040 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 04:45:05.549691 23040 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 04:45:05.554360 23047 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 04:45:05.554427 23040 server_base.cc:1034] running on GCE node
W20260810 04:45:05.554369 23049 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 04:45:05.554369 23052 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 04:45:05.554755 23040 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 04:45:05.554978 23040 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 04:45:05.556129 23040 hybrid_clock.cc:648] HybridClock initialized: now 1786337105556113 us; error 33 us; skew 500 ppm
I20260810 04:45:05.557302 23040 webserver.cc:460] Webserver started at http://127.22.167.67:34559/ using document root <none> and password file <none>
I20260810 04:45:05.557515 23040 fs_manager.cc:359] Metadata directory not provided
I20260810 04:45:05.557567 23040 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 04:45:05.557694 23040 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 04:45:05.558708 23040 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786336863388938-23197-0/minicluster-data/ts-2/data/instance:
uuid: "8923b09535f64d0ba8715c728df6c40b"
format_stamp: "Formatted at 2026-08-10 04:45:05 on dist-test-slave-bqcl"
server_key: "690d36cbdda683eb63fedb3b4f7083a6"
server_key_iv: "3cef0b3680ac9ed8e275bbb38733d0e5"
server_key_version: "encryptionkey@0"
I20260810 04:45:05.559028 23040 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786336863388938-23197-0/minicluster-data/ts-2/wal/instance:
uuid: "8923b09535f64d0ba8715c728df6c40b"
format_stamp: "Formatted at 2026-08-10 04:45:05 on dist-test-slave-bqcl"
server_key: "690d36cbdda683eb63fedb3b4f7083a6"
server_key_iv: "3cef0b3680ac9ed8e275bbb38733d0e5"
server_key_version: "encryptionkey@0"
I20260810 04:45:05.560158 23040 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260810 04:45:05.560760 23060 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 04:45:05.560945 23040 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260810 04:45:05.561034 23040 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786336863388938-23197-0/minicluster-data/ts-2/data,/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786336863388938-23197-0/minicluster-data/ts-2/wal
uuid: "8923b09535f64d0ba8715c728df6c40b"
format_stamp: "Formatted at 2026-08-10 04:45:05 on dist-test-slave-bqcl"
server_key: "690d36cbdda683eb63fedb3b4f7083a6"
server_key_iv: "3cef0b3680ac9ed8e275bbb38733d0e5"
server_key_version: "encryptionkey@0"
I20260810 04:45:05.561100 23040 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786336863388938-23197-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786336863388938-23197-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786336863388938-23197-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 04:45:05.579231 23040 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 04:45:05.579484 23040 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 04:45:05.579617 23040 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 04:45:05.579816 23040 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 04:45:05.580117 23040 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 04:45:05.580159 23040 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 04:45:05.580188 23040 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 04:45:05.580209 23040 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 04:45:05.585600 23040 rpc_server.cc:307] RPC server started. Bound to: 127.22.167.67:45781
I20260810 04:45:05.585639 23255 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.167.67:45781 every 8 connection(s)
I20260810 04:45:05.585939 23040 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786336863388938-23197-0/minicluster-data/ts-2/data/info.pb
I20260810 04:45:05.589603 23197 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskemv_AM/build/release/bin/kudu as pid 23040
I20260810 04:45:05.589706 23197 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786336863388938-23197-0/minicluster-data/ts-2/wal/instance
I20260810 04:45:05.589895 23197 external_mini_cluster.cc:1428] Setting key 43271ce1f78ca9c149d4f111655aa98c
I20260810 04:45:05.591022 22558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:45:05.586702 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.167.67:53401 (local address 127.22.167.126:37355)
0810 04:45:05.586763 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0810 04:45:05.586765 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 04:45:05.586970 (+ 205us) server_negotiation.cc:407] Connection header received
0810 04:45:05.587469 (+ 499us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:45:05.587470 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:45:05.587483 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:45:05.587534 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0810 04:45:05.588182 (+ 648us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:45:05.588684 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:45:05.589674 (+ 990us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:45:05.589916 (+ 242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:45:05.589975 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:45:05.590759 (+ 784us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 04:45:05.590760 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 04:45:05.590761 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 04:45:05.590766 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0810 04:45:05.590782 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 04:45:05.590786 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 04:45:05.590833 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0810 04:45:05.590911 (+ 78us) server_negotiation.cc:299] Negotiation successful
0810 04:45:05.590942 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260810 04:45:05.591212 23260 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:45:05.586677 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.126:37355 (local address 127.22.167.67:53401)
0810 04:45:05.586858 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0810 04:45:05.586890 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0810 04:45:05.587370 (+ 480us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 04:45:05.587603 (+ 233us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 04:45:05.587606 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 04:45:05.587614 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0810 04:45:05.588089 (+ 475us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:45:05.588094 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:45:05.588766 (+ 672us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:45:05.588767 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:45:05.589579 (+ 812us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:45:05.589582 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:45:05.589994 (+ 412us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:45:05.589995 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:45:05.590115 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:45:05.590630 (+ 515us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 04:45:05.590630 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0810 04:45:05.590658 (+ 28us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 04:45:05.590682 (+ 24us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 04:45:05.590839 (+ 157us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 04:45:05.590844 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 04:45:05.590847 (+ 3us) client_negotiation.cc:769] Sending connection context
0810 04:45:05.590908 (+ 61us) client_negotiation.cc:240] Negotiation successful
0810 04:45:05.591004 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":78,"threads_started":1}
I20260810 04:45:05.591631 23258 heartbeater.cc:347] Connected to a master server at 127.22.167.126:37355
I20260810 04:45:05.591713 23258 heartbeater.cc:464] Registering TS with master...
I20260810 04:45:05.591857 23258 heartbeater.cc:511] Master 127.22.167.126:37355 requested a full tablet report, sending...
I20260810 04:45:05.592185 22485 ts_manager.cc:194] Registered new tserver with Master: 8923b09535f64d0ba8715c728df6c40b (127.22.167.67:45781)
I20260810 04:45:05.592506 22485 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.22.167.67:53401
I20260810 04:45:05.601931 23197 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260810 04:45:05.606155 22558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:45:05.603391 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60614 (local address 127.22.167.126:37355)
0810 04:45:05.603435 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0810 04:45:05.603437 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 04:45:05.603533 (+ 96us) server_negotiation.cc:407] Connection header received
0810 04:45:05.603597 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:45:05.603597 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:45:05.603608 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:45:05.603654 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0810 04:45:05.603818 (+ 164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:45:05.604274 (+ 456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:45:05.604992 (+ 718us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:45:05.605180 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:45:05.605243 (+ 63us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:45:05.605880 (+ 637us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 04:45:05.605881 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 04:45:05.605882 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 04:45:05.605886 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0810 04:45:05.605900 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 04:45:05.605904 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 04:45:05.605979 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0810 04:45:05.606049 (+ 70us) server_negotiation.cc:299] Negotiation successful
0810 04:45:05.606059 (+ 10us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260810 04:45:05.607722 22485 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60614:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 04:45:05.609198 22485 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 04:45:05.615113 23268 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:45:05.611126 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.65:33403 (local address 127.0.0.1:46656)
0810 04:45:05.611247 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0810 04:45:05.611289 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0810 04:45:05.611363 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 04:45:05.611662 (+ 299us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 04:45:05.611665 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 04:45:05.611668 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 04:45:05.611736 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:45:05.611738 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:45:05.612861 (+ 1123us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:45:05.612862 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:45:05.614045 (+ 1183us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:45:05.614049 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:45:05.614809 (+ 760us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:45:05.614810 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:45:05.614890 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:45:05.614903 (+ 13us) client_negotiation.cc:769] Sending connection context
0810 04:45:05.614979 (+ 76us) client_negotiation.cc:240] Negotiation successful
0810 04:45:05.615032 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":96,"mutex_wait_us":1,"thread_start_us":58,"threads_started":1}
I20260810 04:45:05.615130 23269 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:45:05.611229 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46656 (local address 127.22.167.65:33403)
0810 04:45:05.611451 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0810 04:45:05.611456 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 04:45:05.611466 (+ 10us) server_negotiation.cc:407] Connection header received
0810 04:45:05.611519 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:45:05.611521 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:45:05.611538 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:45:05.611822 (+ 284us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 04:45:05.611889 (+ 67us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:45:05.612776 (+ 887us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:45:05.614174 (+ 1398us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:45:05.614725 (+ 551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:45:05.614799 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:45:05.614890 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0810 04:45:05.615008 (+ 118us) server_negotiation.cc:299] Negotiation successful
0810 04:45:05.615047 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":185,"thread_start_us":101,"threads_started":1}
I20260810 04:45:05.615357 23272 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:45:05.611444 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.67:45781 (local address 127.0.0.1:54286)
0810 04:45:05.611652 (+ 208us) negotiation.cc:107] Waiting for socket to connect
0810 04:45:05.611662 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0810 04:45:05.611709 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 04:45:05.611949 (+ 240us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 04:45:05.611949 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 04:45:05.611951 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 04:45:05.612079 (+ 128us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:45:05.612082 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:45:05.613231 (+ 1149us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:45:05.613232 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:45:05.613978 (+ 746us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:45:05.613981 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:45:05.614930 (+ 949us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:45:05.614931 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:45:05.614992 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:45:05.614997 (+ 5us) client_negotiation.cc:769] Sending connection context
0810 04:45:05.615044 (+ 47us) client_negotiation.cc:240] Negotiation successful
0810 04:45:05.615083 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":191,"thread_start_us":60,"threads_started":1}
I20260810 04:45:05.615437 23270 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:45:05.611288 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.66:45977 (local address 127.0.0.1:37774)
0810 04:45:05.611596 (+ 308us) negotiation.cc:107] Waiting for socket to connect
0810 04:45:05.611606 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0810 04:45:05.611682 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 04:45:05.611896 (+ 214us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 04:45:05.611897 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 04:45:05.611899 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 04:45:05.611936 (+ 37us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:45:05.611938 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:45:05.613203 (+ 1265us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:45:05.613204 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:45:05.614278 (+ 1074us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:45:05.614281 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:45:05.615226 (+ 945us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:45:05.615227 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:45:05.615292 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:45:05.615295 (+ 3us) client_negotiation.cc:769] Sending connection context
0810 04:45:05.615340 (+ 45us) client_negotiation.cc:240] Negotiation successful
0810 04:45:05.615373 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":290,"mutex_wait_us":2,"thread_start_us":76,"threads_started":1}
I20260810 04:45:05.615387 23273 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:45:05.611610 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54286 (local address 127.22.167.67:45781)
0810 04:45:05.611771 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0810 04:45:05.611775 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 04:45:05.611783 (+ 8us) server_negotiation.cc:407] Connection header received
0810 04:45:05.611816 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:45:05.611817 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:45:05.611832 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:45:05.611895 (+ 63us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 04:45:05.612152 (+ 257us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:45:05.613174 (+ 1022us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:45:05.614061 (+ 887us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:45:05.614749 (+ 688us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:45:05.614813 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:45:05.614892 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0810 04:45:05.615156 (+ 264us) server_negotiation.cc:299] Negotiation successful
0810 04:45:05.615308 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":130,"thread_start_us":76,"threads_started":1}
I20260810 04:45:05.615664 23271 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:45:05.611416 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37774 (local address 127.22.167.66:45977)
0810 04:45:05.611590 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0810 04:45:05.611594 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 04:45:05.611742 (+ 148us) server_negotiation.cc:407] Connection header received
0810 04:45:05.611789 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:45:05.611792 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:45:05.611807 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:45:05.611886 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 04:45:05.612023 (+ 137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:45:05.613114 (+ 1091us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:45:05.614370 (+ 1256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:45:05.615045 (+ 675us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:45:05.615099 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:45:05.615190 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0810 04:45:05.615546 (+ 356us) server_negotiation.cc:299] Negotiation successful
0810 04:45:05.615591 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":130,"thread_start_us":85,"threads_started":1}
I20260810 04:45:05.615959 22713 tablet_service.cc:1467] Processing CreateTablet for tablet 1030515039a74e7cb7f32d343cfbd888 (DEFAULT_TABLE table=test-table [id=dca88c0939e540458d7c6a534fc1e9a9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 04:45:05.616166 23142 tablet_service.cc:1467] Processing CreateTablet for tablet 1030515039a74e7cb7f32d343cfbd888 (DEFAULT_TABLE table=test-table [id=dca88c0939e540458d7c6a534fc1e9a9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 04:45:05.616348 22713 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1030515039a74e7cb7f32d343cfbd888. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 04:45:05.616304 22936 tablet_service.cc:1467] Processing CreateTablet for tablet 1030515039a74e7cb7f32d343cfbd888 (DEFAULT_TABLE table=test-table [id=dca88c0939e540458d7c6a534fc1e9a9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 04:45:05.616521 23142 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1030515039a74e7cb7f32d343cfbd888. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 04:45:05.616636 22936 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1030515039a74e7cb7f32d343cfbd888. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 04:45:05.619818 23275 tablet_bootstrap.cc:492] T 1030515039a74e7cb7f32d343cfbd888 P 672ea98aeed649adbe8fe8dd2d5f6326: Bootstrap starting.
I20260810 04:45:05.620136 23274 tablet_bootstrap.cc:492] T 1030515039a74e7cb7f32d343cfbd888 P 8923b09535f64d0ba8715c728df6c40b: Bootstrap starting.
I20260810 04:45:05.620421 23275 tablet_bootstrap.cc:654] T 1030515039a74e7cb7f32d343cfbd888 P 672ea98aeed649adbe8fe8dd2d5f6326: Neither blocks nor log segments found. Creating new log.
I20260810 04:45:05.620692 23274 tablet_bootstrap.cc:654] T 1030515039a74e7cb7f32d343cfbd888 P 8923b09535f64d0ba8715c728df6c40b: Neither blocks nor log segments found. Creating new log.
I20260810 04:45:05.620685 23275 log.cc:824] T 1030515039a74e7cb7f32d343cfbd888 P 672ea98aeed649adbe8fe8dd2d5f6326: Log is configured to *not* fsync() on all Append() calls
I20260810 04:45:05.620961 23276 tablet_bootstrap.cc:492] T 1030515039a74e7cb7f32d343cfbd888 P 0c59408e485e4e46ac4849c67b9ea749: Bootstrap starting.
I20260810 04:45:05.621019 23274 log.cc:824] T 1030515039a74e7cb7f32d343cfbd888 P 8923b09535f64d0ba8715c728df6c40b: Log is configured to *not* fsync() on all Append() calls
I20260810 04:45:05.621407 23275 tablet_bootstrap.cc:492] T 1030515039a74e7cb7f32d343cfbd888 P 672ea98aeed649adbe8fe8dd2d5f6326: No bootstrap required, opened a new log
I20260810 04:45:05.621467 23275 ts_tablet_manager.cc:1397] T 1030515039a74e7cb7f32d343cfbd888 P 672ea98aeed649adbe8fe8dd2d5f6326: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260810 04:45:05.621534 23276 tablet_bootstrap.cc:654] T 1030515039a74e7cb7f32d343cfbd888 P 0c59408e485e4e46ac4849c67b9ea749: Neither blocks nor log segments found. Creating new log.
I20260810 04:45:05.621795 23276 log.cc:824] T 1030515039a74e7cb7f32d343cfbd888 P 0c59408e485e4e46ac4849c67b9ea749: Log is configured to *not* fsync() on all Append() calls
I20260810 04:45:05.621887 23274 tablet_bootstrap.cc:492] T 1030515039a74e7cb7f32d343cfbd888 P 8923b09535f64d0ba8715c728df6c40b: No bootstrap required, opened a new log
I20260810 04:45:05.621973 23274 ts_tablet_manager.cc:1397] T 1030515039a74e7cb7f32d343cfbd888 P 8923b09535f64d0ba8715c728df6c40b: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260810 04:45:05.622798 23276 tablet_bootstrap.cc:492] T 1030515039a74e7cb7f32d343cfbd888 P 0c59408e485e4e46ac4849c67b9ea749: No bootstrap required, opened a new log
I20260810 04:45:05.622864 23276 ts_tablet_manager.cc:1397] T 1030515039a74e7cb7f32d343cfbd888 P 0c59408e485e4e46ac4849c67b9ea749: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260810 04:45:05.624007 23275 raft_consensus.cc:348] T 1030515039a74e7cb7f32d343cfbd888 P 672ea98aeed649adbe8fe8dd2d5f6326 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "672ea98aeed649adbe8fe8dd2d5f6326" member_type: VOTER last_known_addr { host: "127.22.167.65" port: 33403 } } peers { permanent_uuid: "0c59408e485e4e46ac4849c67b9ea749" member_type: VOTER last_known_addr { host: "127.22.167.66" port: 45977 } } peers { permanent_uuid: "8923b09535f64d0ba8715c728df6c40b" member_type: VOTER last_known_addr { host: "127.22.167.67" port: 45781 } }
I20260810 04:45:05.624184 23275 raft_consensus.cc:374] T 1030515039a74e7cb7f32d343cfbd888 P 672ea98aeed649adbe8fe8dd2d5f6326 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 04:45:05.624224 23275 raft_consensus.cc:729] T 1030515039a74e7cb7f32d343cfbd888 P 672ea98aeed649adbe8fe8dd2d5f6326 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 672ea98aeed649adbe8fe8dd2d5f6326, State: Initialized, Role: FOLLOWER
I20260810 04:45:05.624369 23275 consensus_queue.cc:260] T 1030515039a74e7cb7f32d343cfbd888 P 672ea98aeed649adbe8fe8dd2d5f6326 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "672ea98aeed649adbe8fe8dd2d5f6326" member_type: VOTER last_known_addr { host: "127.22.167.65" port: 33403 } } peers { permanent_uuid: "0c59408e485e4e46ac4849c67b9ea749" member_type: VOTER last_known_addr { host: "127.22.167.66" port: 45977 } } peers { permanent_uuid: "8923b09535f64d0ba8715c728df6c40b" member_type: VOTER last_known_addr { host: "127.22.167.67" port: 45781 } }
I20260810 04:45:05.624507 23274 raft_consensus.cc:348] T 1030515039a74e7cb7f32d343cfbd888 P 8923b09535f64d0ba8715c728df6c40b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "672ea98aeed649adbe8fe8dd2d5f6326" member_type: VOTER last_known_addr { host: "127.22.167.65" port: 33403 } } peers { permanent_uuid: "0c59408e485e4e46ac4849c67b9ea749" member_type: VOTER last_known_addr { host: "127.22.167.66" port: 45977 } } peers { permanent_uuid: "8923b09535f64d0ba8715c728df6c40b" member_type: VOTER last_known_addr { host: "127.22.167.67" port: 45781 } }
I20260810 04:45:05.624670 23274 raft_consensus.cc:374] T 1030515039a74e7cb7f32d343cfbd888 P 8923b09535f64d0ba8715c728df6c40b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 04:45:05.624681 23275 ts_tablet_manager.cc:1428] T 1030515039a74e7cb7f32d343cfbd888 P 672ea98aeed649adbe8fe8dd2d5f6326: Time spent starting tablet: real 0.003s user 0.000s sys 0.004s
I20260810 04:45:05.624722 23274 raft_consensus.cc:729] T 1030515039a74e7cb7f32d343cfbd888 P 8923b09535f64d0ba8715c728df6c40b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8923b09535f64d0ba8715c728df6c40b, State: Initialized, Role: FOLLOWER
I20260810 04:45:05.624755 22832 heartbeater.cc:503] Master 127.22.167.126:37355 was elected leader, sending a full tablet report...
I20260810 04:45:05.624675 23276 raft_consensus.cc:348] T 1030515039a74e7cb7f32d343cfbd888 P 0c59408e485e4e46ac4849c67b9ea749 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "672ea98aeed649adbe8fe8dd2d5f6326" member_type: VOTER last_known_addr { host: "127.22.167.65" port: 33403 } } peers { permanent_uuid: "0c59408e485e4e46ac4849c67b9ea749" member_type: VOTER last_known_addr { host: "127.22.167.66" port: 45977 } } peers { permanent_uuid: "8923b09535f64d0ba8715c728df6c40b" member_type: VOTER last_known_addr { host: "127.22.167.67" port: 45781 } }
I20260810 04:45:05.624835 23276 raft_consensus.cc:374] T 1030515039a74e7cb7f32d343cfbd888 P 0c59408e485e4e46ac4849c67b9ea749 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 04:45:05.624881 23276 raft_consensus.cc:729] T 1030515039a74e7cb7f32d343cfbd888 P 0c59408e485e4e46ac4849c67b9ea749 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0c59408e485e4e46ac4849c67b9ea749, State: Initialized, Role: FOLLOWER
I20260810 04:45:05.624868 23274 consensus_queue.cc:260] T 1030515039a74e7cb7f32d343cfbd888 P 8923b09535f64d0ba8715c728df6c40b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "672ea98aeed649adbe8fe8dd2d5f6326" member_type: VOTER last_known_addr { host: "127.22.167.65" port: 33403 } } peers { permanent_uuid: "0c59408e485e4e46ac4849c67b9ea749" member_type: VOTER last_known_addr { host: "127.22.167.66" port: 45977 } } peers { permanent_uuid: "8923b09535f64d0ba8715c728df6c40b" member_type: VOTER last_known_addr { host: "127.22.167.67" port: 45781 } }
I20260810 04:45:05.625010 23276 consensus_queue.cc:260] T 1030515039a74e7cb7f32d343cfbd888 P 0c59408e485e4e46ac4849c67b9ea749 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "672ea98aeed649adbe8fe8dd2d5f6326" member_type: VOTER last_known_addr { host: "127.22.167.65" port: 33403 } } peers { permanent_uuid: "0c59408e485e4e46ac4849c67b9ea749" member_type: VOTER last_known_addr { host: "127.22.167.66" port: 45977 } } peers { permanent_uuid: "8923b09535f64d0ba8715c728df6c40b" member_type: VOTER last_known_addr { host: "127.22.167.67" port: 45781 } }
I20260810 04:45:05.625150 23274 ts_tablet_manager.cc:1428] T 1030515039a74e7cb7f32d343cfbd888 P 8923b09535f64d0ba8715c728df6c40b: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260810 04:45:05.625248 23258 heartbeater.cc:503] Master 127.22.167.126:37355 was elected leader, sending a full tablet report...
I20260810 04:45:05.625634 23276 ts_tablet_manager.cc:1428] T 1030515039a74e7cb7f32d343cfbd888 P 0c59408e485e4e46ac4849c67b9ea749: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260810 04:45:05.625785 23036 heartbeater.cc:503] Master 127.22.167.126:37355 was elected leader, sending a full tablet report...
W20260810 04:45:05.708161 23038 tablet.cc:2369] T 1030515039a74e7cb7f32d343cfbd888 P 0c59408e485e4e46ac4849c67b9ea749: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 04:45:05.729312 23285 raft_consensus.cc:482] T 1030515039a74e7cb7f32d343cfbd888 P 8923b09535f64d0ba8715c728df6c40b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 04:45:05.729439 23285 raft_consensus.cc:504] T 1030515039a74e7cb7f32d343cfbd888 P 8923b09535f64d0ba8715c728df6c40b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "672ea98aeed649adbe8fe8dd2d5f6326" member_type: VOTER last_known_addr { host: "127.22.167.65" port: 33403 } } peers { permanent_uuid: "0c59408e485e4e46ac4849c67b9ea749" member_type: VOTER last_known_addr { host: "127.22.167.66" port: 45977 } } peers { permanent_uuid: "8923b09535f64d0ba8715c728df6c40b" member_type: VOTER last_known_addr { host: "127.22.167.67" port: 45781 } }
I20260810 04:45:05.729801 23285 leader_election.cc:288] T 1030515039a74e7cb7f32d343cfbd888 P 8923b09535f64d0ba8715c728df6c40b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 672ea98aeed649adbe8fe8dd2d5f6326 (127.22.167.65:33403), 0c59408e485e4e46ac4849c67b9ea749 (127.22.167.66:45977)
I20260810 04:45:05.732692 23260 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:45:05.729941 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.65:33403 (local address 127.22.167.67:60821)
0810 04:45:05.729993 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0810 04:45:05.730008 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0810 04:45:05.730076 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 04:45:05.730290 (+ 214us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 04:45:05.730291 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 04:45:05.730293 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 04:45:05.730364 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:45:05.730365 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:45:05.731041 (+ 676us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:45:05.731042 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:45:05.731856 (+ 814us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:45:05.731859 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:45:05.732446 (+ 587us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:45:05.732448 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:45:05.732520 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:45:05.732523 (+ 3us) client_negotiation.cc:769] Sending connection context
0810 04:45:05.732582 (+ 59us) client_negotiation.cc:240] Negotiation successful
0810 04:45:05.732622 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":30}
I20260810 04:45:05.732704 23269 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:45:05.730030 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.167.67:60821 (local address 127.22.167.65:33403)
0810 04:45:05.730076 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0810 04:45:05.730078 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 04:45:05.730087 (+ 9us) server_negotiation.cc:407] Connection header received
0810 04:45:05.730180 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:45:05.730181 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:45:05.730195 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:45:05.730226 (+ 31us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 04:45:05.730444 (+ 218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:45:05.730975 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:45:05.731946 (+ 971us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:45:05.732380 (+ 434us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:45:05.732433 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:45:05.732440 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0810 04:45:05.732593 (+ 153us) server_negotiation.cc:299] Negotiation successful
0810 04:45:05.732630 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260810 04:45:05.732853 23287 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:45:05.730069 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.66:45977 (local address 127.22.167.67:60643)
0810 04:45:05.730227 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0810 04:45:05.730237 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0810 04:45:05.730297 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 04:45:05.730422 (+ 125us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 04:45:05.730423 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 04:45:05.730424 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 04:45:05.730498 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:45:05.730499 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:45:05.731252 (+ 753us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:45:05.731253 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:45:05.732051 (+ 798us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:45:05.732053 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:45:05.732600 (+ 547us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:45:05.732602 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:45:05.732679 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:45:05.732683 (+ 4us) client_negotiation.cc:769] Sending connection context
0810 04:45:05.732738 (+ 55us) client_negotiation.cc:240] Negotiation successful
0810 04:45:05.732774 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":112,"threads_started":1}
I20260810 04:45:05.733055 22749 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1030515039a74e7cb7f32d343cfbd888" candidate_uuid: "8923b09535f64d0ba8715c728df6c40b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "672ea98aeed649adbe8fe8dd2d5f6326" is_pre_election: true
I20260810 04:45:05.733237 22749 raft_consensus.cc:2454] T 1030515039a74e7cb7f32d343cfbd888 P 672ea98aeed649adbe8fe8dd2d5f6326 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8923b09535f64d0ba8715c728df6c40b in term 0.
I20260810 04:45:05.733245 23271 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:45:05.730145 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.167.67:60643 (local address 127.22.167.66:45977)
0810 04:45:05.730203 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0810 04:45:05.730205 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 04:45:05.730298 (+ 93us) server_negotiation.cc:407] Connection header received
0810 04:45:05.730346 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:45:05.730347 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:45:05.730356 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:45:05.730412 (+ 56us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 04:45:05.730588 (+ 176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:45:05.731165 (+ 577us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:45:05.732136 (+ 971us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:45:05.732526 (+ 390us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:45:05.732591 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:45:05.732598 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0810 04:45:05.733138 (+ 540us) server_negotiation.cc:299] Negotiation successful
0810 04:45:05.733186 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260810 04:45:05.733464 23064 leader_election.cc:302] T 1030515039a74e7cb7f32d343cfbd888 P 8923b09535f64d0ba8715c728df6c40b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 672ea98aeed649adbe8fe8dd2d5f6326, 8923b09535f64d0ba8715c728df6c40b; no voters:
I20260810 04:45:05.733510 22962 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1030515039a74e7cb7f32d343cfbd888" candidate_uuid: "8923b09535f64d0ba8715c728df6c40b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0c59408e485e4e46ac4849c67b9ea749" is_pre_election: true
I20260810 04:45:05.733580 23285 raft_consensus.cc:2781] T 1030515039a74e7cb7f32d343cfbd888 P 8923b09535f64d0ba8715c728df6c40b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 04:45:05.733628 23285 raft_consensus.cc:482] T 1030515039a74e7cb7f32d343cfbd888 P 8923b09535f64d0ba8715c728df6c40b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 04:45:05.733657 23285 raft_consensus.cc:3037] T 1030515039a74e7cb7f32d343cfbd888 P 8923b09535f64d0ba8715c728df6c40b [term 0 FOLLOWER]: Advancing to term 1
I20260810 04:45:05.733669 22962 raft_consensus.cc:2454] T 1030515039a74e7cb7f32d343cfbd888 P 0c59408e485e4e46ac4849c67b9ea749 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8923b09535f64d0ba8715c728df6c40b in term 0.
I20260810 04:45:05.734440 23285 raft_consensus.cc:504] T 1030515039a74e7cb7f32d343cfbd888 P 8923b09535f64d0ba8715c728df6c40b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "672ea98aeed649adbe8fe8dd2d5f6326" member_type: VOTER last_known_addr { host: "127.22.167.65" port: 33403 } } peers { permanent_uuid: "0c59408e485e4e46ac4849c67b9ea749" member_type: VOTER last_known_addr { host: "127.22.167.66" port: 45977 } } peers { permanent_uuid: "8923b09535f64d0ba8715c728df6c40b" member_type: VOTER last_known_addr { host: "127.22.167.67" port: 45781 } }
I20260810 04:45:05.734544 23285 leader_election.cc:288] T 1030515039a74e7cb7f32d343cfbd888 P 8923b09535f64d0ba8715c728df6c40b [CANDIDATE]: Term 1 election: Requested vote from peers 672ea98aeed649adbe8fe8dd2d5f6326 (127.22.167.65:33403), 0c59408e485e4e46ac4849c67b9ea749 (127.22.167.66:45977)
I20260810 04:45:05.734700 22749 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1030515039a74e7cb7f32d343cfbd888" candidate_uuid: "8923b09535f64d0ba8715c728df6c40b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "672ea98aeed649adbe8fe8dd2d5f6326"
I20260810 04:45:05.734740 22962 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1030515039a74e7cb7f32d343cfbd888" candidate_uuid: "8923b09535f64d0ba8715c728df6c40b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0c59408e485e4e46ac4849c67b9ea749"
I20260810 04:45:05.734769 22749 raft_consensus.cc:3037] T 1030515039a74e7cb7f32d343cfbd888 P 672ea98aeed649adbe8fe8dd2d5f6326 [term 0 FOLLOWER]: Advancing to term 1
I20260810 04:45:05.734822 22962 raft_consensus.cc:3037] T 1030515039a74e7cb7f32d343cfbd888 P 0c59408e485e4e46ac4849c67b9ea749 [term 0 FOLLOWER]: Advancing to term 1
I20260810 04:45:05.735504 22749 raft_consensus.cc:2454] T 1030515039a74e7cb7f32d343cfbd888 P 672ea98aeed649adbe8fe8dd2d5f6326 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8923b09535f64d0ba8715c728df6c40b in term 1.
I20260810 04:45:05.735572 22962 raft_consensus.cc:2454] T 1030515039a74e7cb7f32d343cfbd888 P 0c59408e485e4e46ac4849c67b9ea749 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8923b09535f64d0ba8715c728df6c40b in term 1.
I20260810 04:45:05.735672 23064 leader_election.cc:302] T 1030515039a74e7cb7f32d343cfbd888 P 8923b09535f64d0ba8715c728df6c40b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 672ea98aeed649adbe8fe8dd2d5f6326, 8923b09535f64d0ba8715c728df6c40b; no voters:
I20260810 04:45:05.735765 23285 raft_consensus.cc:2781] T 1030515039a74e7cb7f32d343cfbd888 P 8923b09535f64d0ba8715c728df6c40b [term 1 FOLLOWER]: Leader election won for term 1
I20260810 04:45:05.735924 23285 raft_consensus.cc:686] T 1030515039a74e7cb7f32d343cfbd888 P 8923b09535f64d0ba8715c728df6c40b [term 1 LEADER]: Becoming Leader. State: Replica: 8923b09535f64d0ba8715c728df6c40b, State: Running, Role: LEADER
I20260810 04:45:05.736016 23285 consensus_queue.cc:237] T 1030515039a74e7cb7f32d343cfbd888 P 8923b09535f64d0ba8715c728df6c40b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "672ea98aeed649adbe8fe8dd2d5f6326" member_type: VOTER last_known_addr { host: "127.22.167.65" port: 33403 } } peers { permanent_uuid: "0c59408e485e4e46ac4849c67b9ea749" member_type: VOTER last_known_addr { host: "127.22.167.66" port: 45977 } } peers { permanent_uuid: "8923b09535f64d0ba8715c728df6c40b" member_type: VOTER last_known_addr { host: "127.22.167.67" port: 45781 } }
I20260810 04:45:05.736855 22485 catalog_manager.cc:5495] T 1030515039a74e7cb7f32d343cfbd888 P 8923b09535f64d0ba8715c728df6c40b reported cstate change: term changed from 0 to 1, leader changed from <none> to 8923b09535f64d0ba8715c728df6c40b (127.22.167.67). New cstate: current_term: 1 leader_uuid: "8923b09535f64d0ba8715c728df6c40b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "672ea98aeed649adbe8fe8dd2d5f6326" member_type: VOTER last_known_addr { host: "127.22.167.65" port: 33403 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0c59408e485e4e46ac4849c67b9ea749" member_type: VOTER last_known_addr { host: "127.22.167.66" port: 45977 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8923b09535f64d0ba8715c728df6c40b" member_type: VOTER last_known_addr { host: "127.22.167.67" port: 45781 } health_report { overall_health: HEALTHY } } }
I20260810 04:45:05.757580 23273 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:45:05.755485 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38344 (local address 127.22.167.67:45781)
0810 04:45:05.755529 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0810 04:45:05.755530 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0810 04:45:05.755540 (+ 10us) server_negotiation.cc:407] Connection header received
0810 04:45:05.755598 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:45:05.755599 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:45:05.755615 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:45:05.755674 (+ 59us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 04:45:05.755772 (+ 98us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:45:05.756213 (+ 441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:45:05.756950 (+ 737us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:45:05.757142 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:45:05.757193 (+ 51us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:45:05.757301 (+ 108us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 04:45:05.757382 (+ 81us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 04:45:05.757445 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0810 04:45:05.757478 (+ 33us) server_negotiation.cc:299] Negotiation successful
0810 04:45:05.757487 (+ 9us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260810 04:45:05.759284 22749 raft_consensus.cc:1261] T 1030515039a74e7cb7f32d343cfbd888 P 672ea98aeed649adbe8fe8dd2d5f6326 [term 1 FOLLOWER]: Refusing update from remote peer 8923b09535f64d0ba8715c728df6c40b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 04:45:05.759284 22962 raft_consensus.cc:1261] T 1030515039a74e7cb7f32d343cfbd888 P 0c59408e485e4e46ac4849c67b9ea749 [term 1 FOLLOWER]: Refusing update from remote peer 8923b09535f64d0ba8715c728df6c40b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 04:45:05.759655 23289 consensus_queue.cc:1035] T 1030515039a74e7cb7f32d343cfbd888 P 8923b09535f64d0ba8715c728df6c40b [LEADER]: Connected to new peer: Peer: permanent_uuid: "0c59408e485e4e46ac4849c67b9ea749" member_type: VOTER last_known_addr { host: "127.22.167.66" port: 45977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 04:45:05.759764 23285 consensus_queue.cc:1035] T 1030515039a74e7cb7f32d343cfbd888 P 8923b09535f64d0ba8715c728df6c40b [LEADER]: Connected to new peer: Peer: permanent_uuid: "672ea98aeed649adbe8fe8dd2d5f6326" member_type: VOTER last_known_addr { host: "127.22.167.65" port: 33403 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 04:45:05.762023 23301 mvcc.cc:205] Tried to move back new op lower bound from 7316836785187495936 to 7316836785095331840. Current Snapshot: MvccSnapshot[applied={T|T < 7316836785187495936}]
I20260810 04:45:05.764371 22485 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:60614:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 04:45:05.764458 22485 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:60614:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 04:45:05.765111 22485 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 18dabf510a4b471bae892f37eef36188: Sending DeleteTablet for 3 replicas of tablet 1030515039a74e7cb7f32d343cfbd888
I20260810 04:45:05.765499 22713 tablet_service.cc:1514] Processing DeleteTablet for tablet 1030515039a74e7cb7f32d343cfbd888 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 04:45:05 UTC) from {username='slave'} at 127.0.0.1:46656
I20260810 04:45:05.765525 22936 tablet_service.cc:1514] Processing DeleteTablet for tablet 1030515039a74e7cb7f32d343cfbd888 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 04:45:05 UTC) from {username='slave'} at 127.0.0.1:37774
I20260810 04:45:05.765789 23197 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskemv_AM/build/release/bin/kudu with pid 22592
I20260810 04:45:05.765774 23142 tablet_service.cc:1514] Processing DeleteTablet for tablet 1030515039a74e7cb7f32d343cfbd888 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 04:45:05 UTC) from {username='slave'} at 127.0.0.1:54286
I20260810 04:45:05.777318 23320 tablet_replica.cc:330] T 1030515039a74e7cb7f32d343cfbd888 P 8923b09535f64d0ba8715c728df6c40b: stopping tablet replica
I20260810 04:45:05.777432 23320 raft_consensus.cc:2229] T 1030515039a74e7cb7f32d343cfbd888 P 8923b09535f64d0ba8715c728df6c40b [term 1 LEADER]: Raft consensus shutting down.
I20260810 04:45:05.777577 23320 raft_consensus.cc:2258] T 1030515039a74e7cb7f32d343cfbd888 P 8923b09535f64d0ba8715c728df6c40b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 04:45:05.784691 23319 tablet_replica.cc:330] T 1030515039a74e7cb7f32d343cfbd888 P 0c59408e485e4e46ac4849c67b9ea749: stopping tablet replica
I20260810 04:45:05.784840 23319 raft_consensus.cc:2229] T 1030515039a74e7cb7f32d343cfbd888 P 0c59408e485e4e46ac4849c67b9ea749 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 04:45:05.784967 23319 raft_consensus.cc:2258] T 1030515039a74e7cb7f32d343cfbd888 P 0c59408e485e4e46ac4849c67b9ea749 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 04:45:05.785099 23320 ts_tablet_manager.cc:1905] T 1030515039a74e7cb7f32d343cfbd888 P 8923b09535f64d0ba8715c728df6c40b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 04:45:05.785571 23319 ts_tablet_manager.cc:1905] T 1030515039a74e7cb7f32d343cfbd888 P 0c59408e485e4e46ac4849c67b9ea749: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 04:45:05.786767 23320 ts_tablet_manager.cc:1918] T 1030515039a74e7cb7f32d343cfbd888 P 8923b09535f64d0ba8715c728df6c40b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 04:45:05.786845 23320 log.cc:1196] T 1030515039a74e7cb7f32d343cfbd888 P 8923b09535f64d0ba8715c728df6c40b: Deleting WAL directory at /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786336863388938-23197-0/minicluster-data/ts-2/wal/wals/1030515039a74e7cb7f32d343cfbd888
I20260810 04:45:05.787091 23320 ts_tablet_manager.cc:1939] T 1030515039a74e7cb7f32d343cfbd888 P 8923b09535f64d0ba8715c728df6c40b: Deleting consensus metadata
I20260810 04:45:05.787468 23319 ts_tablet_manager.cc:1918] T 1030515039a74e7cb7f32d343cfbd888 P 0c59408e485e4e46ac4849c67b9ea749: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 04:45:05.787509 22461 catalog_manager.cc:4841] TS 8923b09535f64d0ba8715c728df6c40b (127.22.167.67:45781): tablet 1030515039a74e7cb7f32d343cfbd888 (table test-table [id=dca88c0939e540458d7c6a534fc1e9a9]) successfully deleted
I20260810 04:45:05.787644 23319 log.cc:1196] T 1030515039a74e7cb7f32d343cfbd888 P 0c59408e485e4e46ac4849c67b9ea749: Deleting WAL directory at /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786336863388938-23197-0/minicluster-data/ts-1/wal/wals/1030515039a74e7cb7f32d343cfbd888
I20260810 04:45:05.787902 23319 ts_tablet_manager.cc:1939] T 1030515039a74e7cb7f32d343cfbd888 P 0c59408e485e4e46ac4849c67b9ea749: Deleting consensus metadata
I20260810 04:45:05.788337 22461 catalog_manager.cc:4841] TS 0c59408e485e4e46ac4849c67b9ea749 (127.22.167.66:45977): tablet 1030515039a74e7cb7f32d343cfbd888 (table test-table [id=dca88c0939e540458d7c6a534fc1e9a9]) successfully deleted
W20260810 04:45:05.789896 22461 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.22.167.65:33403 (error 108)
I20260810 04:45:05.790091 23197 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskemv_AM/build/release/bin/kudu with pid 22837
I20260810 04:45:05.794920 23270 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:45:05.794764 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.65:33403 (local address 127.0.0.1:47688)
0810 04:45:05.794828 (+ 64us) negotiation.cc:107] Waiting for socket to connect
0810 04:45:05.794881 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.167.65:33403: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
W20260810 04:45:05.795094 22461 catalog_manager.cc:4584] TS 672ea98aeed649adbe8fe8dd2d5f6326 (127.22.167.65:33403): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 1030515039a74e7cb7f32d343cfbd888: Network error: Client connection negotiation failed: client connection to 127.22.167.65:33403: connect: Connection refused (error 111)
I20260810 04:45:05.800951 23197 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskemv_AM/build/release/bin/kudu with pid 23040
I20260810 04:45:05.813094 23197 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskemv_AM/build/release/bin/kudu with pid 22430
2026-08-10T04:45:05Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/3 (883 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/4
Loading random data
Initializing database '/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786336863388938-23197-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 04:45:05 dist-test-slave-bqcl krb5kdc[23335](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 04:45:05 dist-test-slave-bqcl krb5kdc[23335](info): set up 2 sockets
Aug 10 04:45:05 dist-test-slave-bqcl krb5kdc[23335](info): commencing operation
krb5kdc: starting...
W20260810 04:45:06.981758 23197 mini_kdc.cc:121] Time spent starting KDC: real 1.144s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786336863388938-23197-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786336863388938-23197-0/krb5kdc/test-user.keytab.
Aug 10 04:45:07 dist-test-slave-bqcl krb5kdc[23335](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786337107, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T04:45:07Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T04:45:07Z Disabled control of system clock
WARNING: no policy specified for kudu/127.22.167.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.167.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.167.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786336863388938-23197-0/krb5kdc/kudu_127.22.167.126.keytab.
Entry for principal kudu/127.22.167.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786336863388938-23197-0/krb5kdc/kudu_127.22.167.126.keytab.
I20260810 04:45:07.094779 23197 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskemv_AM/build/release/bin/kudu
/tmp/dist-test-taskemv_AM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786336863388938-23197-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786336863388938-23197-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786336863388938-23197-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786336863388938-23197-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.167.126:45405
--webserver_interface=127.22.167.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786336863388938-23197-0/krb5kdc/kudu_127.22.167.126.keytab
--principal=kudu/127.22.167.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.167.84:36897
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.167.126:45405
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786336863388938-23197-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786336863388938-23197-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786336863388938-23197-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 04:45:07.174474 23351 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 04:45:07.174669 23351 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 04:45:07.174690 23351 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 04:45:07.175782 23351 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 04:45:07.175822 23351 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 04:45:07.175834 23351 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 04:45:07.175858 23351 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 04:45:07.175874 23351 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 04:45:07.177285 23351 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.167.84:36897
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786336863388938-23197-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786336863388938-23197-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.22.167.126:45405
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786336863388938-23197-0/krb5kdc/kudu_127.22.167.126.keytab
--principal=kudu/127.22.167.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.167.126:45405
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786336863388938-23197-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.167.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23351
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786336863388938-23197-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 03:54:01 UTC on ccycloud.quasar-icijmt.root.comops.site
build id 4577009
I20260810 04:45:07.177510 23351 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 04:45:07.177723 23351 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 04:45:07.182382 23359 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 04:45:07.182471 23351 server_base.cc:1034] running on GCE node
W20260810 04:45:07.182360 23358 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 04:45:07.182363 23364 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 04:45:07.182806 23351 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 04:45:07.183079 23351 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 04:45:07.184248 23351 hybrid_clock.cc:648] HybridClock initialized: now 1786337107184235 us; error 27 us; skew 500 ppm
Aug 10 04:45:07 dist-test-slave-bqcl krb5kdc[23335](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786337107, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.167.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 04:45:07.186877 23351 init.cc:377] Logged in from keytab as kudu/127.22.167.126@KRBTEST.COM (short username kudu)
I20260810 04:45:07.187311 23351 webserver.cc:460] Webserver started at http://127.22.167.126:40979/ using document root <none> and password file <none>
I20260810 04:45:07.187547 23351 fs_manager.cc:359] Metadata directory not provided
I20260810 04:45:07.187597 23351 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 04:45:07.187727 23351 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 04:45:07.188764 23351 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786336863388938-23197-0/minicluster-data/master-0/data/instance:
uuid: "4b51fe86ac6c4249943ce952ad24ac8a"
format_stamp: "Formatted at 2026-08-10 04:45:07 on dist-test-slave-bqcl"
server_key: "8e7f4a1c6a5444d905ca1d5a8be113ed"
server_key_iv: "eb1046f1bb14dfa47145fbda7377ccb4"
server_key_version: "encryptionkey@0"
I20260810 04:45:07.189096 23351 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786336863388938-23197-0/minicluster-data/master-0/wal/instance:
uuid: "4b51fe86ac6c4249943ce952ad24ac8a"
format_stamp: "Formatted at 2026-08-10 04:45:07 on dist-test-slave-bqcl"
server_key: "8e7f4a1c6a5444d905ca1d5a8be113ed"
server_key_iv: "eb1046f1bb14dfa47145fbda7377ccb4"
server_key_version: "encryptionkey@0"
I20260810 04:45:07.190338 23351 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260810 04:45:07.191001 23372 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 04:45:07.191152 23351 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260810 04:45:07.191227 23351 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786336863388938-23197-0/minicluster-data/master-0/data,/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786336863388938-23197-0/minicluster-data/master-0/wal
uuid: "4b51fe86ac6c4249943ce952ad24ac8a"
format_stamp: "Formatted at 2026-08-10 04:45:07 on dist-test-slave-bqcl"
server_key: "8e7f4a1c6a5444d905ca1d5a8be113ed"
server_key_iv: "eb1046f1bb14dfa47145fbda7377ccb4"
server_key_version: "encryptionkey@0"
I20260810 04:45:07.191288 23351 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786336863388938-23197-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786336863388938-23197-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786336863388938-23197-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 04:45:07.201223 23351 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 04:45:07.203517 23351 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 04:45:07.203652 23351 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 04:45:07.207538 23351 rpc_server.cc:307] RPC server started. Bound to: 127.22.167.126:45405
I20260810 04:45:07.207576 23447 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.167.126:45405 every 8 connection(s)
I20260810 04:45:07.207872 23351 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786336863388938-23197-0/minicluster-data/master-0/data/info.pb
I20260810 04:45:07.208671 23448 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 04:45:07.209015 23197 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskemv_AM/build/release/bin/kudu as pid 23351
I20260810 04:45:07.209126 23197 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786336863388938-23197-0/minicluster-data/master-0/wal/instance
I20260810 04:45:07.209295 23197 external_mini_cluster.cc:1428] Setting key a4556036407e6ef32fe03770a1cb39c7
I20260810 04:45:07.211261 23448 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4b51fe86ac6c4249943ce952ad24ac8a: Bootstrap starting.
I20260810 04:45:07.211834 23448 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4b51fe86ac6c4249943ce952ad24ac8a: Neither blocks nor log segments found. Creating new log.
I20260810 04:45:07.212108 23448 log.cc:824] T 00000000000000000000000000000000 P 4b51fe86ac6c4249943ce952ad24ac8a: Log is configured to *not* fsync() on all Append() calls
I20260810 04:45:07.212770 23448 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4b51fe86ac6c4249943ce952ad24ac8a: No bootstrap required, opened a new log
I20260810 04:45:07.214862 23448 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4b51fe86ac6c4249943ce952ad24ac8a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4b51fe86ac6c4249943ce952ad24ac8a" member_type: VOTER last_known_addr { host: "127.22.167.126" port: 45405 } }
I20260810 04:45:07.214993 23448 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4b51fe86ac6c4249943ce952ad24ac8a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 04:45:07.215025 23448 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4b51fe86ac6c4249943ce952ad24ac8a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4b51fe86ac6c4249943ce952ad24ac8a, State: Initialized, Role: FOLLOWER
I20260810 04:45:07.215149 23448 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4b51fe86ac6c4249943ce952ad24ac8a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4b51fe86ac6c4249943ce952ad24ac8a" member_type: VOTER last_known_addr { host: "127.22.167.126" port: 45405 } }
I20260810 04:45:07.215226 23448 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4b51fe86ac6c4249943ce952ad24ac8a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 04:45:07.215257 23448 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4b51fe86ac6c4249943ce952ad24ac8a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 04:45:07.215304 23448 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4b51fe86ac6c4249943ce952ad24ac8a [term 0 FOLLOWER]: Advancing to term 1
Aug 10 04:45:07 dist-test-slave-bqcl krb5kdc[23335](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786337107, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.167.126@KRBTEST.COM
I20260810 04:45:07.215884 23448 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4b51fe86ac6c4249943ce952ad24ac8a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4b51fe86ac6c4249943ce952ad24ac8a" member_type: VOTER last_known_addr { host: "127.22.167.126" port: 45405 } }
I20260810 04:45:07.215996 23448 leader_election.cc:302] T 00000000000000000000000000000000 P 4b51fe86ac6c4249943ce952ad24ac8a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4b51fe86ac6c4249943ce952ad24ac8a; no voters:
I20260810 04:45:07.216166 23448 leader_election.cc:288] T 00000000000000000000000000000000 P 4b51fe86ac6c4249943ce952ad24ac8a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 04:45:07.216215 23455 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4b51fe86ac6c4249943ce952ad24ac8a [term 1 FOLLOWER]: Leader election won for term 1
I20260810 04:45:07.216367 23455 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4b51fe86ac6c4249943ce952ad24ac8a [term 1 LEADER]: Becoming Leader. State: Replica: 4b51fe86ac6c4249943ce952ad24ac8a, State: Running, Role: LEADER
I20260810 04:45:07.216414 23448 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4b51fe86ac6c4249943ce952ad24ac8a [sys.catalog]: configured and running, proceeding with master startup.
I20260810 04:45:07.216528 23455 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4b51fe86ac6c4249943ce952ad24ac8a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4b51fe86ac6c4249943ce952ad24ac8a" member_type: VOTER last_known_addr { host: "127.22.167.126" port: 45405 } }
I20260810 04:45:07.217049 23456 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4b51fe86ac6c4249943ce952ad24ac8a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4b51fe86ac6c4249943ce952ad24ac8a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4b51fe86ac6c4249943ce952ad24ac8a" member_type: VOTER last_known_addr { host: "127.22.167.126" port: 45405 } } }
I20260810 04:45:07.217306 23456 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4b51fe86ac6c4249943ce952ad24ac8a [sys.catalog]: This master's current role is: LEADER
I20260810 04:45:07.217239 23458 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4b51fe86ac6c4249943ce952ad24ac8a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4b51fe86ac6c4249943ce952ad24ac8a. Latest consensus state: current_term: 1 leader_uuid: "4b51fe86ac6c4249943ce952ad24ac8a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4b51fe86ac6c4249943ce952ad24ac8a" member_type: VOTER last_known_addr { host: "127.22.167.126" port: 45405 } } }
I20260810 04:45:07.217557 23458 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4b51fe86ac6c4249943ce952ad24ac8a [sys.catalog]: This master's current role is: LEADER
I20260810 04:45:07.217748 23474 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 04:45:07.218322 23474 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 04:45:07.219452 23452 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:45:07.209782 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47246 (local address 127.22.167.126:45405)
0810 04:45:07.209941 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0810 04:45:07.209949 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 04:45:07.209974 (+ 25us) server_negotiation.cc:407] Connection header received
0810 04:45:07.210415 (+ 441us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:45:07.210421 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:45:07.210468 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:45:07.210585 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0810 04:45:07.211365 (+ 780us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:45:07.212294 (+ 929us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:45:07.213056 (+ 762us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:45:07.213643 (+ 587us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:45:07.213752 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:45:07.215884 (+ 2132us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 04:45:07.215888 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 04:45:07.215889 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 04:45:07.215900 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0810 04:45:07.217920 (+ 2020us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 04:45:07.218269 (+ 349us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 04:45:07.218272 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 04:45:07.218272 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 04:45:07.218379 (+ 107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 04:45:07.218623 (+ 244us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 04:45:07.218624 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 04:45:07.218624 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 04:45:07.218849 (+ 225us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 04:45:07.218999 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0810 04:45:07.219114 (+ 115us) server_negotiation.cc:299] Negotiation successful
0810 04:45:07.219244 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":113,"thread_start_us":68,"threads_started":1}
I20260810 04:45:07.220147 23474 catalog_manager.cc:1347] Generated new cluster ID: 5c54d69732e74dacbf9021be9d17babc
I20260810 04:45:07.220211 23474 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 04:45:07.233421 23474 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 04:45:07.233928 23474 catalog_manager.cc:1504] Loading token signing keys...
I20260810 04:45:07.240545 23474 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4b51fe86ac6c4249943ce952ad24ac8a: Generated new TSK 0
I20260810 04:45:07.240712 23474 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.22.167.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.167.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.167.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786336863388938-23197-0/krb5kdc/kudu_127.22.167.65.keytab.
Entry for principal kudu/127.22.167.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786336863388938-23197-0/krb5kdc/kudu_127.22.167.65.keytab.
I20260810 04:45:07.270363 23197 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskemv_AM/build/release/bin/kudu
/tmp/dist-test-taskemv_AM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786336863388938-23197-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786336863388938-23197-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786336863388938-23197-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786336863388938-23197-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.167.65:0
--local_ip_for_outbound_sockets=127.22.167.65
--webserver_interface=127.22.167.65
--webserver_port=0
--tserver_master_addrs=127.22.167.126:45405
--keytab_file=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786336863388938-23197-0/krb5kdc/kudu_127.22.167.65.keytab
--principal=kudu/127.22.167.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.167.84:36897
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786336863388938-23197-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786336863388938-23197-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786336863388938-23197-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 04:45:07.350636 23490 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 04:45:07.350834 23490 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 04:45:07.350864 23490 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 04:45:07.351936 23490 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 04:45:07.351974 23490 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 04:45:07.352008 23490 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.167.65
I20260810 04:45:07.353392 23490 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.167.84:36897
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786336863388938-23197-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786336863388938-23197-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786336863388938-23197-0/krb5kdc/kudu_127.22.167.65.keytab
--principal=kudu/127.22.167.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.167.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786336863388938-23197-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.22.167.65
--webserver_port=0
--tserver_master_addrs=127.22.167.126:45405
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23490
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.167.65
--log_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786336863388938-23197-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 03:54:01 UTC on ccycloud.quasar-icijmt.root.comops.site
build id 4577009
I20260810 04:45:07.353621 23490 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 04:45:07.353818 23490 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 04:45:07.358515 23502 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 04:45:07.358505 23507 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 04:45:07.358656 23490 server_base.cc:1034] running on GCE node
W20260810 04:45:07.358531 23505 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 04:45:07.359036 23490 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 04:45:07.359287 23490 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 04:45:07.360430 23490 hybrid_clock.cc:648] HybridClock initialized: now 1786337107360417 us; error 28 us; skew 500 ppm
Aug 10 04:45:07 dist-test-slave-bqcl krb5kdc[23335](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786337107, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.167.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 04:45:07.362907 23490 init.cc:377] Logged in from keytab as kudu/127.22.167.65@KRBTEST.COM (short username kudu)
I20260810 04:45:07.363368 23490 webserver.cc:460] Webserver started at http://127.22.167.65:44129/ using document root <none> and password file <none>
I20260810 04:45:07.363582 23490 fs_manager.cc:359] Metadata directory not provided
I20260810 04:45:07.363637 23490 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 04:45:07.363770 23490 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 04:45:07.364756 23490 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786336863388938-23197-0/minicluster-data/ts-0/data/instance:
uuid: "4451340e55fb40b6af3c77554164810e"
format_stamp: "Formatted at 2026-08-10 04:45:07 on dist-test-slave-bqcl"
server_key: "3dc0185e9dc0499b0d3c3aec2fed5bab"
server_key_iv: "e1d8bfd86091430c87fbbe882bcb375a"
server_key_version: "encryptionkey@0"
I20260810 04:45:07.365121 23490 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786336863388938-23197-0/minicluster-data/ts-0/wal/instance:
uuid: "4451340e55fb40b6af3c77554164810e"
format_stamp: "Formatted at 2026-08-10 04:45:07 on dist-test-slave-bqcl"
server_key: "3dc0185e9dc0499b0d3c3aec2fed5bab"
server_key_iv: "e1d8bfd86091430c87fbbe882bcb375a"
server_key_version: "encryptionkey@0"
I20260810 04:45:07.366475 23490 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.002s
I20260810 04:45:07.367154 23517 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 04:45:07.367312 23490 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260810 04:45:07.367401 23490 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786336863388938-23197-0/minicluster-data/ts-0/data,/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786336863388938-23197-0/minicluster-data/ts-0/wal
uuid: "4451340e55fb40b6af3c77554164810e"
format_stamp: "Formatted at 2026-08-10 04:45:07 on dist-test-slave-bqcl"
server_key: "3dc0185e9dc0499b0d3c3aec2fed5bab"
server_key_iv: "e1d8bfd86091430c87fbbe882bcb375a"
server_key_version: "encryptionkey@0"
I20260810 04:45:07.367465 23490 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786336863388938-23197-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786336863388938-23197-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786336863388938-23197-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 04:45:07.396458 23490 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 04:45:07.398941 23490 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 04:45:07.399115 23490 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 04:45:07.399363 23490 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 04:45:07.399688 23490 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 04:45:07.399727 23490 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 04:45:07.399762 23490 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 04:45:07.399789 23490 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 04:45:07.406299 23490 rpc_server.cc:307] RPC server started. Bound to: 127.22.167.65:46777
I20260810 04:45:07.406368 23701 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.167.65:46777 every 8 connection(s)
I20260810 04:45:07.406684 23490 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786336863388938-23197-0/minicluster-data/ts-0/data/info.pb
Aug 10 04:45:07 dist-test-slave-bqcl krb5kdc[23335](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786337107, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.167.65@KRBTEST.COM for kudu/127.22.167.126@KRBTEST.COM
I20260810 04:45:07.415418 23197 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskemv_AM/build/release/bin/kudu as pid 23490
I20260810 04:45:07.415544 23197 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786336863388938-23197-0/minicluster-data/ts-0/wal/instance
I20260810 04:45:07.415747 23197 external_mini_cluster.cc:1428] Setting key 17ea3274b7ea63b1271610c605c77181
I20260810 04:45:07.415810 23452 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:45:07.407400 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.167.65:55963 (local address 127.22.167.126:45405)
0810 04:45:07.407457 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0810 04:45:07.407459 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 04:45:07.407671 (+ 212us) server_negotiation.cc:407] Connection header received
0810 04:45:07.408163 (+ 492us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:45:07.408164 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:45:07.408176 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:45:07.408242 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0810 04:45:07.409186 (+ 944us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:45:07.409719 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:45:07.410703 (+ 984us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:45:07.410909 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:45:07.410963 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:45:07.413040 (+ 2077us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 04:45:07.413041 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 04:45:07.413042 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 04:45:07.413050 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0810 04:45:07.414494 (+ 1444us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 04:45:07.414861 (+ 367us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 04:45:07.414862 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 04:45:07.414863 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 04:45:07.414928 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 04:45:07.415127 (+ 199us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 04:45:07.415128 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 04:45:07.415128 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 04:45:07.415256 (+ 128us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 04:45:07.415309 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0810 04:45:07.415647 (+ 338us) server_negotiation.cc:299] Negotiation successful
0810 04:45:07.415734 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260810 04:45:07.416028 23705 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:45:07.407349 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.126:45405 (local address 127.22.167.65:55963)
0810 04:45:07.407516 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0810 04:45:07.407560 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0810 04:45:07.408048 (+ 488us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 04:45:07.408312 (+ 264us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 04:45:07.408315 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 04:45:07.408894 (+ 579us) client_negotiation.cc:189] Negotiated authn=SASL
0810 04:45:07.409099 (+ 205us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:45:07.409105 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:45:07.409783 (+ 678us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:45:07.409784 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:45:07.410612 (+ 828us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:45:07.410615 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:45:07.410973 (+ 358us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:45:07.410974 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:45:07.411078 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:45:07.411563 (+ 485us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 04:45:07.411571 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0810 04:45:07.412947 (+ 1376us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 04:45:07.414561 (+ 1614us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 04:45:07.414566 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 04:45:07.414567 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 04:45:07.414780 (+ 213us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 04:45:07.414992 (+ 212us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 04:45:07.414993 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 04:45:07.414993 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 04:45:07.415084 (+ 91us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 04:45:07.415311 (+ 227us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 04:45:07.415317 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 04:45:07.415486 (+ 169us) client_negotiation.cc:769] Sending connection context
0810 04:45:07.415612 (+ 126us) client_negotiation.cc:240] Negotiation successful
0810 04:45:07.415800 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":71,"threads_started":1}
I20260810 04:45:07.416536 23702 heartbeater.cc:347] Connected to a master server at 127.22.167.126:45405
I20260810 04:45:07.416651 23702 heartbeater.cc:464] Registering TS with master...
I20260810 04:45:07.416851 23702 heartbeater.cc:511] Master 127.22.167.126:45405 requested a full tablet report, sending...
I20260810 04:45:07.417378 23394 ts_manager.cc:194] Registered new tserver with Master: 4451340e55fb40b6af3c77554164810e (127.22.167.65:46777)
I20260810 04:45:07.418093 23394 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.167.65@KRBTEST.COM'} at 127.22.167.65:55963
WARNING: no policy specified for kudu/127.22.167.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.167.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.167.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786336863388938-23197-0/krb5kdc/kudu_127.22.167.66.keytab.
Entry for principal kudu/127.22.167.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786336863388938-23197-0/krb5kdc/kudu_127.22.167.66.keytab.
I20260810 04:45:07.441928 23197 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskemv_AM/build/release/bin/kudu
/tmp/dist-test-taskemv_AM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786336863388938-23197-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786336863388938-23197-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786336863388938-23197-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786336863388938-23197-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.167.66:0
--local_ip_for_outbound_sockets=127.22.167.66
--webserver_interface=127.22.167.66
--webserver_port=0
--tserver_master_addrs=127.22.167.126:45405
--keytab_file=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786336863388938-23197-0/krb5kdc/kudu_127.22.167.66.keytab
--principal=kudu/127.22.167.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.167.84:36897
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786336863388938-23197-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786336863388938-23197-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786336863388938-23197-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 04:45:07.524181 23709 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 04:45:07.524371 23709 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 04:45:07.524392 23709 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 04:45:07.525506 23709 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 04:45:07.525554 23709 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 04:45:07.525583 23709 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.167.66
I20260810 04:45:07.527040 23709 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.167.84:36897
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786336863388938-23197-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786336863388938-23197-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786336863388938-23197-0/krb5kdc/kudu_127.22.167.66.keytab
--principal=kudu/127.22.167.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.167.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786336863388938-23197-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.22.167.66
--webserver_port=0
--tserver_master_addrs=127.22.167.126:45405
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23709
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.167.66
--log_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786336863388938-23197-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 03:54:01 UTC on ccycloud.quasar-icijmt.root.comops.site
build id 4577009
I20260810 04:45:07.527259 23709 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 04:45:07.527467 23709 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 04:45:07.532167 23718 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 04:45:07.532191 23716 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 04:45:07.532238 23709 server_base.cc:1034] running on GCE node
W20260810 04:45:07.532196 23715 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 04:45:07.532526 23709 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 04:45:07.532771 23709 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 04:45:07.533914 23709 hybrid_clock.cc:648] HybridClock initialized: now 1786337107533904 us; error 30 us; skew 500 ppm
Aug 10 04:45:07 dist-test-slave-bqcl krb5kdc[23335](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786337107, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.167.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 04:45:07.536338 23709 init.cc:377] Logged in from keytab as kudu/127.22.167.66@KRBTEST.COM (short username kudu)
I20260810 04:45:07.536790 23709 webserver.cc:460] Webserver started at http://127.22.167.66:35443/ using document root <none> and password file <none>
I20260810 04:45:07.537022 23709 fs_manager.cc:359] Metadata directory not provided
I20260810 04:45:07.537070 23709 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 04:45:07.537194 23709 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 04:45:07.538264 23709 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786336863388938-23197-0/minicluster-data/ts-1/data/instance:
uuid: "10ca15d771b343c3a7cdc7842f7c56bf"
format_stamp: "Formatted at 2026-08-10 04:45:07 on dist-test-slave-bqcl"
server_key: "2c2d1ddb95ae80abb840874248cdd5e0"
server_key_iv: "8e350316698c70c93fa8e545b654f8ed"
server_key_version: "encryptionkey@0"
I20260810 04:45:07.538578 23709 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786336863388938-23197-0/minicluster-data/ts-1/wal/instance:
uuid: "10ca15d771b343c3a7cdc7842f7c56bf"
format_stamp: "Formatted at 2026-08-10 04:45:07 on dist-test-slave-bqcl"
server_key: "2c2d1ddb95ae80abb840874248cdd5e0"
server_key_iv: "8e350316698c70c93fa8e545b654f8ed"
server_key_version: "encryptionkey@0"
I20260810 04:45:07.539876 23709 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260810 04:45:07.540553 23728 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 04:45:07.540747 23709 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260810 04:45:07.540841 23709 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786336863388938-23197-0/minicluster-data/ts-1/data,/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786336863388938-23197-0/minicluster-data/ts-1/wal
uuid: "10ca15d771b343c3a7cdc7842f7c56bf"
format_stamp: "Formatted at 2026-08-10 04:45:07 on dist-test-slave-bqcl"
server_key: "2c2d1ddb95ae80abb840874248cdd5e0"
server_key_iv: "8e350316698c70c93fa8e545b654f8ed"
server_key_version: "encryptionkey@0"
I20260810 04:45:07.540899 23709 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786336863388938-23197-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786336863388938-23197-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786336863388938-23197-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 04:45:07.553259 23709 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 04:45:07.555395 23709 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 04:45:07.555505 23709 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 04:45:07.555692 23709 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 04:45:07.555946 23709 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 04:45:07.555976 23709 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 04:45:07.556010 23709 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 04:45:07.556025 23709 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 04:45:07.561518 23709 rpc_server.cc:307] RPC server started. Bound to: 127.22.167.66:38317
I20260810 04:45:07.561645 23908 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.167.66:38317 every 8 connection(s)
I20260810 04:45:07.561865 23709 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786336863388938-23197-0/minicluster-data/ts-1/data/info.pb
I20260810 04:45:07.567067 23197 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskemv_AM/build/release/bin/kudu as pid 23709
I20260810 04:45:07.567185 23197 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786336863388938-23197-0/minicluster-data/ts-1/wal/instance
Aug 10 04:45:07 dist-test-slave-bqcl krb5kdc[23335](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786337107, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.167.66@KRBTEST.COM for kudu/127.22.167.126@KRBTEST.COM
I20260810 04:45:07.567369 23197 external_mini_cluster.cc:1428] Setting key 060737f1bf84aa81926aad6862e7ffca
I20260810 04:45:07.570495 23452 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:45:07.562556 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.167.66:52845 (local address 127.22.167.126:45405)
0810 04:45:07.562611 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0810 04:45:07.562614 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 04:45:07.562794 (+ 180us) server_negotiation.cc:407] Connection header received
0810 04:45:07.563267 (+ 473us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:45:07.563268 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:45:07.563278 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:45:07.563315 (+ 37us) server_negotiation.cc:226] Negotiated authn=SASL
0810 04:45:07.564217 (+ 902us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:45:07.564705 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:45:07.565630 (+ 925us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:45:07.565805 (+ 175us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:45:07.565847 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:45:07.567882 (+ 2035us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 04:45:07.567883 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 04:45:07.567884 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 04:45:07.567891 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 04:45:07.569077 (+ 1186us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 04:45:07.569496 (+ 419us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 04:45:07.569497 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 04:45:07.569497 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 04:45:07.569544 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 04:45:07.569788 (+ 244us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 04:45:07.569789 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 04:45:07.569789 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 04:45:07.569935 (+ 146us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 04:45:07.570016 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0810 04:45:07.570317 (+ 301us) server_negotiation.cc:299] Negotiation successful
0810 04:45:07.570404 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260810 04:45:07.570569 23912 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:45:07.562530 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.126:45405 (local address 127.22.167.66:52845)
0810 04:45:07.562683 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0810 04:45:07.562718 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0810 04:45:07.563177 (+ 459us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 04:45:07.563373 (+ 196us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 04:45:07.563376 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 04:45:07.563948 (+ 572us) client_negotiation.cc:189] Negotiated authn=SASL
0810 04:45:07.564140 (+ 192us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:45:07.564145 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:45:07.564764 (+ 619us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:45:07.564765 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:45:07.565569 (+ 804us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:45:07.565571 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:45:07.565857 (+ 286us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:45:07.565857 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:45:07.565957 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:45:07.566380 (+ 423us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 04:45:07.566389 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0810 04:45:07.567802 (+ 1413us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 04:45:07.569149 (+ 1347us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 04:45:07.569157 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 04:45:07.569160 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 04:45:07.569430 (+ 270us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 04:45:07.569598 (+ 168us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 04:45:07.569598 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 04:45:07.569599 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 04:45:07.569725 (+ 126us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 04:45:07.570007 (+ 282us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 04:45:07.570013 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 04:45:07.570187 (+ 174us) client_negotiation.cc:769] Sending connection context
0810 04:45:07.570278 (+ 91us) client_negotiation.cc:240] Negotiation successful
0810 04:45:07.570402 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":87,"threads_started":1}
I20260810 04:45:07.571040 23909 heartbeater.cc:347] Connected to a master server at 127.22.167.126:45405
I20260810 04:45:07.571148 23909 heartbeater.cc:464] Registering TS with master...
I20260810 04:45:07.571324 23909 heartbeater.cc:511] Master 127.22.167.126:45405 requested a full tablet report, sending...
I20260810 04:45:07.571693 23394 ts_manager.cc:194] Registered new tserver with Master: 10ca15d771b343c3a7cdc7842f7c56bf (127.22.167.66:38317)
I20260810 04:45:07.572145 23394 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.167.66@KRBTEST.COM'} at 127.22.167.66:52845
WARNING: no policy specified for kudu/127.22.167.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.167.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.167.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786336863388938-23197-0/krb5kdc/kudu_127.22.167.67.keytab.
Entry for principal kudu/127.22.167.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786336863388938-23197-0/krb5kdc/kudu_127.22.167.67.keytab.
I20260810 04:45:07.593335 23197 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskemv_AM/build/release/bin/kudu
/tmp/dist-test-taskemv_AM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786336863388938-23197-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786336863388938-23197-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786336863388938-23197-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786336863388938-23197-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.167.67:0
--local_ip_for_outbound_sockets=127.22.167.67
--webserver_interface=127.22.167.67
--webserver_port=0
--tserver_master_addrs=127.22.167.126:45405
--keytab_file=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786336863388938-23197-0/krb5kdc/kudu_127.22.167.67.keytab
--principal=kudu/127.22.167.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.167.84:36897
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786336863388938-23197-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786336863388938-23197-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786336863388938-23197-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 04:45:07.675050 23918 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 04:45:07.675241 23918 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 04:45:07.675271 23918 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 04:45:07.676378 23918 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 04:45:07.676417 23918 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 04:45:07.676451 23918 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.167.67
I20260810 04:45:07.677863 23918 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.167.84:36897
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786336863388938-23197-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786336863388938-23197-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786336863388938-23197-0/krb5kdc/kudu_127.22.167.67.keytab
--principal=kudu/127.22.167.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.167.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786336863388938-23197-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.22.167.67
--webserver_port=0
--tserver_master_addrs=127.22.167.126:45405
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23918
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.167.67
--log_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786336863388938-23197-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 03:54:01 UTC on ccycloud.quasar-icijmt.root.comops.site
build id 4577009
I20260810 04:45:07.678093 23918 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 04:45:07.678319 23918 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 04:45:07.683022 23924 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 04:45:07.683012 23926 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 04:45:07.683009 23923 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 04:45:07.683318 23918 server_base.cc:1034] running on GCE node
I20260810 04:45:07.683475 23918 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 04:45:07.683686 23918 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 04:45:07.684811 23918 hybrid_clock.cc:648] HybridClock initialized: now 1786337107684802 us; error 26 us; skew 500 ppm
Aug 10 04:45:07 dist-test-slave-bqcl krb5kdc[23335](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786337107, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.167.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 04:45:07.687299 23918 init.cc:377] Logged in from keytab as kudu/127.22.167.67@KRBTEST.COM (short username kudu)
I20260810 04:45:07.687683 23918 webserver.cc:460] Webserver started at http://127.22.167.67:39319/ using document root <none> and password file <none>
I20260810 04:45:07.687873 23918 fs_manager.cc:359] Metadata directory not provided
I20260810 04:45:07.687912 23918 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 04:45:07.688005 23918 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 04:45:07.689003 23918 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786336863388938-23197-0/minicluster-data/ts-2/data/instance:
uuid: "8bdc1ba2271444108715609e0ab576df"
format_stamp: "Formatted at 2026-08-10 04:45:07 on dist-test-slave-bqcl"
server_key: "5b19b8cc531673512f55bd39d4c8e535"
server_key_iv: "c6eaa5185f4184b35490ae580e92c9f1"
server_key_version: "encryptionkey@0"
I20260810 04:45:07.689294 23918 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786336863388938-23197-0/minicluster-data/ts-2/wal/instance:
uuid: "8bdc1ba2271444108715609e0ab576df"
format_stamp: "Formatted at 2026-08-10 04:45:07 on dist-test-slave-bqcl"
server_key: "5b19b8cc531673512f55bd39d4c8e535"
server_key_iv: "c6eaa5185f4184b35490ae580e92c9f1"
server_key_version: "encryptionkey@0"
I20260810 04:45:07.690450 23918 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260810 04:45:07.691089 23938 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 04:45:07.691263 23918 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260810 04:45:07.691349 23918 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786336863388938-23197-0/minicluster-data/ts-2/data,/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786336863388938-23197-0/minicluster-data/ts-2/wal
uuid: "8bdc1ba2271444108715609e0ab576df"
format_stamp: "Formatted at 2026-08-10 04:45:07 on dist-test-slave-bqcl"
server_key: "5b19b8cc531673512f55bd39d4c8e535"
server_key_iv: "c6eaa5185f4184b35490ae580e92c9f1"
server_key_version: "encryptionkey@0"
I20260810 04:45:07.691417 23918 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786336863388938-23197-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786336863388938-23197-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786336863388938-23197-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 04:45:07.704478 23918 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 04:45:07.706557 23918 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 04:45:07.706682 23918 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 04:45:07.706864 23918 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 04:45:07.707137 23918 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 04:45:07.707170 23918 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 04:45:07.707190 23918 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 04:45:07.707213 23918 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 04:45:07.712647 23918 rpc_server.cc:307] RPC server started. Bound to: 127.22.167.67:44447
I20260810 04:45:07.712705 24125 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.167.67:44447 every 8 connection(s)
I20260810 04:45:07.712976 23918 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786336863388938-23197-0/minicluster-data/ts-2/data/info.pb
Aug 10 04:45:07 dist-test-slave-bqcl krb5kdc[23335](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786337107, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.167.67@KRBTEST.COM for kudu/127.22.167.126@KRBTEST.COM
I20260810 04:45:07.718703 23197 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskemv_AM/build/release/bin/kudu as pid 23918
I20260810 04:45:07.718818 23197 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786336863388938-23197-0/minicluster-data/ts-2/wal/instance
I20260810 04:45:07.719008 23197 external_mini_cluster.cc:1428] Setting key 713392e6793c597b057f9713fee2cf1f
I20260810 04:45:07.721529 23452 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:45:07.713608 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.167.67:55475 (local address 127.22.167.126:45405)
0810 04:45:07.713656 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0810 04:45:07.713658 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 04:45:07.713859 (+ 201us) server_negotiation.cc:407] Connection header received
0810 04:45:07.714355 (+ 496us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:45:07.714356 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:45:07.714367 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:45:07.714414 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0810 04:45:07.715236 (+ 822us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:45:07.715717 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:45:07.716708 (+ 991us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:45:07.716893 (+ 185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:45:07.716965 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:45:07.719159 (+ 2194us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 04:45:07.719160 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 04:45:07.719161 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 04:45:07.719170 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0810 04:45:07.720524 (+ 1354us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 04:45:07.720861 (+ 337us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 04:45:07.720861 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 04:45:07.720862 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 04:45:07.720910 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 04:45:07.721082 (+ 172us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 04:45:07.721082 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 04:45:07.721082 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 04:45:07.721171 (+ 89us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 04:45:07.721215 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0810 04:45:07.721400 (+ 185us) server_negotiation.cc:299] Negotiation successful
0810 04:45:07.721451 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260810 04:45:07.721668 24130 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:45:07.713577 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.126:45405 (local address 127.22.167.67:55475)
0810 04:45:07.713755 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0810 04:45:07.713789 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0810 04:45:07.714277 (+ 488us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 04:45:07.714465 (+ 188us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 04:45:07.714469 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 04:45:07.714991 (+ 522us) client_negotiation.cc:189] Negotiated authn=SASL
0810 04:45:07.715175 (+ 184us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:45:07.715179 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:45:07.715775 (+ 596us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:45:07.715776 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:45:07.716642 (+ 866us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:45:07.716644 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:45:07.716979 (+ 335us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:45:07.716980 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:45:07.717090 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:45:07.717572 (+ 482us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 04:45:07.717583 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0810 04:45:07.719065 (+ 1482us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 04:45:07.720587 (+ 1522us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 04:45:07.720594 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 04:45:07.720595 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 04:45:07.720804 (+ 209us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 04:45:07.720954 (+ 150us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 04:45:07.720955 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 04:45:07.720955 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 04:45:07.721042 (+ 87us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 04:45:07.721212 (+ 170us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 04:45:07.721215 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 04:45:07.721315 (+ 100us) client_negotiation.cc:769] Sending connection context
0810 04:45:07.721388 (+ 73us) client_negotiation.cc:240] Negotiation successful
0810 04:45:07.721497 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":115,"threads_started":1}
I20260810 04:45:07.722072 24126 heartbeater.cc:347] Connected to a master server at 127.22.167.126:45405
I20260810 04:45:07.722187 24126 heartbeater.cc:464] Registering TS with master...
I20260810 04:45:07.722337 24126 heartbeater.cc:511] Master 127.22.167.126:45405 requested a full tablet report, sending...
I20260810 04:45:07.722647 23394 ts_manager.cc:194] Registered new tserver with Master: 8bdc1ba2271444108715609e0ab576df (127.22.167.67:44447)
I20260810 04:45:07.722965 23394 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.167.67@KRBTEST.COM'} at 127.22.167.67:55475
I20260810 04:45:07.730867 23197 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260810 04:45:07.736955 23452 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:45:07.732074 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47256 (local address 127.22.167.126:45405)
0810 04:45:07.732124 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0810 04:45:07.732125 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0810 04:45:07.732243 (+ 118us) server_negotiation.cc:407] Connection header received
0810 04:45:07.732280 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:45:07.732281 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:45:07.732289 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:45:07.732332 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0810 04:45:07.732814 (+ 482us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:45:07.733271 (+ 457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:45:07.733916 (+ 645us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:45:07.734098 (+ 182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:45:07.734207 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:45:07.735139 (+ 932us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 04:45:07.735141 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 04:45:07.735141 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 04:45:07.735147 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0810 04:45:07.736164 (+ 1017us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 04:45:07.736429 (+ 265us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 04:45:07.736430 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 04:45:07.736430 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 04:45:07.736468 (+ 38us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 04:45:07.736577 (+ 109us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 04:45:07.736577 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 04:45:07.736577 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 04:45:07.736691 (+ 114us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 04:45:07.736759 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0810 04:45:07.736826 (+ 67us) server_negotiation.cc:299] Negotiation successful
0810 04:45:07.736876 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260810 04:45:07.738312 23394 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47256:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 04:45:07.739811 23394 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 04:45:07.745849 24141 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:45:07.741787 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.65:46777 (local address 127.0.0.1:50196)
0810 04:45:07.742033 (+ 246us) negotiation.cc:107] Waiting for socket to connect
0810 04:45:07.742042 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0810 04:45:07.742084 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 04:45:07.742646 (+ 562us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 04:45:07.742647 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 04:45:07.742649 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 04:45:07.742703 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:45:07.742704 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:45:07.743643 (+ 939us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:45:07.743644 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:45:07.744919 (+ 1275us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:45:07.744921 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:45:07.745565 (+ 644us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:45:07.745566 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:45:07.745621 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:45:07.745638 (+ 17us) client_negotiation.cc:769] Sending connection context
0810 04:45:07.745683 (+ 45us) client_negotiation.cc:240] Negotiation successful
0810 04:45:07.745732 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":235,"mutex_wait_us":1,"thread_start_us":51,"threads_started":1}
I20260810 04:45:07.745904 24146 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:45:07.741881 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59942 (local address 127.22.167.66:38317)
0810 04:45:07.742246 (+ 365us) server_negotiation.cc:206] Beginning negotiation
0810 04:45:07.742251 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 04:45:07.742260 (+ 9us) server_negotiation.cc:407] Connection header received
0810 04:45:07.742298 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:45:07.742299 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:45:07.742313 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:45:07.742417 (+ 104us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 04:45:07.742599 (+ 182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:45:07.743703 (+ 1104us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:45:07.744786 (+ 1083us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:45:07.745443 (+ 657us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:45:07.745515 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:45:07.745571 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0810 04:45:07.745780 (+ 209us) server_negotiation.cc:299] Negotiation successful
0810 04:45:07.745827 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":329,"thread_start_us":74,"threads_started":1}
I20260810 04:45:07.745913 24145 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:45:07.741776 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.66:38317 (local address 127.0.0.1:59942)
0810 04:45:07.741940 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0810 04:45:07.741978 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0810 04:45:07.742058 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 04:45:07.742434 (+ 376us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 04:45:07.742437 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 04:45:07.742439 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 04:45:07.742510 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:45:07.742513 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:45:07.743814 (+ 1301us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:45:07.743815 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:45:07.744701 (+ 886us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:45:07.744704 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:45:07.745538 (+ 834us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:45:07.745539 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:45:07.745621 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:45:07.745638 (+ 17us) client_negotiation.cc:769] Sending connection context
0810 04:45:07.745702 (+ 64us) client_negotiation.cc:240] Negotiation successful
0810 04:45:07.745732 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":141,"mutex_wait_us":10,"thread_start_us":66,"threads_started":1}
I20260810 04:45:07.745999 24147 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:45:07.742017 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56766 (local address 127.22.167.67:44447)
0810 04:45:07.742208 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0810 04:45:07.742213 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 04:45:07.742225 (+ 12us) server_negotiation.cc:407] Connection header received
0810 04:45:07.742264 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:45:07.742267 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:45:07.742282 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:45:07.742365 (+ 83us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 04:45:07.742591 (+ 226us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:45:07.743665 (+ 1074us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:45:07.744975 (+ 1310us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:45:07.745611 (+ 636us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:45:07.745678 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:45:07.745727 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0810 04:45:07.745888 (+ 161us) server_negotiation.cc:299] Negotiation successful
0810 04:45:07.745927 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":161,"thread_start_us":75,"threads_started":1}
I20260810 04:45:07.746021 24144 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:45:07.741787 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.67:44447 (local address 127.0.0.1:56766)
0810 04:45:07.742163 (+ 376us) negotiation.cc:107] Waiting for socket to connect
0810 04:45:07.742171 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0810 04:45:07.742213 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 04:45:07.742434 (+ 221us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 04:45:07.742437 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 04:45:07.742441 (+ 4us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 04:45:07.742510 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:45:07.742512 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:45:07.743777 (+ 1265us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:45:07.743778 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:45:07.744890 (+ 1112us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:45:07.744893 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:45:07.745676 (+ 783us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:45:07.745677 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:45:07.745738 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:45:07.745742 (+ 4us) client_negotiation.cc:769] Sending connection context
0810 04:45:07.745789 (+ 47us) client_negotiation.cc:240] Negotiation successful
0810 04:45:07.745832 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":368,"mutex_wait_us":38,"thread_start_us":52,"threads_started":1}
I20260810 04:45:07.746397 24148 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:45:07.742209 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50196 (local address 127.22.167.65:46777)
0810 04:45:07.742392 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0810 04:45:07.742396 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 04:45:07.742405 (+ 9us) server_negotiation.cc:407] Connection header received
0810 04:45:07.742442 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:45:07.742444 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:45:07.742457 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:45:07.742541 (+ 84us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 04:45:07.742785 (+ 244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:45:07.743556 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:45:07.744996 (+ 1440us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:45:07.745475 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:45:07.746208 (+ 733us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:45:07.746274 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0810 04:45:07.746300 (+ 26us) server_negotiation.cc:299] Negotiation successful
0810 04:45:07.746339 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":146,"thread_start_us":101,"threads_started":1}
I20260810 04:45:07.746842 23805 tablet_service.cc:1467] Processing CreateTablet for tablet 7a5a5768132641129da1182fc937174d (DEFAULT_TABLE table=test-table [id=58c402f09d3b445288bfada09861a217]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 04:45:07.746953 24018 tablet_service.cc:1467] Processing CreateTablet for tablet 7a5a5768132641129da1182fc937174d (DEFAULT_TABLE table=test-table [id=58c402f09d3b445288bfada09861a217]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 04:45:07.747103 23591 tablet_service.cc:1467] Processing CreateTablet for tablet 7a5a5768132641129da1182fc937174d (DEFAULT_TABLE table=test-table [id=58c402f09d3b445288bfada09861a217]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 04:45:07.747197 24018 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7a5a5768132641129da1182fc937174d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 04:45:07.747197 23805 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7a5a5768132641129da1182fc937174d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 04:45:07.747350 23591 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7a5a5768132641129da1182fc937174d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 04:45:07.749898 24152 tablet_bootstrap.cc:492] T 7a5a5768132641129da1182fc937174d P 4451340e55fb40b6af3c77554164810e: Bootstrap starting.
I20260810 04:45:07.750319 24153 tablet_bootstrap.cc:492] T 7a5a5768132641129da1182fc937174d P 8bdc1ba2271444108715609e0ab576df: Bootstrap starting.
I20260810 04:45:07.750545 24154 tablet_bootstrap.cc:492] T 7a5a5768132641129da1182fc937174d P 10ca15d771b343c3a7cdc7842f7c56bf: Bootstrap starting.
I20260810 04:45:07.750574 24152 tablet_bootstrap.cc:654] T 7a5a5768132641129da1182fc937174d P 4451340e55fb40b6af3c77554164810e: Neither blocks nor log segments found. Creating new log.
I20260810 04:45:07.751132 24152 log.cc:824] T 7a5a5768132641129da1182fc937174d P 4451340e55fb40b6af3c77554164810e: Log is configured to *not* fsync() on all Append() calls
I20260810 04:45:07.751186 24154 tablet_bootstrap.cc:654] T 7a5a5768132641129da1182fc937174d P 10ca15d771b343c3a7cdc7842f7c56bf: Neither blocks nor log segments found. Creating new log.
I20260810 04:45:07.751722 24154 log.cc:824] T 7a5a5768132641129da1182fc937174d P 10ca15d771b343c3a7cdc7842f7c56bf: Log is configured to *not* fsync() on all Append() calls
I20260810 04:45:07.751878 24152 tablet_bootstrap.cc:492] T 7a5a5768132641129da1182fc937174d P 4451340e55fb40b6af3c77554164810e: No bootstrap required, opened a new log
I20260810 04:45:07.751950 24152 ts_tablet_manager.cc:1397] T 7a5a5768132641129da1182fc937174d P 4451340e55fb40b6af3c77554164810e: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20260810 04:45:07.752386 24154 tablet_bootstrap.cc:492] T 7a5a5768132641129da1182fc937174d P 10ca15d771b343c3a7cdc7842f7c56bf: No bootstrap required, opened a new log
I20260810 04:45:07.752449 24154 ts_tablet_manager.cc:1397] T 7a5a5768132641129da1182fc937174d P 10ca15d771b343c3a7cdc7842f7c56bf: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260810 04:45:07.752553 24153 tablet_bootstrap.cc:654] T 7a5a5768132641129da1182fc937174d P 8bdc1ba2271444108715609e0ab576df: Neither blocks nor log segments found. Creating new log.
I20260810 04:45:07.753080 24153 log.cc:824] T 7a5a5768132641129da1182fc937174d P 8bdc1ba2271444108715609e0ab576df: Log is configured to *not* fsync() on all Append() calls
I20260810 04:45:07.753805 24153 tablet_bootstrap.cc:492] T 7a5a5768132641129da1182fc937174d P 8bdc1ba2271444108715609e0ab576df: No bootstrap required, opened a new log
I20260810 04:45:07.753880 24153 ts_tablet_manager.cc:1397] T 7a5a5768132641129da1182fc937174d P 8bdc1ba2271444108715609e0ab576df: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20260810 04:45:07.753962 24152 raft_consensus.cc:348] T 7a5a5768132641129da1182fc937174d P 4451340e55fb40b6af3c77554164810e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "10ca15d771b343c3a7cdc7842f7c56bf" member_type: VOTER last_known_addr { host: "127.22.167.66" port: 38317 } } peers { permanent_uuid: "4451340e55fb40b6af3c77554164810e" member_type: VOTER last_known_addr { host: "127.22.167.65" port: 46777 } } peers { permanent_uuid: "8bdc1ba2271444108715609e0ab576df" member_type: VOTER last_known_addr { host: "127.22.167.67" port: 44447 } }
I20260810 04:45:07.754112 24152 raft_consensus.cc:374] T 7a5a5768132641129da1182fc937174d P 4451340e55fb40b6af3c77554164810e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 04:45:07.754190 24152 raft_consensus.cc:729] T 7a5a5768132641129da1182fc937174d P 4451340e55fb40b6af3c77554164810e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4451340e55fb40b6af3c77554164810e, State: Initialized, Role: FOLLOWER
I20260810 04:45:07.754191 24154 raft_consensus.cc:348] T 7a5a5768132641129da1182fc937174d P 10ca15d771b343c3a7cdc7842f7c56bf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "10ca15d771b343c3a7cdc7842f7c56bf" member_type: VOTER last_known_addr { host: "127.22.167.66" port: 38317 } } peers { permanent_uuid: "4451340e55fb40b6af3c77554164810e" member_type: VOTER last_known_addr { host: "127.22.167.65" port: 46777 } } peers { permanent_uuid: "8bdc1ba2271444108715609e0ab576df" member_type: VOTER last_known_addr { host: "127.22.167.67" port: 44447 } }
I20260810 04:45:07.754305 24152 consensus_queue.cc:260] T 7a5a5768132641129da1182fc937174d P 4451340e55fb40b6af3c77554164810e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "10ca15d771b343c3a7cdc7842f7c56bf" member_type: VOTER last_known_addr { host: "127.22.167.66" port: 38317 } } peers { permanent_uuid: "4451340e55fb40b6af3c77554164810e" member_type: VOTER last_known_addr { host: "127.22.167.65" port: 46777 } } peers { permanent_uuid: "8bdc1ba2271444108715609e0ab576df" member_type: VOTER last_known_addr { host: "127.22.167.67" port: 44447 } }
I20260810 04:45:07.754359 24154 raft_consensus.cc:374] T 7a5a5768132641129da1182fc937174d P 10ca15d771b343c3a7cdc7842f7c56bf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 04:45:07.754398 24154 raft_consensus.cc:729] T 7a5a5768132641129da1182fc937174d P 10ca15d771b343c3a7cdc7842f7c56bf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 10ca15d771b343c3a7cdc7842f7c56bf, State: Initialized, Role: FOLLOWER
I20260810 04:45:07.754556 24152 ts_tablet_manager.cc:1428] T 7a5a5768132641129da1182fc937174d P 4451340e55fb40b6af3c77554164810e: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260810 04:45:07.754532 24154 consensus_queue.cc:260] T 7a5a5768132641129da1182fc937174d P 10ca15d771b343c3a7cdc7842f7c56bf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "10ca15d771b343c3a7cdc7842f7c56bf" member_type: VOTER last_known_addr { host: "127.22.167.66" port: 38317 } } peers { permanent_uuid: "4451340e55fb40b6af3c77554164810e" member_type: VOTER last_known_addr { host: "127.22.167.65" port: 46777 } } peers { permanent_uuid: "8bdc1ba2271444108715609e0ab576df" member_type: VOTER last_known_addr { host: "127.22.167.67" port: 44447 } }
I20260810 04:45:07.754678 23702 heartbeater.cc:503] Master 127.22.167.126:45405 was elected leader, sending a full tablet report...
I20260810 04:45:07.754808 24154 ts_tablet_manager.cc:1428] T 7a5a5768132641129da1182fc937174d P 10ca15d771b343c3a7cdc7842f7c56bf: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260810 04:45:07.754916 23909 heartbeater.cc:503] Master 127.22.167.126:45405 was elected leader, sending a full tablet report...
I20260810 04:45:07.755725 24153 raft_consensus.cc:348] T 7a5a5768132641129da1182fc937174d P 8bdc1ba2271444108715609e0ab576df [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "10ca15d771b343c3a7cdc7842f7c56bf" member_type: VOTER last_known_addr { host: "127.22.167.66" port: 38317 } } peers { permanent_uuid: "4451340e55fb40b6af3c77554164810e" member_type: VOTER last_known_addr { host: "127.22.167.65" port: 46777 } } peers { permanent_uuid: "8bdc1ba2271444108715609e0ab576df" member_type: VOTER last_known_addr { host: "127.22.167.67" port: 44447 } }
I20260810 04:45:07.755892 24153 raft_consensus.cc:374] T 7a5a5768132641129da1182fc937174d P 8bdc1ba2271444108715609e0ab576df [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 04:45:07.755935 24153 raft_consensus.cc:729] T 7a5a5768132641129da1182fc937174d P 8bdc1ba2271444108715609e0ab576df [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8bdc1ba2271444108715609e0ab576df, State: Initialized, Role: FOLLOWER
I20260810 04:45:07.756042 24153 consensus_queue.cc:260] T 7a5a5768132641129da1182fc937174d P 8bdc1ba2271444108715609e0ab576df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "10ca15d771b343c3a7cdc7842f7c56bf" member_type: VOTER last_known_addr { host: "127.22.167.66" port: 38317 } } peers { permanent_uuid: "4451340e55fb40b6af3c77554164810e" member_type: VOTER last_known_addr { host: "127.22.167.65" port: 46777 } } peers { permanent_uuid: "8bdc1ba2271444108715609e0ab576df" member_type: VOTER last_known_addr { host: "127.22.167.67" port: 44447 } }
I20260810 04:45:07.756287 24153 ts_tablet_manager.cc:1428] T 7a5a5768132641129da1182fc937174d P 8bdc1ba2271444108715609e0ab576df: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260810 04:45:07.756346 24126 heartbeater.cc:503] Master 127.22.167.126:45405 was elected leader, sending a full tablet report...
W20260810 04:45:07.812443 23911 tablet.cc:2369] T 7a5a5768132641129da1182fc937174d P 10ca15d771b343c3a7cdc7842f7c56bf: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260810 04:45:07.907418 23704 tablet.cc:2369] T 7a5a5768132641129da1182fc937174d P 4451340e55fb40b6af3c77554164810e: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260810 04:45:07.963518 24128 tablet.cc:2369] T 7a5a5768132641129da1182fc937174d P 8bdc1ba2271444108715609e0ab576df: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 04:45:07.976774 24164 raft_consensus.cc:482] T 7a5a5768132641129da1182fc937174d P 8bdc1ba2271444108715609e0ab576df [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 04:45:07.976914 24164 raft_consensus.cc:504] T 7a5a5768132641129da1182fc937174d P 8bdc1ba2271444108715609e0ab576df [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "10ca15d771b343c3a7cdc7842f7c56bf" member_type: VOTER last_known_addr { host: "127.22.167.66" port: 38317 } } peers { permanent_uuid: "4451340e55fb40b6af3c77554164810e" member_type: VOTER last_known_addr { host: "127.22.167.65" port: 46777 } } peers { permanent_uuid: "8bdc1ba2271444108715609e0ab576df" member_type: VOTER last_known_addr { host: "127.22.167.67" port: 44447 } }
I20260810 04:45:07.977299 24164 leader_election.cc:288] T 7a5a5768132641129da1182fc937174d P 8bdc1ba2271444108715609e0ab576df [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 10ca15d771b343c3a7cdc7842f7c56bf (127.22.167.66:38317), 4451340e55fb40b6af3c77554164810e (127.22.167.65:46777)
I20260810 04:45:07.980050 24146 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:45:07.977603 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.167.67:39749 (local address 127.22.167.66:38317)
0810 04:45:07.977657 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0810 04:45:07.977670 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0810 04:45:07.977678 (+ 8us) server_negotiation.cc:407] Connection header received
0810 04:45:07.977702 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:45:07.977703 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:45:07.977711 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:45:07.977757 (+ 46us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 04:45:07.977905 (+ 148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:45:07.978407 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:45:07.979302 (+ 895us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:45:07.979738 (+ 436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:45:07.979804 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:45:07.979814 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0810 04:45:07.979949 (+ 135us) server_negotiation.cc:299] Negotiation successful
0810 04:45:07.979984 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260810 04:45:07.980050 24130 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:45:07.977484 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.66:38317 (local address 127.22.167.67:39749)
0810 04:45:07.977548 (+ 64us) negotiation.cc:107] Waiting for socket to connect
0810 04:45:07.977560 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0810 04:45:07.977605 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 04:45:07.977773 (+ 168us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 04:45:07.977774 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 04:45:07.977775 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 04:45:07.977845 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:45:07.977847 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:45:07.978480 (+ 633us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:45:07.978481 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:45:07.979211 (+ 730us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:45:07.979213 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:45:07.979810 (+ 597us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:45:07.979811 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:45:07.979885 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:45:07.979888 (+ 3us) client_negotiation.cc:769] Sending connection context
0810 04:45:07.979934 (+ 46us) client_negotiation.cc:240] Negotiation successful
0810 04:45:07.979974 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":43}
I20260810 04:45:07.980151 24148 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:45:07.977541 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.167.67:49671 (local address 127.22.167.65:46777)
0810 04:45:07.977584 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0810 04:45:07.977586 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 04:45:07.977769 (+ 183us) server_negotiation.cc:407] Connection header received
0810 04:45:07.977806 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:45:07.977807 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:45:07.977817 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:45:07.977864 (+ 47us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 04:45:07.978003 (+ 139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:45:07.978540 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:45:07.979457 (+ 917us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:45:07.979827 (+ 370us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:45:07.979889 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:45:07.979899 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0810 04:45:07.980035 (+ 136us) server_negotiation.cc:299] Negotiation successful
0810 04:45:07.980081 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260810 04:45:07.980165 24165 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:45:07.977497 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.65:46777 (local address 127.22.167.67:49671)
0810 04:45:07.977691 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0810 04:45:07.977700 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0810 04:45:07.977758 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 04:45:07.977872 (+ 114us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 04:45:07.977873 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 04:45:07.977874 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 04:45:07.977944 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:45:07.977945 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:45:07.978617 (+ 672us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:45:07.978618 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:45:07.979366 (+ 748us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:45:07.979368 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:45:07.979904 (+ 536us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:45:07.979905 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:45:07.979972 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:45:07.979976 (+ 4us) client_negotiation.cc:769] Sending connection context
0810 04:45:07.980020 (+ 44us) client_negotiation.cc:240] Negotiation successful
0810 04:45:07.980059 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":174,"thread_start_us":121,"threads_started":1}
I20260810 04:45:07.980420 23834 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7a5a5768132641129da1182fc937174d" candidate_uuid: "8bdc1ba2271444108715609e0ab576df" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "10ca15d771b343c3a7cdc7842f7c56bf" is_pre_election: true
I20260810 04:45:07.980479 23629 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7a5a5768132641129da1182fc937174d" candidate_uuid: "8bdc1ba2271444108715609e0ab576df" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4451340e55fb40b6af3c77554164810e" is_pre_election: true
I20260810 04:45:07.980602 23834 raft_consensus.cc:2454] T 7a5a5768132641129da1182fc937174d P 10ca15d771b343c3a7cdc7842f7c56bf [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8bdc1ba2271444108715609e0ab576df in term 0.
I20260810 04:45:07.980616 23629 raft_consensus.cc:2454] T 7a5a5768132641129da1182fc937174d P 4451340e55fb40b6af3c77554164810e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8bdc1ba2271444108715609e0ab576df in term 0.
I20260810 04:45:07.980849 23939 leader_election.cc:302] T 7a5a5768132641129da1182fc937174d P 8bdc1ba2271444108715609e0ab576df [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 10ca15d771b343c3a7cdc7842f7c56bf, 8bdc1ba2271444108715609e0ab576df; no voters:
I20260810 04:45:07.981020 24164 raft_consensus.cc:2781] T 7a5a5768132641129da1182fc937174d P 8bdc1ba2271444108715609e0ab576df [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 04:45:07.981076 24164 raft_consensus.cc:482] T 7a5a5768132641129da1182fc937174d P 8bdc1ba2271444108715609e0ab576df [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 04:45:07.981091 24164 raft_consensus.cc:3037] T 7a5a5768132641129da1182fc937174d P 8bdc1ba2271444108715609e0ab576df [term 0 FOLLOWER]: Advancing to term 1
I20260810 04:45:07.981798 24164 raft_consensus.cc:504] T 7a5a5768132641129da1182fc937174d P 8bdc1ba2271444108715609e0ab576df [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "10ca15d771b343c3a7cdc7842f7c56bf" member_type: VOTER last_known_addr { host: "127.22.167.66" port: 38317 } } peers { permanent_uuid: "4451340e55fb40b6af3c77554164810e" member_type: VOTER last_known_addr { host: "127.22.167.65" port: 46777 } } peers { permanent_uuid: "8bdc1ba2271444108715609e0ab576df" member_type: VOTER last_known_addr { host: "127.22.167.67" port: 44447 } }
I20260810 04:45:07.981927 24164 leader_election.cc:288] T 7a5a5768132641129da1182fc937174d P 8bdc1ba2271444108715609e0ab576df [CANDIDATE]: Term 1 election: Requested vote from peers 10ca15d771b343c3a7cdc7842f7c56bf (127.22.167.66:38317), 4451340e55fb40b6af3c77554164810e (127.22.167.65:46777)
I20260810 04:45:07.982089 23834 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7a5a5768132641129da1182fc937174d" candidate_uuid: "8bdc1ba2271444108715609e0ab576df" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "10ca15d771b343c3a7cdc7842f7c56bf"
I20260810 04:45:07.982177 23629 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7a5a5768132641129da1182fc937174d" candidate_uuid: "8bdc1ba2271444108715609e0ab576df" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4451340e55fb40b6af3c77554164810e"
I20260810 04:45:07.982193 23834 raft_consensus.cc:3037] T 7a5a5768132641129da1182fc937174d P 10ca15d771b343c3a7cdc7842f7c56bf [term 0 FOLLOWER]: Advancing to term 1
I20260810 04:45:07.982259 23629 raft_consensus.cc:3037] T 7a5a5768132641129da1182fc937174d P 4451340e55fb40b6af3c77554164810e [term 0 FOLLOWER]: Advancing to term 1
I20260810 04:45:07.982939 23834 raft_consensus.cc:2454] T 7a5a5768132641129da1182fc937174d P 10ca15d771b343c3a7cdc7842f7c56bf [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8bdc1ba2271444108715609e0ab576df in term 1.
I20260810 04:45:07.982944 23629 raft_consensus.cc:2454] T 7a5a5768132641129da1182fc937174d P 4451340e55fb40b6af3c77554164810e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8bdc1ba2271444108715609e0ab576df in term 1.
I20260810 04:45:07.983103 23939 leader_election.cc:302] T 7a5a5768132641129da1182fc937174d P 8bdc1ba2271444108715609e0ab576df [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 10ca15d771b343c3a7cdc7842f7c56bf, 8bdc1ba2271444108715609e0ab576df; no voters:
I20260810 04:45:07.983189 24164 raft_consensus.cc:2781] T 7a5a5768132641129da1182fc937174d P 8bdc1ba2271444108715609e0ab576df [term 1 FOLLOWER]: Leader election won for term 1
I20260810 04:45:07.983327 24164 raft_consensus.cc:686] T 7a5a5768132641129da1182fc937174d P 8bdc1ba2271444108715609e0ab576df [term 1 LEADER]: Becoming Leader. State: Replica: 8bdc1ba2271444108715609e0ab576df, State: Running, Role: LEADER
I20260810 04:45:07.983415 24164 consensus_queue.cc:237] T 7a5a5768132641129da1182fc937174d P 8bdc1ba2271444108715609e0ab576df [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "10ca15d771b343c3a7cdc7842f7c56bf" member_type: VOTER last_known_addr { host: "127.22.167.66" port: 38317 } } peers { permanent_uuid: "4451340e55fb40b6af3c77554164810e" member_type: VOTER last_known_addr { host: "127.22.167.65" port: 46777 } } peers { permanent_uuid: "8bdc1ba2271444108715609e0ab576df" member_type: VOTER last_known_addr { host: "127.22.167.67" port: 44447 } }
I20260810 04:45:07.984200 23394 catalog_manager.cc:5495] T 7a5a5768132641129da1182fc937174d P 8bdc1ba2271444108715609e0ab576df reported cstate change: term changed from 0 to 1, leader changed from <none> to 8bdc1ba2271444108715609e0ab576df (127.22.167.67). New cstate: current_term: 1 leader_uuid: "8bdc1ba2271444108715609e0ab576df" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "10ca15d771b343c3a7cdc7842f7c56bf" member_type: VOTER last_known_addr { host: "127.22.167.66" port: 38317 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4451340e55fb40b6af3c77554164810e" member_type: VOTER last_known_addr { host: "127.22.167.65" port: 46777 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8bdc1ba2271444108715609e0ab576df" member_type: VOTER last_known_addr { host: "127.22.167.67" port: 44447 } health_report { overall_health: HEALTHY } } }
I20260810 04:45:08.025745 24147 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:45:08.023494 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56778 (local address 127.22.167.67:44447)
0810 04:45:08.023544 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0810 04:45:08.023546 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 04:45:08.023553 (+ 7us) server_negotiation.cc:407] Connection header received
0810 04:45:08.023595 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:45:08.023596 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:45:08.023614 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:45:08.023676 (+ 62us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 04:45:08.023818 (+ 142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:45:08.024291 (+ 473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:45:08.025011 (+ 720us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:45:08.025206 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:45:08.025275 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:45:08.025418 (+ 143us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 04:45:08.025526 (+ 108us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 04:45:08.025599 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0810 04:45:08.025622 (+ 23us) server_negotiation.cc:299] Negotiation successful
0810 04:45:08.025643 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260810 04:45:08.027498 23629 raft_consensus.cc:1261] T 7a5a5768132641129da1182fc937174d P 4451340e55fb40b6af3c77554164810e [term 1 FOLLOWER]: Refusing update from remote peer 8bdc1ba2271444108715609e0ab576df: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 04:45:08.027498 23834 raft_consensus.cc:1261] T 7a5a5768132641129da1182fc937174d P 10ca15d771b343c3a7cdc7842f7c56bf [term 1 FOLLOWER]: Refusing update from remote peer 8bdc1ba2271444108715609e0ab576df: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 04:45:08.027870 24168 consensus_queue.cc:1035] T 7a5a5768132641129da1182fc937174d P 8bdc1ba2271444108715609e0ab576df [LEADER]: Connected to new peer: Peer: permanent_uuid: "4451340e55fb40b6af3c77554164810e" member_type: VOTER last_known_addr { host: "127.22.167.65" port: 46777 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 04:45:08.027997 24164 consensus_queue.cc:1035] T 7a5a5768132641129da1182fc937174d P 8bdc1ba2271444108715609e0ab576df [LEADER]: Connected to new peer: Peer: permanent_uuid: "10ca15d771b343c3a7cdc7842f7c56bf" member_type: VOTER last_known_addr { host: "127.22.167.66" port: 38317 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 04:45:08.030659 24179 mvcc.cc:205] Tried to move back new op lower bound from 7316836794478088192 to 7316836794300555264. Current Snapshot: MvccSnapshot[applied={T|T < 7316836794478088192}]
I20260810 04:45:08.030694 24178 mvcc.cc:205] Tried to move back new op lower bound from 7316836794478088192 to 7316836794300555264. Current Snapshot: MvccSnapshot[applied={T|T < 7316836794478088192}]
I20260810 04:45:08.031277 24177 mvcc.cc:205] Tried to move back new op lower bound from 7316836794478088192 to 7316836794300555264. Current Snapshot: MvccSnapshot[applied={T|T < 7316836794478088192 or (T in {7316836794478088192})}]
I20260810 04:45:08.032486 23393 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47256:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 04:45:08.032581 23393 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47256:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 04:45:08.033250 23393 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 4b51fe86ac6c4249943ce952ad24ac8a: Sending DeleteTablet for 3 replicas of tablet 7a5a5768132641129da1182fc937174d
I20260810 04:45:08.033567 24018 tablet_service.cc:1514] Processing DeleteTablet for tablet 7a5a5768132641129da1182fc937174d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 04:45:08 UTC) from {username='kudu', principal='kudu/127.22.167.126@KRBTEST.COM'} at 127.0.0.1:56766
I20260810 04:45:08.033577 23591 tablet_service.cc:1514] Processing DeleteTablet for tablet 7a5a5768132641129da1182fc937174d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 04:45:08 UTC) from {username='kudu', principal='kudu/127.22.167.126@KRBTEST.COM'} at 127.0.0.1:50196
I20260810 04:45:08.033806 23197 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskemv_AM/build/release/bin/kudu with pid 23490
I20260810 04:45:08.033905 23805 tablet_service.cc:1514] Processing DeleteTablet for tablet 7a5a5768132641129da1182fc937174d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 04:45:08 UTC) from {username='kudu', principal='kudu/127.22.167.126@KRBTEST.COM'} at 127.0.0.1:59942
I20260810 04:45:08.036711 24195 tablet_replica.cc:330] T 7a5a5768132641129da1182fc937174d P 10ca15d771b343c3a7cdc7842f7c56bf: stopping tablet replica
I20260810 04:45:08.036834 24195 raft_consensus.cc:2229] T 7a5a5768132641129da1182fc937174d P 10ca15d771b343c3a7cdc7842f7c56bf [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 04:45:08.036952 24195 raft_consensus.cc:2258] T 7a5a5768132641129da1182fc937174d P 10ca15d771b343c3a7cdc7842f7c56bf [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 04:45:08.037137 24193 tablet_replica.cc:330] T 7a5a5768132641129da1182fc937174d P 8bdc1ba2271444108715609e0ab576df: stopping tablet replica
I20260810 04:45:08.037286 24193 raft_consensus.cc:2229] T 7a5a5768132641129da1182fc937174d P 8bdc1ba2271444108715609e0ab576df [term 1 LEADER]: Raft consensus shutting down.
I20260810 04:45:08.037429 24193 raft_consensus.cc:2258] T 7a5a5768132641129da1182fc937174d P 8bdc1ba2271444108715609e0ab576df [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 04:45:08.037474 24195 ts_tablet_manager.cc:1905] T 7a5a5768132641129da1182fc937174d P 10ca15d771b343c3a7cdc7842f7c56bf: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 04:45:08.037868 24193 ts_tablet_manager.cc:1905] T 7a5a5768132641129da1182fc937174d P 8bdc1ba2271444108715609e0ab576df: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 04:45:08.039243 24195 ts_tablet_manager.cc:1918] T 7a5a5768132641129da1182fc937174d P 10ca15d771b343c3a7cdc7842f7c56bf: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 04:45:08.039362 24195 log.cc:1196] T 7a5a5768132641129da1182fc937174d P 10ca15d771b343c3a7cdc7842f7c56bf: Deleting WAL directory at /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786336863388938-23197-0/minicluster-data/ts-1/wal/wals/7a5a5768132641129da1182fc937174d
I20260810 04:45:08.039707 24195 ts_tablet_manager.cc:1939] T 7a5a5768132641129da1182fc937174d P 10ca15d771b343c3a7cdc7842f7c56bf: Deleting consensus metadata
I20260810 04:45:08.039749 24193 ts_tablet_manager.cc:1918] T 7a5a5768132641129da1182fc937174d P 8bdc1ba2271444108715609e0ab576df: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 04:45:08.039909 24193 log.cc:1196] T 7a5a5768132641129da1182fc937174d P 8bdc1ba2271444108715609e0ab576df: Deleting WAL directory at /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786336863388938-23197-0/minicluster-data/ts-2/wal/wals/7a5a5768132641129da1182fc937174d
I20260810 04:45:08.040199 23373 catalog_manager.cc:4841] TS 10ca15d771b343c3a7cdc7842f7c56bf (127.22.167.66:38317): tablet 7a5a5768132641129da1182fc937174d (table test-table [id=58c402f09d3b445288bfada09861a217]) successfully deleted
I20260810 04:45:08.040284 24193 ts_tablet_manager.cc:1939] T 7a5a5768132641129da1182fc937174d P 8bdc1ba2271444108715609e0ab576df: Deleting consensus metadata
I20260810 04:45:08.040771 23375 catalog_manager.cc:4841] TS 8bdc1ba2271444108715609e0ab576df (127.22.167.67:44447): tablet 7a5a5768132641129da1182fc937174d (table test-table [id=58c402f09d3b445288bfada09861a217]) successfully deleted
W20260810 04:45:08.047775 23377 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.22.167.65:46777 (error 108)
I20260810 04:45:08.048035 23197 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskemv_AM/build/release/bin/kudu with pid 23709
I20260810 04:45:08.049880 24144 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:45:08.048139 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.65:46777 (local address 127.0.0.1:50198)
0810 04:45:08.049801 (+ 1662us) negotiation.cc:107] Waiting for socket to connect
0810 04:45:08.049846 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.167.65:46777: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1638}
W20260810 04:45:08.052480 23377 catalog_manager.cc:4584] TS 4451340e55fb40b6af3c77554164810e (127.22.167.65:46777): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 7a5a5768132641129da1182fc937174d: Network error: Client connection negotiation failed: client connection to 127.22.167.65:46777: connect: Connection refused (error 111)
I20260810 04:45:08.059918 23197 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskemv_AM/build/release/bin/kudu with pid 23918
I20260810 04:45:08.069434 23197 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskemv_AM/build/release/bin/kudu with pid 23351
2026-08-10T04:45:08Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/4 (2256 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/5
2026-08-10T04:45:08Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T04:45:08Z Disabled control of system clock
I20260810 04:45:08.102443 23197 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskemv_AM/build/release/bin/kudu
/tmp/dist-test-taskemv_AM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786336863388938-23197-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786336863388938-23197-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786336863388938-23197-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786336863388938-23197-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.167.126:46507
--webserver_interface=127.22.167.126
--webserver_port=0
--builtin_ntp_servers=127.22.167.84:35545
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.167.126:46507
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260810 04:45:08.181798 24200 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 04:45:08.181994 24200 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 04:45:08.182013 24200 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 04:45:08.183137 24200 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 04:45:08.183192 24200 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 04:45:08.183204 24200 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 04:45:08.183215 24200 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 04:45:08.183226 24200 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 04:45:08.184544 24200 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.167.84:35545
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786336863388938-23197-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786336863388938-23197-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.22.167.126:46507
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.167.126:46507
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786336863388938-23197-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.167.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24200
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786336863388938-23197-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 03:54:01 UTC on ccycloud.quasar-icijmt.root.comops.site
build id 4577009
I20260810 04:45:08.184774 24200 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 04:45:08.185060 24200 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260810 04:45:08.189893 24200 server_base.cc:1034] running on GCE node
W20260810 04:45:08.189864 24211 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 04:45:08.189857 24213 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 04:45:08.189940 24210 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 04:45:08.190421 24200 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 04:45:08.190698 24200 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 04:45:08.191838 24200 hybrid_clock.cc:648] HybridClock initialized: now 1786337108191819 us; error 31 us; skew 500 ppm
I20260810 04:45:08.193013 24200 webserver.cc:460] Webserver started at http://127.22.167.126:37107/ using document root <none> and password file <none>
I20260810 04:45:08.193241 24200 fs_manager.cc:359] Metadata directory not provided
I20260810 04:45:08.193292 24200 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 04:45:08.193416 24200 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 04:45:08.194466 24200 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786336863388938-23197-0/minicluster-data/master-0/data/instance:
uuid: "69ad14ffadd1405ca18b013b786b181d"
format_stamp: "Formatted at 2026-08-10 04:45:08 on dist-test-slave-bqcl"
server_key: "d00b0f3dfd5e94d2ca74e666565bf9bb"
server_key_iv: "00d661e16bd6c119a089886e405864da"
server_key_version: "encryptionkey@0"
I20260810 04:45:08.194789 24200 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786336863388938-23197-0/minicluster-data/master-0/wal/instance:
uuid: "69ad14ffadd1405ca18b013b786b181d"
format_stamp: "Formatted at 2026-08-10 04:45:08 on dist-test-slave-bqcl"
server_key: "d00b0f3dfd5e94d2ca74e666565bf9bb"
server_key_iv: "00d661e16bd6c119a089886e405864da"
server_key_version: "encryptionkey@0"
I20260810 04:45:08.195972 24200 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.001s
I20260810 04:45:08.196729 24220 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 04:45:08.196889 24200 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 04:45:08.196974 24200 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786336863388938-23197-0/minicluster-data/master-0/data,/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786336863388938-23197-0/minicluster-data/master-0/wal
uuid: "69ad14ffadd1405ca18b013b786b181d"
format_stamp: "Formatted at 2026-08-10 04:45:08 on dist-test-slave-bqcl"
server_key: "d00b0f3dfd5e94d2ca74e666565bf9bb"
server_key_iv: "00d661e16bd6c119a089886e405864da"
server_key_version: "encryptionkey@0"
I20260810 04:45:08.197032 24200 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786336863388938-23197-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786336863388938-23197-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786336863388938-23197-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 04:45:08.234472 24200 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 04:45:08.234798 24200 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 04:45:08.234935 24200 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 04:45:08.238883 24200 rpc_server.cc:307] RPC server started. Bound to: 127.22.167.126:46507
I20260810 04:45:08.238929 24306 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.167.126:46507 every 8 connection(s)
I20260810 04:45:08.239252 24200 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786336863388938-23197-0/minicluster-data/master-0/data/info.pb
I20260810 04:45:08.240244 24307 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 04:45:08.242718 24307 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 69ad14ffadd1405ca18b013b786b181d: Bootstrap starting.
I20260810 04:45:08.243242 24307 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 69ad14ffadd1405ca18b013b786b181d: Neither blocks nor log segments found. Creating new log.
I20260810 04:45:08.243475 24307 log.cc:824] T 00000000000000000000000000000000 P 69ad14ffadd1405ca18b013b786b181d: Log is configured to *not* fsync() on all Append() calls
I20260810 04:45:08.244099 24307 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 69ad14ffadd1405ca18b013b786b181d: No bootstrap required, opened a new log
I20260810 04:45:08.245651 24307 raft_consensus.cc:348] T 00000000000000000000000000000000 P 69ad14ffadd1405ca18b013b786b181d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "69ad14ffadd1405ca18b013b786b181d" member_type: VOTER last_known_addr { host: "127.22.167.126" port: 46507 } }
I20260810 04:45:08.245787 24307 raft_consensus.cc:374] T 00000000000000000000000000000000 P 69ad14ffadd1405ca18b013b786b181d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 04:45:08.245808 24307 raft_consensus.cc:729] T 00000000000000000000000000000000 P 69ad14ffadd1405ca18b013b786b181d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 69ad14ffadd1405ca18b013b786b181d, State: Initialized, Role: FOLLOWER
I20260810 04:45:08.245913 24307 consensus_queue.cc:260] T 00000000000000000000000000000000 P 69ad14ffadd1405ca18b013b786b181d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "69ad14ffadd1405ca18b013b786b181d" member_type: VOTER last_known_addr { host: "127.22.167.126" port: 46507 } }
I20260810 04:45:08.245965 24307 raft_consensus.cc:388] T 00000000000000000000000000000000 P 69ad14ffadd1405ca18b013b786b181d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 04:45:08.245985 24307 raft_consensus.cc:482] T 00000000000000000000000000000000 P 69ad14ffadd1405ca18b013b786b181d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 04:45:08.246008 24307 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 69ad14ffadd1405ca18b013b786b181d [term 0 FOLLOWER]: Advancing to term 1
I20260810 04:45:08.246526 24307 raft_consensus.cc:504] T 00000000000000000000000000000000 P 69ad14ffadd1405ca18b013b786b181d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "69ad14ffadd1405ca18b013b786b181d" member_type: VOTER last_known_addr { host: "127.22.167.126" port: 46507 } }
I20260810 04:45:08.246614 24307 leader_election.cc:302] T 00000000000000000000000000000000 P 69ad14ffadd1405ca18b013b786b181d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 69ad14ffadd1405ca18b013b786b181d; no voters:
I20260810 04:45:08.246745 24307 leader_election.cc:288] T 00000000000000000000000000000000 P 69ad14ffadd1405ca18b013b786b181d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 04:45:08.246804 24311 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 69ad14ffadd1405ca18b013b786b181d [term 1 FOLLOWER]: Leader election won for term 1
I20260810 04:45:08.246942 23197 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskemv_AM/build/release/bin/kudu as pid 24200
I20260810 04:45:08.246996 24307 sys_catalog.cc:564] T 00000000000000000000000000000000 P 69ad14ffadd1405ca18b013b786b181d [sys.catalog]: configured and running, proceeding with master startup.
I20260810 04:45:08.247025 24311 raft_consensus.cc:686] T 00000000000000000000000000000000 P 69ad14ffadd1405ca18b013b786b181d [term 1 LEADER]: Becoming Leader. State: Replica: 69ad14ffadd1405ca18b013b786b181d, State: Running, Role: LEADER
I20260810 04:45:08.247083 23197 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786336863388938-23197-0/minicluster-data/master-0/wal/instance
I20260810 04:45:08.247121 24311 consensus_queue.cc:237] T 00000000000000000000000000000000 P 69ad14ffadd1405ca18b013b786b181d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "69ad14ffadd1405ca18b013b786b181d" member_type: VOTER last_known_addr { host: "127.22.167.126" port: 46507 } }
I20260810 04:45:08.247291 23197 external_mini_cluster.cc:1428] Setting key fa212517d774bef8e05ecc4c7c71d391
I20260810 04:45:08.247578 24315 sys_catalog.cc:455] T 00000000000000000000000000000000 P 69ad14ffadd1405ca18b013b786b181d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 69ad14ffadd1405ca18b013b786b181d. Latest consensus state: current_term: 1 leader_uuid: "69ad14ffadd1405ca18b013b786b181d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "69ad14ffadd1405ca18b013b786b181d" member_type: VOTER last_known_addr { host: "127.22.167.126" port: 46507 } } }
I20260810 04:45:08.247664 24315 sys_catalog.cc:458] T 00000000000000000000000000000000 P 69ad14ffadd1405ca18b013b786b181d [sys.catalog]: This master's current role is: LEADER
I20260810 04:45:08.247659 24314 sys_catalog.cc:455] T 00000000000000000000000000000000 P 69ad14ffadd1405ca18b013b786b181d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "69ad14ffadd1405ca18b013b786b181d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "69ad14ffadd1405ca18b013b786b181d" member_type: VOTER last_known_addr { host: "127.22.167.126" port: 46507 } } }
I20260810 04:45:08.247905 24314 sys_catalog.cc:458] T 00000000000000000000000000000000 P 69ad14ffadd1405ca18b013b786b181d [sys.catalog]: This master's current role is: LEADER
I20260810 04:45:08.248383 24328 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 04:45:08.248931 24328 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 04:45:08.250555 24328 catalog_manager.cc:1347] Generated new cluster ID: 88cc58da9a8a4d959881d06eca5bd163
I20260810 04:45:08.250607 24328 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 04:45:08.253474 24334 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:45:08.247962 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49946 (local address 127.22.167.126:46507)
0810 04:45:08.248187 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0810 04:45:08.248196 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 04:45:08.248411 (+ 215us) server_negotiation.cc:407] Connection header received
0810 04:45:08.248666 (+ 255us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:45:08.248675 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:45:08.248749 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:45:08.248889 (+ 140us) server_negotiation.cc:226] Negotiated authn=SASL
0810 04:45:08.249105 (+ 216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:45:08.250321 (+ 1216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:45:08.251425 (+ 1104us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:45:08.252013 (+ 588us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:45:08.252129 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:45:08.252804 (+ 675us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 04:45:08.252808 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 04:45:08.252808 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 04:45:08.253006 (+ 198us) server_negotiation.cc:922] calling sasl_server_start()
0810 04:45:08.253046 (+ 40us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 04:45:08.253059 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 04:45:08.253119 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0810 04:45:08.253201 (+ 82us) server_negotiation.cc:299] Negotiation successful
0810 04:45:08.253273 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":151,"thread_start_us":99,"threads_started":1}
I20260810 04:45:08.268879 24328 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 04:45:08.269603 24328 catalog_manager.cc:1504] Loading token signing keys...
I20260810 04:45:08.274326 24328 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 69ad14ffadd1405ca18b013b786b181d: Generated new TSK 0
I20260810 04:45:08.274515 24328 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 04:45:08.280660 23197 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskemv_AM/build/release/bin/kudu
/tmp/dist-test-taskemv_AM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786336863388938-23197-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786336863388938-23197-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786336863388938-23197-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786336863388938-23197-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.167.65:0
--local_ip_for_outbound_sockets=127.22.167.65
--webserver_interface=127.22.167.65
--webserver_port=0
--tserver_master_addrs=127.22.167.126:46507
--builtin_ntp_servers=127.22.167.84:35545
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260810 04:45:08.362097 24344 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 04:45:08.362310 24344 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 04:45:08.362339 24344 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 04:45:08.363428 24344 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 04:45:08.363468 24344 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 04:45:08.363500 24344 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.167.65
I20260810 04:45:08.364785 24344 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.167.84:35545
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786336863388938-23197-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786336863388938-23197-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.167.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786336863388938-23197-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.22.167.65
--webserver_port=0
--tserver_master_addrs=127.22.167.126:46507
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24344
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.167.65
--log_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786336863388938-23197-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 03:54:01 UTC on ccycloud.quasar-icijmt.root.comops.site
build id 4577009
I20260810 04:45:08.365005 24344 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 04:45:08.365293 24344 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 04:45:08.370096 24353 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 04:45:08.370070 24352 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 04:45:08.370105 24356 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 04:45:08.370157 24344 server_base.cc:1034] running on GCE node
I20260810 04:45:08.370509 24344 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 04:45:08.370734 24344 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 04:45:08.371878 24344 hybrid_clock.cc:648] HybridClock initialized: now 1786337108371862 us; error 28 us; skew 500 ppm
I20260810 04:45:08.373059 24344 webserver.cc:460] Webserver started at http://127.22.167.65:38597/ using document root <none> and password file <none>
I20260810 04:45:08.373275 24344 fs_manager.cc:359] Metadata directory not provided
I20260810 04:45:08.373327 24344 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 04:45:08.373451 24344 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 04:45:08.374539 24344 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786336863388938-23197-0/minicluster-data/ts-0/data/instance:
uuid: "bc3ff21d2db748e19cd1035ab4e18950"
format_stamp: "Formatted at 2026-08-10 04:45:08 on dist-test-slave-bqcl"
server_key: "4a9270a5355209b93dbef0ad8fc0c441"
server_key_iv: "57c847d116a05daacaf71f0eeaae11a2"
server_key_version: "encryptionkey@0"
I20260810 04:45:08.374889 24344 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786336863388938-23197-0/minicluster-data/ts-0/wal/instance:
uuid: "bc3ff21d2db748e19cd1035ab4e18950"
format_stamp: "Formatted at 2026-08-10 04:45:08 on dist-test-slave-bqcl"
server_key: "4a9270a5355209b93dbef0ad8fc0c441"
server_key_iv: "57c847d116a05daacaf71f0eeaae11a2"
server_key_version: "encryptionkey@0"
I20260810 04:45:08.376082 24344 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260810 04:45:08.376763 24365 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 04:45:08.376931 24344 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260810 04:45:08.377013 24344 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786336863388938-23197-0/minicluster-data/ts-0/data,/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786336863388938-23197-0/minicluster-data/ts-0/wal
uuid: "bc3ff21d2db748e19cd1035ab4e18950"
format_stamp: "Formatted at 2026-08-10 04:45:08 on dist-test-slave-bqcl"
server_key: "4a9270a5355209b93dbef0ad8fc0c441"
server_key_iv: "57c847d116a05daacaf71f0eeaae11a2"
server_key_version: "encryptionkey@0"
I20260810 04:45:08.377070 24344 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786336863388938-23197-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786336863388938-23197-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786336863388938-23197-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 04:45:08.389918 24344 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 04:45:08.390242 24344 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 04:45:08.390372 24344 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 04:45:08.390578 24344 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 04:45:08.390884 24344 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 04:45:08.390918 24344 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 04:45:08.390945 24344 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 04:45:08.390964 24344 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 04:45:08.396580 24344 rpc_server.cc:307] RPC server started. Bound to: 127.22.167.65:38545
I20260810 04:45:08.396651 24559 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.167.65:38545 every 8 connection(s)
I20260810 04:45:08.396927 24344 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786336863388938-23197-0/minicluster-data/ts-0/data/info.pb
I20260810 04:45:08.401664 24334 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:45:08.397544 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.167.65:43271 (local address 127.22.167.126:46507)
0810 04:45:08.397616 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0810 04:45:08.397619 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 04:45:08.397838 (+ 219us) server_negotiation.cc:407] Connection header received
0810 04:45:08.398358 (+ 520us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:45:08.398359 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:45:08.398371 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:45:08.398416 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0810 04:45:08.399041 (+ 625us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:45:08.399540 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:45:08.400496 (+ 956us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:45:08.400712 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:45:08.400776 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:45:08.401483 (+ 707us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 04:45:08.401483 (+ 0us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 04:45:08.401484 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 04:45:08.401488 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0810 04:45:08.401498 (+ 10us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 04:45:08.401501 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 04:45:08.401533 (+ 32us) server_negotiation.cc:1035] Waiting for connection context
0810 04:45:08.401584 (+ 51us) server_negotiation.cc:299] Negotiation successful
0810 04:45:08.401605 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260810 04:45:08.401839 24563 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:45:08.397546 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.126:46507 (local address 127.22.167.65:43271)
0810 04:45:08.397732 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0810 04:45:08.397764 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0810 04:45:08.398280 (+ 516us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 04:45:08.398474 (+ 194us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 04:45:08.398477 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 04:45:08.398485 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0810 04:45:08.398952 (+ 467us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:45:08.398957 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:45:08.399640 (+ 683us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:45:08.399641 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:45:08.400426 (+ 785us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:45:08.400428 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:45:08.400785 (+ 357us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:45:08.400785 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:45:08.400895 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:45:08.401368 (+ 473us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 04:45:08.401368 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0810 04:45:08.401396 (+ 28us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 04:45:08.401419 (+ 23us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 04:45:08.401535 (+ 116us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 04:45:08.401537 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 04:45:08.401539 (+ 2us) client_negotiation.cc:769] Sending connection context
0810 04:45:08.401581 (+ 42us) client_negotiation.cc:240] Negotiation successful
0810 04:45:08.401659 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":93,"threads_started":1}
I20260810 04:45:08.402333 24561 heartbeater.cc:347] Connected to a master server at 127.22.167.126:46507
I20260810 04:45:08.402415 24561 heartbeater.cc:464] Registering TS with master...
I20260810 04:45:08.402599 24561 heartbeater.cc:511] Master 127.22.167.126:46507 requested a full tablet report, sending...
I20260810 04:45:08.403092 24252 ts_manager.cc:194] Registered new tserver with Master: bc3ff21d2db748e19cd1035ab4e18950 (127.22.167.65:38545)
I20260810 04:45:08.403689 24252 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.22.167.65:43271
I20260810 04:45:08.406306 23197 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskemv_AM/build/release/bin/kudu as pid 24344
I20260810 04:45:08.406389 23197 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786336863388938-23197-0/minicluster-data/ts-0/wal/instance
I20260810 04:45:08.406545 23197 external_mini_cluster.cc:1428] Setting key 60b85a8f1f7823931794da87a5eaee6b
I20260810 04:45:08.407299 23197 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskemv_AM/build/release/bin/kudu
/tmp/dist-test-taskemv_AM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786336863388938-23197-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786336863388938-23197-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786336863388938-23197-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786336863388938-23197-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.167.66:0
--local_ip_for_outbound_sockets=127.22.167.66
--webserver_interface=127.22.167.66
--webserver_port=0
--tserver_master_addrs=127.22.167.126:46507
--builtin_ntp_servers=127.22.167.84:35545
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260810 04:45:08.487259 24564 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 04:45:08.487454 24564 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 04:45:08.487483 24564 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 04:45:08.488555 24564 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 04:45:08.488593 24564 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 04:45:08.488626 24564 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.167.66
I20260810 04:45:08.489925 24564 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.167.84:35545
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786336863388938-23197-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786336863388938-23197-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.167.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786336863388938-23197-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.22.167.66
--webserver_port=0
--tserver_master_addrs=127.22.167.126:46507
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24564
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.167.66
--log_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786336863388938-23197-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 03:54:01 UTC on ccycloud.quasar-icijmt.root.comops.site
build id 4577009
I20260810 04:45:08.490172 24564 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 04:45:08.490442 24564 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 04:45:08.495217 24576 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 04:45:08.495311 24573 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 04:45:08.495361 24580 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 04:45:08.495383 24564 server_base.cc:1034] running on GCE node
I20260810 04:45:08.495651 24564 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 04:45:08.495867 24564 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 04:45:08.496999 24564 hybrid_clock.cc:648] HybridClock initialized: now 1786337108496986 us; error 30 us; skew 500 ppm
I20260810 04:45:08.498253 24564 webserver.cc:460] Webserver started at http://127.22.167.66:38703/ using document root <none> and password file <none>
I20260810 04:45:08.498489 24564 fs_manager.cc:359] Metadata directory not provided
I20260810 04:45:08.498534 24564 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 04:45:08.498677 24564 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 04:45:08.499753 24564 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786336863388938-23197-0/minicluster-data/ts-1/data/instance:
uuid: "f3256047df96413cba5af99e7ad958d5"
format_stamp: "Formatted at 2026-08-10 04:45:08 on dist-test-slave-bqcl"
server_key: "792ee024b1f99b4d0222c0d2b0d14c7f"
server_key_iv: "6167d0d7112b7009634ee18899ea264e"
server_key_version: "encryptionkey@0"
I20260810 04:45:08.500098 24564 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786336863388938-23197-0/minicluster-data/ts-1/wal/instance:
uuid: "f3256047df96413cba5af99e7ad958d5"
format_stamp: "Formatted at 2026-08-10 04:45:08 on dist-test-slave-bqcl"
server_key: "792ee024b1f99b4d0222c0d2b0d14c7f"
server_key_iv: "6167d0d7112b7009634ee18899ea264e"
server_key_version: "encryptionkey@0"
I20260810 04:45:08.501333 24564 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260810 04:45:08.501978 24588 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 04:45:08.502157 24564 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260810 04:45:08.502250 24564 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786336863388938-23197-0/minicluster-data/ts-1/data,/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786336863388938-23197-0/minicluster-data/ts-1/wal
uuid: "f3256047df96413cba5af99e7ad958d5"
format_stamp: "Formatted at 2026-08-10 04:45:08 on dist-test-slave-bqcl"
server_key: "792ee024b1f99b4d0222c0d2b0d14c7f"
server_key_iv: "6167d0d7112b7009634ee18899ea264e"
server_key_version: "encryptionkey@0"
I20260810 04:45:08.502314 24564 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786336863388938-23197-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786336863388938-23197-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786336863388938-23197-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 04:45:08.521235 24564 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 04:45:08.521483 24564 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 04:45:08.521600 24564 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 04:45:08.521808 24564 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 04:45:08.522102 24564 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 04:45:08.522218 24564 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 04:45:08.522258 24564 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 04:45:08.522284 24564 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 04:45:08.527516 24564 rpc_server.cc:307] RPC server started. Bound to: 127.22.167.66:32895
I20260810 04:45:08.527571 24772 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.167.66:32895 every 8 connection(s)
I20260810 04:45:08.527873 24564 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786336863388938-23197-0/minicluster-data/ts-1/data/info.pb
I20260810 04:45:08.532442 23197 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskemv_AM/build/release/bin/kudu as pid 24564
I20260810 04:45:08.532544 23197 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786336863388938-23197-0/minicluster-data/ts-1/wal/instance
I20260810 04:45:08.532536 24334 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:45:08.528478 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.167.66:48075 (local address 127.22.167.126:46507)
0810 04:45:08.528521 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0810 04:45:08.528524 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 04:45:08.528734 (+ 210us) server_negotiation.cc:407] Connection header received
0810 04:45:08.529226 (+ 492us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:45:08.529228 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:45:08.529240 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:45:08.529294 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0810 04:45:08.529926 (+ 632us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:45:08.530435 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:45:08.531351 (+ 916us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:45:08.531548 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:45:08.531602 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:45:08.532320 (+ 718us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 04:45:08.532321 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 04:45:08.532321 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 04:45:08.532325 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0810 04:45:08.532336 (+ 11us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 04:45:08.532339 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 04:45:08.532372 (+ 33us) server_negotiation.cc:1035] Waiting for connection context
0810 04:45:08.532444 (+ 72us) server_negotiation.cc:299] Negotiation successful
0810 04:45:08.532464 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260810 04:45:08.532753 23197 external_mini_cluster.cc:1428] Setting key 5304ca0e9bd3b1672808eaf89afb6655
I20260810 04:45:08.532784 24777 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:45:08.528469 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.126:46507 (local address 127.22.167.66:48075)
0810 04:45:08.528625 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0810 04:45:08.528664 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0810 04:45:08.529111 (+ 447us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 04:45:08.529352 (+ 241us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 04:45:08.529355 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 04:45:08.529362 (+ 7us) client_negotiation.cc:189] Negotiated authn=SASL
0810 04:45:08.529836 (+ 474us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:45:08.529841 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:45:08.530498 (+ 657us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:45:08.530498 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:45:08.531275 (+ 777us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:45:08.531278 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:45:08.531609 (+ 331us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:45:08.531609 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:45:08.531725 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:45:08.532195 (+ 470us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 04:45:08.532195 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0810 04:45:08.532223 (+ 28us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 04:45:08.532246 (+ 23us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 04:45:08.532377 (+ 131us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 04:45:08.532382 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 04:45:08.532385 (+ 3us) client_negotiation.cc:769] Sending connection context
0810 04:45:08.532448 (+ 63us) client_negotiation.cc:240] Negotiation successful
0810 04:45:08.532558 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":100,"thread_start_us":71,"threads_started":1}
I20260810 04:45:08.533190 24774 heartbeater.cc:347] Connected to a master server at 127.22.167.126:46507
I20260810 04:45:08.533280 24774 heartbeater.cc:464] Registering TS with master...
I20260810 04:45:08.533470 24774 heartbeater.cc:511] Master 127.22.167.126:46507 requested a full tablet report, sending...
I20260810 04:45:08.533694 23197 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskemv_AM/build/release/bin/kudu
/tmp/dist-test-taskemv_AM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786336863388938-23197-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786336863388938-23197-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786336863388938-23197-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786336863388938-23197-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.167.67:0
--local_ip_for_outbound_sockets=127.22.167.67
--webserver_interface=127.22.167.67
--webserver_port=0
--tserver_master_addrs=127.22.167.126:46507
--builtin_ntp_servers=127.22.167.84:35545
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260810 04:45:08.533885 24252 ts_manager.cc:194] Registered new tserver with Master: f3256047df96413cba5af99e7ad958d5 (127.22.167.66:32895)
I20260810 04:45:08.534340 24252 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.22.167.66:48075
W20260810 04:45:08.614281 24778 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 04:45:08.614471 24778 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 04:45:08.614491 24778 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 04:45:08.615551 24778 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 04:45:08.615581 24778 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 04:45:08.615604 24778 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.167.67
I20260810 04:45:08.616858 24778 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.167.84:35545
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786336863388938-23197-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786336863388938-23197-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.167.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786336863388938-23197-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.22.167.67
--webserver_port=0
--tserver_master_addrs=127.22.167.126:46507
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24778
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.167.67
--log_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786336863388938-23197-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 03:54:01 UTC on ccycloud.quasar-icijmt.root.comops.site
build id 4577009
I20260810 04:45:08.617030 24778 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 04:45:08.617260 24778 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 04:45:08.621785 24792 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 04:45:08.621809 24786 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 04:45:08.621830 24789 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 04:45:08.622187 24778 server_base.cc:1034] running on GCE node
I20260810 04:45:08.622378 24778 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 04:45:08.622575 24778 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 04:45:08.623708 24778 hybrid_clock.cc:648] HybridClock initialized: now 1786337108623696 us; error 28 us; skew 500 ppm
I20260810 04:45:08.624859 24778 webserver.cc:460] Webserver started at http://127.22.167.67:43401/ using document root <none> and password file <none>
I20260810 04:45:08.625072 24778 fs_manager.cc:359] Metadata directory not provided
I20260810 04:45:08.625119 24778 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 04:45:08.625258 24778 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 04:45:08.626310 24778 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786336863388938-23197-0/minicluster-data/ts-2/data/instance:
uuid: "bc39d83190e1457bb87290ab89cb14d6"
format_stamp: "Formatted at 2026-08-10 04:45:08 on dist-test-slave-bqcl"
server_key: "86af2e53c515f1ee5a938f09349e4577"
server_key_iv: "2731c07687d590394c68dba9c1b60f02"
server_key_version: "encryptionkey@0"
I20260810 04:45:08.626639 24778 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786336863388938-23197-0/minicluster-data/ts-2/wal/instance:
uuid: "bc39d83190e1457bb87290ab89cb14d6"
format_stamp: "Formatted at 2026-08-10 04:45:08 on dist-test-slave-bqcl"
server_key: "86af2e53c515f1ee5a938f09349e4577"
server_key_iv: "2731c07687d590394c68dba9c1b60f02"
server_key_version: "encryptionkey@0"
I20260810 04:45:08.627804 24778 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260810 04:45:08.628435 24800 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 04:45:08.628588 24778 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260810 04:45:08.628669 24778 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786336863388938-23197-0/minicluster-data/ts-2/data,/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786336863388938-23197-0/minicluster-data/ts-2/wal
uuid: "bc39d83190e1457bb87290ab89cb14d6"
format_stamp: "Formatted at 2026-08-10 04:45:08 on dist-test-slave-bqcl"
server_key: "86af2e53c515f1ee5a938f09349e4577"
server_key_iv: "2731c07687d590394c68dba9c1b60f02"
server_key_version: "encryptionkey@0"
I20260810 04:45:08.628728 24778 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786336863388938-23197-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786336863388938-23197-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786336863388938-23197-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 04:45:08.635159 24778 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 04:45:08.635385 24778 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 04:45:08.635497 24778 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 04:45:08.635681 24778 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 04:45:08.635962 24778 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 04:45:08.635995 24778 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 04:45:08.636030 24778 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 04:45:08.636055 24778 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 04:45:08.641423 24778 rpc_server.cc:307] RPC server started. Bound to: 127.22.167.67:41433
I20260810 04:45:08.641475 24985 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.167.67:41433 every 8 connection(s)
I20260810 04:45:08.641755 24778 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786336863388938-23197-0/minicluster-data/ts-2/data/info.pb
I20260810 04:45:08.646587 24334 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:45:08.642402 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.167.67:54759 (local address 127.22.167.126:46507)
0810 04:45:08.642454 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0810 04:45:08.642456 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 04:45:08.642654 (+ 198us) server_negotiation.cc:407] Connection header received
0810 04:45:08.643132 (+ 478us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:45:08.643134 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:45:08.643146 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:45:08.643208 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0810 04:45:08.643848 (+ 640us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:45:08.644346 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:45:08.645297 (+ 951us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:45:08.645499 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:45:08.645554 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:45:08.646322 (+ 768us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 04:45:08.646323 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 04:45:08.646323 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 04:45:08.646333 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0810 04:45:08.646349 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 04:45:08.646354 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 04:45:08.646399 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0810 04:45:08.646476 (+ 77us) server_negotiation.cc:299] Negotiation successful
0810 04:45:08.646507 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260810 04:45:08.646780 24988 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:45:08.642364 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.126:46507 (local address 127.22.167.67:54759)
0810 04:45:08.642552 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0810 04:45:08.642583 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0810 04:45:08.643039 (+ 456us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 04:45:08.643274 (+ 235us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 04:45:08.643278 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 04:45:08.643288 (+ 10us) client_negotiation.cc:189] Negotiated authn=SASL
0810 04:45:08.643762 (+ 474us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:45:08.643767 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:45:08.644426 (+ 659us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:45:08.644427 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:45:08.645203 (+ 776us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:45:08.645205 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:45:08.645568 (+ 363us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:45:08.645568 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:45:08.645703 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:45:08.646191 (+ 488us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 04:45:08.646191 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0810 04:45:08.646218 (+ 27us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 04:45:08.646241 (+ 23us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 04:45:08.646405 (+ 164us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 04:45:08.646411 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 04:45:08.646414 (+ 3us) client_negotiation.cc:769] Sending connection context
0810 04:45:08.646475 (+ 61us) client_negotiation.cc:240] Negotiation successful
0810 04:45:08.646575 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":86,"threads_started":1}
I20260810 04:45:08.647264 24986 heartbeater.cc:347] Connected to a master server at 127.22.167.126:46507
I20260810 04:45:08.647357 24986 heartbeater.cc:464] Registering TS with master...
I20260810 04:45:08.647531 24986 heartbeater.cc:511] Master 127.22.167.126:46507 requested a full tablet report, sending...
I20260810 04:45:08.647876 24252 ts_manager.cc:194] Registered new tserver with Master: bc39d83190e1457bb87290ab89cb14d6 (127.22.167.67:41433)
I20260810 04:45:08.648247 24252 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.22.167.67:54759
I20260810 04:45:08.648952 23197 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskemv_AM/build/release/bin/kudu as pid 24778
I20260810 04:45:08.649039 23197 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786336863388938-23197-0/minicluster-data/ts-2/wal/instance
I20260810 04:45:08.649219 23197 external_mini_cluster.cc:1428] Setting key ac850479ef3fdbc470b9a5231eb46f5d
I20260810 04:45:08.650004 23197 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260810 04:45:08.653587 24334 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:45:08.651290 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49952 (local address 127.22.167.126:46507)
0810 04:45:08.651327 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0810 04:45:08.651329 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 04:45:08.651428 (+ 99us) server_negotiation.cc:407] Connection header received
0810 04:45:08.651483 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:45:08.651483 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:45:08.651494 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:45:08.651528 (+ 34us) server_negotiation.cc:226] Negotiated authn=SASL
0810 04:45:08.651643 (+ 115us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:45:08.652072 (+ 429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:45:08.652760 (+ 688us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:45:08.652922 (+ 162us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:45:08.652963 (+ 41us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:45:08.653432 (+ 469us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 04:45:08.653433 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 04:45:08.653433 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 04:45:08.653437 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0810 04:45:08.653448 (+ 11us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 04:45:08.653452 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 04:45:08.653487 (+ 35us) server_negotiation.cc:1035] Waiting for connection context
0810 04:45:08.653517 (+ 30us) server_negotiation.cc:299] Negotiation successful
0810 04:45:08.653536 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260810 04:45:08.654970 24252 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:49952:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 04:45:08.656401 24252 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 04:45:08.662206 25005 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:45:08.658494 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54352 (local address 127.22.167.67:41433)
0810 04:45:08.658697 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0810 04:45:08.658702 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 04:45:08.658905 (+ 203us) server_negotiation.cc:407] Connection header received
0810 04:45:08.658983 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:45:08.658985 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:45:08.659000 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:45:08.659076 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 04:45:08.659246 (+ 170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:45:08.660299 (+ 1053us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:45:08.661200 (+ 901us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:45:08.661794 (+ 594us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:45:08.661867 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:45:08.661946 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0810 04:45:08.662069 (+ 123us) server_negotiation.cc:299] Negotiation successful
0810 04:45:08.662107 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":175,"thread_start_us":117,"threads_started":1}
I20260810 04:45:08.662209 24999 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:45:08.658353 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.67:41433 (local address 127.0.0.1:54352)
0810 04:45:08.658848 (+ 495us) negotiation.cc:107] Waiting for socket to connect
0810 04:45:08.658859 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0810 04:45:08.658911 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 04:45:08.659136 (+ 225us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 04:45:08.659137 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 04:45:08.659139 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 04:45:08.659171 (+ 32us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:45:08.659172 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:45:08.660386 (+ 1214us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:45:08.660387 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:45:08.661122 (+ 735us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:45:08.661124 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:45:08.661878 (+ 754us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:45:08.661878 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:45:08.661960 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:45:08.661973 (+ 13us) client_negotiation.cc:769] Sending connection context
0810 04:45:08.662042 (+ 69us) client_negotiation.cc:240] Negotiation successful
0810 04:45:08.662087 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":479,"thread_start_us":81,"threads_started":1}
I20260810 04:45:08.662631 25003 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:45:08.658511 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.66:32895 (local address 127.0.0.1:44568)
0810 04:45:08.659018 (+ 507us) negotiation.cc:107] Waiting for socket to connect
0810 04:45:08.659027 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0810 04:45:08.659075 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 04:45:08.659331 (+ 256us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 04:45:08.659332 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 04:45:08.659333 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 04:45:08.659377 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:45:08.659378 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:45:08.660193 (+ 815us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:45:08.660194 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:45:08.661370 (+ 1176us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:45:08.661373 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:45:08.662384 (+ 1011us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:45:08.662385 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:45:08.662451 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:45:08.662454 (+ 3us) client_negotiation.cc:769] Sending connection context
0810 04:45:08.662498 (+ 44us) client_negotiation.cc:240] Negotiation successful
0810 04:45:08.662539 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":491,"mutex_wait_us":1,"thread_start_us":96,"threads_started":1}
I20260810 04:45:08.662717 25002 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:45:08.658429 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53410 (local address 127.22.167.65:38545)
0810 04:45:08.658760 (+ 331us) server_negotiation.cc:206] Beginning negotiation
0810 04:45:08.658765 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 04:45:08.658776 (+ 11us) server_negotiation.cc:407] Connection header received
0810 04:45:08.658818 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:45:08.658820 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:45:08.658834 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:45:08.658923 (+ 89us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 04:45:08.659125 (+ 202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:45:08.660269 (+ 1144us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:45:08.661680 (+ 1411us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:45:08.662399 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:45:08.662458 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:45:08.662540 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0810 04:45:08.662592 (+ 52us) server_negotiation.cc:299] Negotiation successful
0810 04:45:08.662629 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":291,"thread_start_us":100,"threads_started":1}
I20260810 04:45:08.662719 25001 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:45:08.658322 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.65:38545 (local address 127.0.0.1:53410)
0810 04:45:08.658487 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0810 04:45:08.658528 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0810 04:45:08.658591 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 04:45:08.658936 (+ 345us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 04:45:08.658939 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 04:45:08.658941 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 04:45:08.659026 (+ 85us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:45:08.659029 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:45:08.660353 (+ 1324us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:45:08.660354 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:45:08.661603 (+ 1249us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:45:08.661605 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:45:08.662460 (+ 855us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:45:08.662461 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:45:08.662523 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:45:08.662526 (+ 3us) client_negotiation.cc:769] Sending connection context
0810 04:45:08.662575 (+ 49us) client_negotiation.cc:240] Negotiation successful
0810 04:45:08.662613 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":133,"mutex_wait_us":7,"thread_start_us":55,"threads_started":1}
I20260810 04:45:08.663064 25007 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:45:08.658690 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44568 (local address 127.22.167.66:32895)
0810 04:45:08.658859 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0810 04:45:08.658862 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 04:45:08.659129 (+ 267us) server_negotiation.cc:407] Connection header received
0810 04:45:08.659175 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:45:08.659177 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:45:08.659193 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:45:08.659261 (+ 68us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 04:45:08.659446 (+ 185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:45:08.660119 (+ 673us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:45:08.661447 (+ 1328us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:45:08.662187 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:45:08.662253 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:45:08.662349 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0810 04:45:08.662938 (+ 589us) server_negotiation.cc:299] Negotiation successful
0810 04:45:08.662984 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":135,"thread_start_us":93,"threads_started":1}
I20260810 04:45:08.663093 24876 tablet_service.cc:1467] Processing CreateTablet for tablet 0638bc19aaad4597b0045fc9b586c6ff (DEFAULT_TABLE table=test-table [id=9c98091788f94e30aa51808402a15cc3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 04:45:08.663470 24876 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0638bc19aaad4597b0045fc9b586c6ff. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 04:45:08.663542 24443 tablet_service.cc:1467] Processing CreateTablet for tablet 0638bc19aaad4597b0045fc9b586c6ff (DEFAULT_TABLE table=test-table [id=9c98091788f94e30aa51808402a15cc3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 04:45:08.663717 24664 tablet_service.cc:1467] Processing CreateTablet for tablet 0638bc19aaad4597b0045fc9b586c6ff (DEFAULT_TABLE table=test-table [id=9c98091788f94e30aa51808402a15cc3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 04:45:08.663908 24443 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0638bc19aaad4597b0045fc9b586c6ff. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 04:45:08.664034 24664 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0638bc19aaad4597b0045fc9b586c6ff. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 04:45:08.666038 25008 tablet_bootstrap.cc:492] T 0638bc19aaad4597b0045fc9b586c6ff P bc39d83190e1457bb87290ab89cb14d6: Bootstrap starting.
I20260810 04:45:08.666673 25008 tablet_bootstrap.cc:654] T 0638bc19aaad4597b0045fc9b586c6ff P bc39d83190e1457bb87290ab89cb14d6: Neither blocks nor log segments found. Creating new log.
I20260810 04:45:08.666944 25008 log.cc:824] T 0638bc19aaad4597b0045fc9b586c6ff P bc39d83190e1457bb87290ab89cb14d6: Log is configured to *not* fsync() on all Append() calls
I20260810 04:45:08.667351 25010 tablet_bootstrap.cc:492] T 0638bc19aaad4597b0045fc9b586c6ff P bc3ff21d2db748e19cd1035ab4e18950: Bootstrap starting.
I20260810 04:45:08.667517 25009 tablet_bootstrap.cc:492] T 0638bc19aaad4597b0045fc9b586c6ff P f3256047df96413cba5af99e7ad958d5: Bootstrap starting.
I20260810 04:45:08.667636 25008 tablet_bootstrap.cc:492] T 0638bc19aaad4597b0045fc9b586c6ff P bc39d83190e1457bb87290ab89cb14d6: No bootstrap required, opened a new log
I20260810 04:45:08.667704 25008 ts_tablet_manager.cc:1397] T 0638bc19aaad4597b0045fc9b586c6ff P bc39d83190e1457bb87290ab89cb14d6: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260810 04:45:08.668087 25010 tablet_bootstrap.cc:654] T 0638bc19aaad4597b0045fc9b586c6ff P bc3ff21d2db748e19cd1035ab4e18950: Neither blocks nor log segments found. Creating new log.
I20260810 04:45:08.668087 25009 tablet_bootstrap.cc:654] T 0638bc19aaad4597b0045fc9b586c6ff P f3256047df96413cba5af99e7ad958d5: Neither blocks nor log segments found. Creating new log.
I20260810 04:45:08.668416 25009 log.cc:824] T 0638bc19aaad4597b0045fc9b586c6ff P f3256047df96413cba5af99e7ad958d5: Log is configured to *not* fsync() on all Append() calls
I20260810 04:45:08.668416 25010 log.cc:824] T 0638bc19aaad4597b0045fc9b586c6ff P bc3ff21d2db748e19cd1035ab4e18950: Log is configured to *not* fsync() on all Append() calls
I20260810 04:45:08.669121 25010 tablet_bootstrap.cc:492] T 0638bc19aaad4597b0045fc9b586c6ff P bc3ff21d2db748e19cd1035ab4e18950: No bootstrap required, opened a new log
I20260810 04:45:08.669204 25010 ts_tablet_manager.cc:1397] T 0638bc19aaad4597b0045fc9b586c6ff P bc3ff21d2db748e19cd1035ab4e18950: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260810 04:45:08.669409 25009 tablet_bootstrap.cc:492] T 0638bc19aaad4597b0045fc9b586c6ff P f3256047df96413cba5af99e7ad958d5: No bootstrap required, opened a new log
I20260810 04:45:08.669476 25009 ts_tablet_manager.cc:1397] T 0638bc19aaad4597b0045fc9b586c6ff P f3256047df96413cba5af99e7ad958d5: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260810 04:45:08.669813 25008 raft_consensus.cc:348] T 0638bc19aaad4597b0045fc9b586c6ff P bc39d83190e1457bb87290ab89cb14d6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc3ff21d2db748e19cd1035ab4e18950" member_type: VOTER last_known_addr { host: "127.22.167.65" port: 38545 } } peers { permanent_uuid: "bc39d83190e1457bb87290ab89cb14d6" member_type: VOTER last_known_addr { host: "127.22.167.67" port: 41433 } } peers { permanent_uuid: "f3256047df96413cba5af99e7ad958d5" member_type: VOTER last_known_addr { host: "127.22.167.66" port: 32895 } }
I20260810 04:45:08.669948 25008 raft_consensus.cc:374] T 0638bc19aaad4597b0045fc9b586c6ff P bc39d83190e1457bb87290ab89cb14d6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 04:45:08.669981 25008 raft_consensus.cc:729] T 0638bc19aaad4597b0045fc9b586c6ff P bc39d83190e1457bb87290ab89cb14d6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bc39d83190e1457bb87290ab89cb14d6, State: Initialized, Role: FOLLOWER
I20260810 04:45:08.670152 25008 consensus_queue.cc:260] T 0638bc19aaad4597b0045fc9b586c6ff P bc39d83190e1457bb87290ab89cb14d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc3ff21d2db748e19cd1035ab4e18950" member_type: VOTER last_known_addr { host: "127.22.167.65" port: 38545 } } peers { permanent_uuid: "bc39d83190e1457bb87290ab89cb14d6" member_type: VOTER last_known_addr { host: "127.22.167.67" port: 41433 } } peers { permanent_uuid: "f3256047df96413cba5af99e7ad958d5" member_type: VOTER last_known_addr { host: "127.22.167.66" port: 32895 } }
I20260810 04:45:08.670452 25008 ts_tablet_manager.cc:1428] T 0638bc19aaad4597b0045fc9b586c6ff P bc39d83190e1457bb87290ab89cb14d6: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260810 04:45:08.670575 24986 heartbeater.cc:503] Master 127.22.167.126:46507 was elected leader, sending a full tablet report...
I20260810 04:45:08.671636 25010 raft_consensus.cc:348] T 0638bc19aaad4597b0045fc9b586c6ff P bc3ff21d2db748e19cd1035ab4e18950 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc3ff21d2db748e19cd1035ab4e18950" member_type: VOTER last_known_addr { host: "127.22.167.65" port: 38545 } } peers { permanent_uuid: "bc39d83190e1457bb87290ab89cb14d6" member_type: VOTER last_known_addr { host: "127.22.167.67" port: 41433 } } peers { permanent_uuid: "f3256047df96413cba5af99e7ad958d5" member_type: VOTER last_known_addr { host: "127.22.167.66" port: 32895 } }
I20260810 04:45:08.671803 25010 raft_consensus.cc:374] T 0638bc19aaad4597b0045fc9b586c6ff P bc3ff21d2db748e19cd1035ab4e18950 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 04:45:08.671847 25010 raft_consensus.cc:729] T 0638bc19aaad4597b0045fc9b586c6ff P bc3ff21d2db748e19cd1035ab4e18950 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bc3ff21d2db748e19cd1035ab4e18950, State: Initialized, Role: FOLLOWER
I20260810 04:45:08.671957 25010 consensus_queue.cc:260] T 0638bc19aaad4597b0045fc9b586c6ff P bc3ff21d2db748e19cd1035ab4e18950 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc3ff21d2db748e19cd1035ab4e18950" member_type: VOTER last_known_addr { host: "127.22.167.65" port: 38545 } } peers { permanent_uuid: "bc39d83190e1457bb87290ab89cb14d6" member_type: VOTER last_known_addr { host: "127.22.167.67" port: 41433 } } peers { permanent_uuid: "f3256047df96413cba5af99e7ad958d5" member_type: VOTER last_known_addr { host: "127.22.167.66" port: 32895 } }
I20260810 04:45:08.672060 25009 raft_consensus.cc:348] T 0638bc19aaad4597b0045fc9b586c6ff P f3256047df96413cba5af99e7ad958d5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc3ff21d2db748e19cd1035ab4e18950" member_type: VOTER last_known_addr { host: "127.22.167.65" port: 38545 } } peers { permanent_uuid: "bc39d83190e1457bb87290ab89cb14d6" member_type: VOTER last_known_addr { host: "127.22.167.67" port: 41433 } } peers { permanent_uuid: "f3256047df96413cba5af99e7ad958d5" member_type: VOTER last_known_addr { host: "127.22.167.66" port: 32895 } }
I20260810 04:45:08.672210 25009 raft_consensus.cc:374] T 0638bc19aaad4597b0045fc9b586c6ff P f3256047df96413cba5af99e7ad958d5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 04:45:08.672222 25010 ts_tablet_manager.cc:1428] T 0638bc19aaad4597b0045fc9b586c6ff P bc3ff21d2db748e19cd1035ab4e18950: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260810 04:45:08.672247 25009 raft_consensus.cc:729] T 0638bc19aaad4597b0045fc9b586c6ff P f3256047df96413cba5af99e7ad958d5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f3256047df96413cba5af99e7ad958d5, State: Initialized, Role: FOLLOWER
I20260810 04:45:08.672250 24561 heartbeater.cc:503] Master 127.22.167.126:46507 was elected leader, sending a full tablet report...
I20260810 04:45:08.672374 25009 consensus_queue.cc:260] T 0638bc19aaad4597b0045fc9b586c6ff P f3256047df96413cba5af99e7ad958d5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc3ff21d2db748e19cd1035ab4e18950" member_type: VOTER last_known_addr { host: "127.22.167.65" port: 38545 } } peers { permanent_uuid: "bc39d83190e1457bb87290ab89cb14d6" member_type: VOTER last_known_addr { host: "127.22.167.67" port: 41433 } } peers { permanent_uuid: "f3256047df96413cba5af99e7ad958d5" member_type: VOTER last_known_addr { host: "127.22.167.66" port: 32895 } }
I20260810 04:45:08.672631 25009 ts_tablet_manager.cc:1428] T 0638bc19aaad4597b0045fc9b586c6ff P f3256047df96413cba5af99e7ad958d5: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260810 04:45:08.672654 24774 heartbeater.cc:503] Master 127.22.167.126:46507 was elected leader, sending a full tablet report...
W20260810 04:45:08.778378 24775 tablet.cc:2369] T 0638bc19aaad4597b0045fc9b586c6ff P f3256047df96413cba5af99e7ad958d5: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260810 04:45:08.892592 24987 tablet.cc:2369] T 0638bc19aaad4597b0045fc9b586c6ff P bc39d83190e1457bb87290ab89cb14d6: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260810 04:45:08.897547 24562 tablet.cc:2369] T 0638bc19aaad4597b0045fc9b586c6ff P bc3ff21d2db748e19cd1035ab4e18950: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 04:45:09.057305 25017 raft_consensus.cc:482] T 0638bc19aaad4597b0045fc9b586c6ff P bc39d83190e1457bb87290ab89cb14d6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 04:45:09.057444 25017 raft_consensus.cc:504] T 0638bc19aaad4597b0045fc9b586c6ff P bc39d83190e1457bb87290ab89cb14d6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc3ff21d2db748e19cd1035ab4e18950" member_type: VOTER last_known_addr { host: "127.22.167.65" port: 38545 } } peers { permanent_uuid: "bc39d83190e1457bb87290ab89cb14d6" member_type: VOTER last_known_addr { host: "127.22.167.67" port: 41433 } } peers { permanent_uuid: "f3256047df96413cba5af99e7ad958d5" member_type: VOTER last_known_addr { host: "127.22.167.66" port: 32895 } }
I20260810 04:45:09.057801 25017 leader_election.cc:288] T 0638bc19aaad4597b0045fc9b586c6ff P bc39d83190e1457bb87290ab89cb14d6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bc3ff21d2db748e19cd1035ab4e18950 (127.22.167.65:38545), f3256047df96413cba5af99e7ad958d5 (127.22.167.66:32895)
I20260810 04:45:09.060587 25002 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:45:09.058089 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.167.67:58693 (local address 127.22.167.65:38545)
0810 04:45:09.058174 (+ 85us) server_negotiation.cc:206] Beginning negotiation
0810 04:45:09.058176 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 04:45:09.058190 (+ 14us) server_negotiation.cc:407] Connection header received
0810 04:45:09.058269 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:45:09.058270 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:45:09.058283 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:45:09.058357 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 04:45:09.058522 (+ 165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:45:09.058995 (+ 473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:45:09.059911 (+ 916us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:45:09.060271 (+ 360us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:45:09.060332 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:45:09.060341 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0810 04:45:09.060479 (+ 138us) server_negotiation.cc:299] Negotiation successful
0810 04:45:09.060520 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260810 04:45:09.060587 24988 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:45:09.058016 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.65:38545 (local address 127.22.167.67:58693)
0810 04:45:09.058079 (+ 63us) negotiation.cc:107] Waiting for socket to connect
0810 04:45:09.058092 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0810 04:45:09.058176 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 04:45:09.058379 (+ 203us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 04:45:09.058380 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 04:45:09.058383 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 04:45:09.058454 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:45:09.058456 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:45:09.059071 (+ 615us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:45:09.059072 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:45:09.059820 (+ 748us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:45:09.059822 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:45:09.060346 (+ 524us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:45:09.060347 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:45:09.060415 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:45:09.060418 (+ 3us) client_negotiation.cc:769] Sending connection context
0810 04:45:09.060465 (+ 47us) client_negotiation.cc:240] Negotiation successful
0810 04:45:09.060507 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":38}
I20260810 04:45:09.060916 25007 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:45:09.058258 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.167.67:39657 (local address 127.22.167.66:32895)
0810 04:45:09.058312 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0810 04:45:09.058314 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 04:45:09.058432 (+ 118us) server_negotiation.cc:407] Connection header received
0810 04:45:09.058518 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:45:09.058519 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:45:09.058528 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:45:09.058582 (+ 54us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 04:45:09.058738 (+ 156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:45:09.059241 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:45:09.060171 (+ 930us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:45:09.060528 (+ 357us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:45:09.060598 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:45:09.060606 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0810 04:45:09.060794 (+ 188us) server_negotiation.cc:299] Negotiation successful
0810 04:45:09.060840 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260810 04:45:09.060971 24474 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0638bc19aaad4597b0045fc9b586c6ff" candidate_uuid: "bc39d83190e1457bb87290ab89cb14d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bc3ff21d2db748e19cd1035ab4e18950" is_pre_election: true
I20260810 04:45:09.061151 24474 raft_consensus.cc:2454] T 0638bc19aaad4597b0045fc9b586c6ff P bc3ff21d2db748e19cd1035ab4e18950 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc39d83190e1457bb87290ab89cb14d6 in term 0.
I20260810 04:45:09.061167 25021 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:45:09.058180 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.66:32895 (local address 127.22.167.67:39657)
0810 04:45:09.058366 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0810 04:45:09.058375 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0810 04:45:09.058428 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 04:45:09.058592 (+ 164us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 04:45:09.058594 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 04:45:09.058596 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 04:45:09.058673 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:45:09.058675 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:45:09.059324 (+ 649us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:45:09.059326 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:45:09.060079 (+ 753us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:45:09.060081 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:45:09.060617 (+ 536us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:45:09.060618 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:45:09.060687 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:45:09.060692 (+ 5us) client_negotiation.cc:769] Sending connection context
0810 04:45:09.061036 (+ 344us) client_negotiation.cc:240] Negotiation successful
0810 04:45:09.061083 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":169,"thread_start_us":106,"threads_started":1}
I20260810 04:45:09.061405 24802 leader_election.cc:302] T 0638bc19aaad4597b0045fc9b586c6ff P bc39d83190e1457bb87290ab89cb14d6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bc39d83190e1457bb87290ab89cb14d6, bc3ff21d2db748e19cd1035ab4e18950; no voters:
I20260810 04:45:09.061492 24698 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0638bc19aaad4597b0045fc9b586c6ff" candidate_uuid: "bc39d83190e1457bb87290ab89cb14d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f3256047df96413cba5af99e7ad958d5" is_pre_election: true
I20260810 04:45:09.061583 25017 raft_consensus.cc:2781] T 0638bc19aaad4597b0045fc9b586c6ff P bc39d83190e1457bb87290ab89cb14d6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 04:45:09.061657 25017 raft_consensus.cc:482] T 0638bc19aaad4597b0045fc9b586c6ff P bc39d83190e1457bb87290ab89cb14d6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 04:45:09.061663 24698 raft_consensus.cc:2454] T 0638bc19aaad4597b0045fc9b586c6ff P f3256047df96413cba5af99e7ad958d5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc39d83190e1457bb87290ab89cb14d6 in term 0.
I20260810 04:45:09.061692 25017 raft_consensus.cc:3037] T 0638bc19aaad4597b0045fc9b586c6ff P bc39d83190e1457bb87290ab89cb14d6 [term 0 FOLLOWER]: Advancing to term 1
I20260810 04:45:09.062490 25017 raft_consensus.cc:504] T 0638bc19aaad4597b0045fc9b586c6ff P bc39d83190e1457bb87290ab89cb14d6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc3ff21d2db748e19cd1035ab4e18950" member_type: VOTER last_known_addr { host: "127.22.167.65" port: 38545 } } peers { permanent_uuid: "bc39d83190e1457bb87290ab89cb14d6" member_type: VOTER last_known_addr { host: "127.22.167.67" port: 41433 } } peers { permanent_uuid: "f3256047df96413cba5af99e7ad958d5" member_type: VOTER last_known_addr { host: "127.22.167.66" port: 32895 } }
I20260810 04:45:09.062615 25017 leader_election.cc:288] T 0638bc19aaad4597b0045fc9b586c6ff P bc39d83190e1457bb87290ab89cb14d6 [CANDIDATE]: Term 1 election: Requested vote from peers bc3ff21d2db748e19cd1035ab4e18950 (127.22.167.65:38545), f3256047df96413cba5af99e7ad958d5 (127.22.167.66:32895)
I20260810 04:45:09.062770 24474 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0638bc19aaad4597b0045fc9b586c6ff" candidate_uuid: "bc39d83190e1457bb87290ab89cb14d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bc3ff21d2db748e19cd1035ab4e18950"
I20260810 04:45:09.062777 24698 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0638bc19aaad4597b0045fc9b586c6ff" candidate_uuid: "bc39d83190e1457bb87290ab89cb14d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f3256047df96413cba5af99e7ad958d5"
I20260810 04:45:09.062847 24698 raft_consensus.cc:3037] T 0638bc19aaad4597b0045fc9b586c6ff P f3256047df96413cba5af99e7ad958d5 [term 0 FOLLOWER]: Advancing to term 1
I20260810 04:45:09.062855 24474 raft_consensus.cc:3037] T 0638bc19aaad4597b0045fc9b586c6ff P bc3ff21d2db748e19cd1035ab4e18950 [term 0 FOLLOWER]: Advancing to term 1
I20260810 04:45:09.063560 24698 raft_consensus.cc:2454] T 0638bc19aaad4597b0045fc9b586c6ff P f3256047df96413cba5af99e7ad958d5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bc39d83190e1457bb87290ab89cb14d6 in term 1.
I20260810 04:45:09.063601 24474 raft_consensus.cc:2454] T 0638bc19aaad4597b0045fc9b586c6ff P bc3ff21d2db748e19cd1035ab4e18950 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bc39d83190e1457bb87290ab89cb14d6 in term 1.
I20260810 04:45:09.063746 24802 leader_election.cc:302] T 0638bc19aaad4597b0045fc9b586c6ff P bc39d83190e1457bb87290ab89cb14d6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bc39d83190e1457bb87290ab89cb14d6, bc3ff21d2db748e19cd1035ab4e18950; no voters:
I20260810 04:45:09.063846 25017 raft_consensus.cc:2781] T 0638bc19aaad4597b0045fc9b586c6ff P bc39d83190e1457bb87290ab89cb14d6 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 04:45:09.063993 25017 raft_consensus.cc:686] T 0638bc19aaad4597b0045fc9b586c6ff P bc39d83190e1457bb87290ab89cb14d6 [term 1 LEADER]: Becoming Leader. State: Replica: bc39d83190e1457bb87290ab89cb14d6, State: Running, Role: LEADER
I20260810 04:45:09.064082 25017 consensus_queue.cc:237] T 0638bc19aaad4597b0045fc9b586c6ff P bc39d83190e1457bb87290ab89cb14d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc3ff21d2db748e19cd1035ab4e18950" member_type: VOTER last_known_addr { host: "127.22.167.65" port: 38545 } } peers { permanent_uuid: "bc39d83190e1457bb87290ab89cb14d6" member_type: VOTER last_known_addr { host: "127.22.167.67" port: 41433 } } peers { permanent_uuid: "f3256047df96413cba5af99e7ad958d5" member_type: VOTER last_known_addr { host: "127.22.167.66" port: 32895 } }
I20260810 04:45:09.064927 24251 catalog_manager.cc:5495] T 0638bc19aaad4597b0045fc9b586c6ff P bc39d83190e1457bb87290ab89cb14d6 reported cstate change: term changed from 0 to 1, leader changed from <none> to bc39d83190e1457bb87290ab89cb14d6 (127.22.167.67). New cstate: current_term: 1 leader_uuid: "bc39d83190e1457bb87290ab89cb14d6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc3ff21d2db748e19cd1035ab4e18950" member_type: VOTER last_known_addr { host: "127.22.167.65" port: 38545 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bc39d83190e1457bb87290ab89cb14d6" member_type: VOTER last_known_addr { host: "127.22.167.67" port: 41433 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f3256047df96413cba5af99e7ad958d5" member_type: VOTER last_known_addr { host: "127.22.167.66" port: 32895 } health_report { overall_health: UNKNOWN } } }
I20260810 04:45:09.104286 25005 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:45:09.101976 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54358 (local address 127.22.167.67:41433)
0810 04:45:09.102020 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0810 04:45:09.102022 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 04:45:09.102032 (+ 10us) server_negotiation.cc:407] Connection header received
0810 04:45:09.102154 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:45:09.102155 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:45:09.102178 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:45:09.102237 (+ 59us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 04:45:09.102333 (+ 96us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:45:09.102798 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:45:09.103484 (+ 686us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:45:09.103710 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:45:09.103770 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:45:09.103943 (+ 173us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 04:45:09.104056 (+ 113us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 04:45:09.104129 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0810 04:45:09.104156 (+ 27us) server_negotiation.cc:299] Negotiation successful
0810 04:45:09.104174 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260810 04:45:09.106222 24474 raft_consensus.cc:1261] T 0638bc19aaad4597b0045fc9b586c6ff P bc3ff21d2db748e19cd1035ab4e18950 [term 1 FOLLOWER]: Refusing update from remote peer bc39d83190e1457bb87290ab89cb14d6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 04:45:09.106222 24698 raft_consensus.cc:1261] T 0638bc19aaad4597b0045fc9b586c6ff P f3256047df96413cba5af99e7ad958d5 [term 1 FOLLOWER]: Refusing update from remote peer bc39d83190e1457bb87290ab89cb14d6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 04:45:09.106583 25017 consensus_queue.cc:1035] T 0638bc19aaad4597b0045fc9b586c6ff P bc39d83190e1457bb87290ab89cb14d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc3ff21d2db748e19cd1035ab4e18950" member_type: VOTER last_known_addr { host: "127.22.167.65" port: 38545 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 04:45:09.106745 25023 consensus_queue.cc:1035] T 0638bc19aaad4597b0045fc9b586c6ff P bc39d83190e1457bb87290ab89cb14d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3256047df96413cba5af99e7ad958d5" member_type: VOTER last_known_addr { host: "127.22.167.66" port: 32895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 04:45:09.109519 25030 mvcc.cc:205] Tried to move back new op lower bound from 7316836798896087040 to 7316836798727016448. Current Snapshot: MvccSnapshot[applied={T|T < 7316836798896087040}]
I20260810 04:45:09.111481 24252 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:49952:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 04:45:09.111584 24252 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:49952:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 04:45:09.112349 24252 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 69ad14ffadd1405ca18b013b786b181d: Sending DeleteTablet for 3 replicas of tablet 0638bc19aaad4597b0045fc9b586c6ff
I20260810 04:45:09.112715 24876 tablet_service.cc:1514] Processing DeleteTablet for tablet 0638bc19aaad4597b0045fc9b586c6ff with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 04:45:09 UTC) from {username='slave'} at 127.0.0.1:54352
I20260810 04:45:09.112725 24443 tablet_service.cc:1514] Processing DeleteTablet for tablet 0638bc19aaad4597b0045fc9b586c6ff with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 04:45:09 UTC) from {username='slave'} at 127.0.0.1:53410
I20260810 04:45:09.112952 25052 tablet_replica.cc:330] T 0638bc19aaad4597b0045fc9b586c6ff P bc3ff21d2db748e19cd1035ab4e18950: stopping tablet replica
I20260810 04:45:09.112942 24664 tablet_service.cc:1514] Processing DeleteTablet for tablet 0638bc19aaad4597b0045fc9b586c6ff with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 04:45:09 UTC) from {username='slave'} at 127.0.0.1:44568
I20260810 04:45:09.113050 25052 raft_consensus.cc:2229] T 0638bc19aaad4597b0045fc9b586c6ff P bc3ff21d2db748e19cd1035ab4e18950 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 04:45:09.113171 25055 tablet_replica.cc:330] T 0638bc19aaad4597b0045fc9b586c6ff P f3256047df96413cba5af99e7ad958d5: stopping tablet replica
I20260810 04:45:09.113292 25055 raft_consensus.cc:2229] T 0638bc19aaad4597b0045fc9b586c6ff P f3256047df96413cba5af99e7ad958d5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 04:45:09.113415 25055 raft_consensus.cc:2258] T 0638bc19aaad4597b0045fc9b586c6ff P f3256047df96413cba5af99e7ad958d5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 04:45:09.113408 25051 tablet_replica.cc:330] T 0638bc19aaad4597b0045fc9b586c6ff P bc39d83190e1457bb87290ab89cb14d6: stopping tablet replica
I20260810 04:45:09.113454 23197 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskemv_AM/build/release/bin/kudu with pid 24344
I20260810 04:45:09.113500 25051 raft_consensus.cc:2229] T 0638bc19aaad4597b0045fc9b586c6ff P bc39d83190e1457bb87290ab89cb14d6 [term 1 LEADER]: Raft consensus shutting down.
I20260810 04:45:09.113719 25051 raft_consensus.cc:2258] T 0638bc19aaad4597b0045fc9b586c6ff P bc39d83190e1457bb87290ab89cb14d6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 04:45:09.144593 25055 ts_tablet_manager.cc:1905] T 0638bc19aaad4597b0045fc9b586c6ff P f3256047df96413cba5af99e7ad958d5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 04:45:09.146445 25055 ts_tablet_manager.cc:1918] T 0638bc19aaad4597b0045fc9b586c6ff P f3256047df96413cba5af99e7ad958d5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 04:45:09.146641 25055 log.cc:1196] T 0638bc19aaad4597b0045fc9b586c6ff P f3256047df96413cba5af99e7ad958d5: Deleting WAL directory at /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786336863388938-23197-0/minicluster-data/ts-1/wal/wals/0638bc19aaad4597b0045fc9b586c6ff
I20260810 04:45:09.146641 25051 ts_tablet_manager.cc:1905] T 0638bc19aaad4597b0045fc9b586c6ff P bc39d83190e1457bb87290ab89cb14d6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 04:45:09.147086 25055 ts_tablet_manager.cc:1939] T 0638bc19aaad4597b0045fc9b586c6ff P f3256047df96413cba5af99e7ad958d5: Deleting consensus metadata
I20260810 04:45:09.147800 24226 catalog_manager.cc:4841] TS f3256047df96413cba5af99e7ad958d5 (127.22.167.66:32895): tablet 0638bc19aaad4597b0045fc9b586c6ff (table test-table [id=9c98091788f94e30aa51808402a15cc3]) successfully deleted
I20260810 04:45:09.148361 25051 ts_tablet_manager.cc:1918] T 0638bc19aaad4597b0045fc9b586c6ff P bc39d83190e1457bb87290ab89cb14d6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 04:45:09.148437 25051 log.cc:1196] T 0638bc19aaad4597b0045fc9b586c6ff P bc39d83190e1457bb87290ab89cb14d6: Deleting WAL directory at /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786336863388938-23197-0/minicluster-data/ts-2/wal/wals/0638bc19aaad4597b0045fc9b586c6ff
I20260810 04:45:09.148739 25051 ts_tablet_manager.cc:1939] T 0638bc19aaad4597b0045fc9b586c6ff P bc39d83190e1457bb87290ab89cb14d6: Deleting consensus metadata
I20260810 04:45:09.149152 24229 catalog_manager.cc:4841] TS bc39d83190e1457bb87290ab89cb14d6 (127.22.167.67:41433): tablet 0638bc19aaad4597b0045fc9b586c6ff (table test-table [id=9c98091788f94e30aa51808402a15cc3]) successfully deleted
W20260810 04:45:09.149776 24226 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.22.167.65:38545 (error 108)
I20260810 04:45:09.150009 23197 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskemv_AM/build/release/bin/kudu with pid 24564
I20260810 04:45:09.150355 25001 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:45:09.150213 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.65:38545 (local address 127.0.0.1:53424)
0810 04:45:09.150261 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0810 04:45:09.150319 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.167.65:38545: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":21}
W20260810 04:45:09.150471 24226 catalog_manager.cc:4584] TS bc3ff21d2db748e19cd1035ab4e18950 (127.22.167.65:38545): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 0638bc19aaad4597b0045fc9b586c6ff: Network error: Client connection negotiation failed: client connection to 127.22.167.65:38545: connect: Connection refused (error 111)
I20260810 04:45:09.158114 23197 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskemv_AM/build/release/bin/kudu with pid 24778
I20260810 04:45:09.171059 23197 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskemv_AM/build/release/bin/kudu with pid 24200
2026-08-10T04:45:09Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/5 (1099 ms)
[----------] 6 tests from EncryptionPolicyTest (9662 ms total)
[----------] 18 tests from AuthTokenIssuingTest
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/0
Loading random data
Initializing database '/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786336863388938-23197-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 04:45:09 dist-test-slave-bqcl krb5kdc[25061](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 04:45:09 dist-test-slave-bqcl krb5kdc[25061](info): set up 2 sockets
Aug 10 04:45:09 dist-test-slave-bqcl krb5kdc[25061](info): commencing operation
krb5kdc: starting...
W20260810 04:45:10.322786 23197 mini_kdc.cc:121] Time spent starting KDC: real 1.128s user 0.003s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786336863388938-23197-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786336863388938-23197-0/krb5kdc/test-user.keytab.
Aug 10 04:45:10 dist-test-slave-bqcl krb5kdc[25061](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786337110, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T04:45:10Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T04:45:10Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786336863388938-23197-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786336863388938-23197-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260810 04:45:10.434464 23197 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskemv_AM/build/release/bin/kudu
/tmp/dist-test-taskemv_AM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786336863388938-23197-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786336863388938-23197-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786336863388938-23197-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786336863388938-23197-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34163
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786336863388938-23197-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:39651
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:34163
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786336863388938-23197-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786336863388938-23197-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786336863388938-23197-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 04:45:10.514153 25076 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 04:45:10.514358 25076 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 04:45:10.514380 25076 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 04:45:10.515496 25076 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 04:45:10.515537 25076 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 04:45:10.515549 25076 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 04:45:10.515560 25076 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 04:45:10.515571 25076 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 04:45:10.517069 25076 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:39651
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786336863388938-23197-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786336863388938-23197-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:34163
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786336863388938-23197-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34163
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786336863388938-23197-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25076
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786336863388938-23197-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 03:54:01 UTC on ccycloud.quasar-icijmt.root.comops.site
build id 4577009
I20260810 04:45:10.517293 25076 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 04:45:10.517527 25076 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 04:45:10.522150 25087 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 04:45:10.522184 25085 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 04:45:10.522150 25090 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 04:45:10.522213 25076 server_base.cc:1034] running on GCE node
I20260810 04:45:10.522534 25076 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 04:45:10.522789 25076 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 04:45:10.523932 25076 hybrid_clock.cc:648] HybridClock initialized: now 1786337110523918 us; error 30 us; skew 500 ppm
Aug 10 04:45:10 dist-test-slave-bqcl krb5kdc[25061](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786337110, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 04:45:10.529134 25076 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260810 04:45:10.529559 25076 webserver.cc:460] Webserver started at http://127.0.0.1:34485/ using document root <none> and password file <none>
I20260810 04:45:10.529775 25076 fs_manager.cc:359] Metadata directory not provided
I20260810 04:45:10.529824 25076 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 04:45:10.529953 25076 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 04:45:10.530984 25076 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786336863388938-23197-0/minicluster-data/master-0/data/instance:
uuid: "5ceb3d5c41654733b5aec508fdd6ec61"
format_stamp: "Formatted at 2026-08-10 04:45:10 on dist-test-slave-bqcl"
server_key: "a12e1938cac881c1244d2956c90375c5"
server_key_iv: "7d9511a8115ab3393ee9e743870e99bc"
server_key_version: "encryptionkey@0"
I20260810 04:45:10.531316 25076 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786336863388938-23197-0/minicluster-data/master-0/wal/instance:
uuid: "5ceb3d5c41654733b5aec508fdd6ec61"
format_stamp: "Formatted at 2026-08-10 04:45:10 on dist-test-slave-bqcl"
server_key: "a12e1938cac881c1244d2956c90375c5"
server_key_iv: "7d9511a8115ab3393ee9e743870e99bc"
server_key_version: "encryptionkey@0"
I20260810 04:45:10.532532 25076 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260810 04:45:10.533177 25101 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 04:45:10.533330 25076 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260810 04:45:10.533412 25076 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786336863388938-23197-0/minicluster-data/master-0/data,/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786336863388938-23197-0/minicluster-data/master-0/wal
uuid: "5ceb3d5c41654733b5aec508fdd6ec61"
format_stamp: "Formatted at 2026-08-10 04:45:10 on dist-test-slave-bqcl"
server_key: "a12e1938cac881c1244d2956c90375c5"
server_key_iv: "7d9511a8115ab3393ee9e743870e99bc"
server_key_version: "encryptionkey@0"
I20260810 04:45:10.533478 25076 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786336863388938-23197-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786336863388938-23197-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786336863388938-23197-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 04:45:10.549492 25076 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 04:45:10.551622 25076 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 04:45:10.551767 25076 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 04:45:10.555711 25076 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:34163
I20260810 04:45:10.555751 25191 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:34163 every 8 connection(s)
I20260810 04:45:10.556066 25076 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786336863388938-23197-0/minicluster-data/master-0/data/info.pb
I20260810 04:45:10.556905 25195 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 04:45:10.558733 23197 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskemv_AM/build/release/bin/kudu as pid 25076
I20260810 04:45:10.558832 23197 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786336863388938-23197-0/minicluster-data/master-0/wal/instance
I20260810 04:45:10.558987 23197 external_mini_cluster.cc:1428] Setting key 8b043312e0e2abeb0e67037ce3295fef
I20260810 04:45:10.559160 25195 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5ceb3d5c41654733b5aec508fdd6ec61: Bootstrap starting.
I20260810 04:45:10.559876 25195 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5ceb3d5c41654733b5aec508fdd6ec61: Neither blocks nor log segments found. Creating new log.
I20260810 04:45:10.560202 25195 log.cc:824] T 00000000000000000000000000000000 P 5ceb3d5c41654733b5aec508fdd6ec61: Log is configured to *not* fsync() on all Append() calls
I20260810 04:45:10.560895 25195 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5ceb3d5c41654733b5aec508fdd6ec61: No bootstrap required, opened a new log
I20260810 04:45:10.563143 25195 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5ceb3d5c41654733b5aec508fdd6ec61 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5ceb3d5c41654733b5aec508fdd6ec61" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34163 } }
I20260810 04:45:10.563293 25195 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5ceb3d5c41654733b5aec508fdd6ec61 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 04:45:10.563337 25195 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5ceb3d5c41654733b5aec508fdd6ec61 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5ceb3d5c41654733b5aec508fdd6ec61, State: Initialized, Role: FOLLOWER
I20260810 04:45:10.563450 25195 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5ceb3d5c41654733b5aec508fdd6ec61 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5ceb3d5c41654733b5aec508fdd6ec61" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34163 } }
I20260810 04:45:10.563517 25195 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5ceb3d5c41654733b5aec508fdd6ec61 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 04:45:10.563553 25195 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5ceb3d5c41654733b5aec508fdd6ec61 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 04:45:10.563599 25195 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5ceb3d5c41654733b5aec508fdd6ec61 [term 0 FOLLOWER]: Advancing to term 1
I20260810 04:45:10.564257 25195 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5ceb3d5c41654733b5aec508fdd6ec61 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5ceb3d5c41654733b5aec508fdd6ec61" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34163 } }
I20260810 04:45:10.564352 25195 leader_election.cc:302] T 00000000000000000000000000000000 P 5ceb3d5c41654733b5aec508fdd6ec61 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5ceb3d5c41654733b5aec508fdd6ec61; no voters:
I20260810 04:45:10.564505 25195 leader_election.cc:288] T 00000000000000000000000000000000 P 5ceb3d5c41654733b5aec508fdd6ec61 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 04:45:10.564536 25204 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5ceb3d5c41654733b5aec508fdd6ec61 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 04:45:10.564754 25195 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5ceb3d5c41654733b5aec508fdd6ec61 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 04:45:10.564741 25204 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5ceb3d5c41654733b5aec508fdd6ec61 [term 1 LEADER]: Becoming Leader. State: Replica: 5ceb3d5c41654733b5aec508fdd6ec61, State: Running, Role: LEADER
I20260810 04:45:10.564917 25204 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5ceb3d5c41654733b5aec508fdd6ec61 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5ceb3d5c41654733b5aec508fdd6ec61" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34163 } }
I20260810 04:45:10.565270 25205 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5ceb3d5c41654733b5aec508fdd6ec61 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5ceb3d5c41654733b5aec508fdd6ec61" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5ceb3d5c41654733b5aec508fdd6ec61" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34163 } } }
I20260810 04:45:10.565348 25205 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5ceb3d5c41654733b5aec508fdd6ec61 [sys.catalog]: This master's current role is: LEADER
I20260810 04:45:10.565567 25206 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5ceb3d5c41654733b5aec508fdd6ec61 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5ceb3d5c41654733b5aec508fdd6ec61. Latest consensus state: current_term: 1 leader_uuid: "5ceb3d5c41654733b5aec508fdd6ec61" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5ceb3d5c41654733b5aec508fdd6ec61" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34163 } } }
I20260810 04:45:10.565855 25206 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5ceb3d5c41654733b5aec508fdd6ec61 [sys.catalog]: This master's current role is: LEADER
I20260810 04:45:10.566020 25213 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
Aug 10 04:45:10 dist-test-slave-bqcl krb5kdc[25061](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786337110, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260810 04:45:10.566569 25213 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 04:45:10.568131 25213 catalog_manager.cc:1347] Generated new cluster ID: 3ce167ecd1454a59bcfc6a0eec970da7
I20260810 04:45:10.568220 25213 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 04:45:10.570312 25199 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:45:10.559466 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55735 (local address 127.0.0.1:34163)
0810 04:45:10.559637 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0810 04:45:10.559646 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 04:45:10.559680 (+ 34us) server_negotiation.cc:407] Connection header received
0810 04:45:10.560222 (+ 542us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:45:10.560231 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:45:10.560302 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:45:10.560445 (+ 143us) server_negotiation.cc:226] Negotiated authn=SASL
0810 04:45:10.561090 (+ 645us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:45:10.562512 (+ 1422us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:45:10.563241 (+ 729us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:45:10.563842 (+ 601us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:45:10.563958 (+ 116us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:45:10.566761 (+ 2803us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 04:45:10.566765 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 04:45:10.566765 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 04:45:10.566776 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0810 04:45:10.568652 (+ 1876us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 04:45:10.569090 (+ 438us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 04:45:10.569092 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 04:45:10.569093 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 04:45:10.569197 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 04:45:10.569430 (+ 233us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 04:45:10.569431 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 04:45:10.569431 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 04:45:10.569703 (+ 272us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 04:45:10.569852 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0810 04:45:10.569977 (+ 125us) server_negotiation.cc:299] Negotiation successful
0810 04:45:10.570067 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":117,"thread_start_us":79,"threads_started":1}
I20260810 04:45:10.571179 23197 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 04:45:10.579493 25199 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:45:10.572715 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35319 (local address 127.0.0.1:34163)
0810 04:45:10.572748 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0810 04:45:10.572750 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 04:45:10.572757 (+ 7us) server_negotiation.cc:407] Connection header received
0810 04:45:10.572775 (+ 18us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:45:10.572776 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:45:10.572786 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:45:10.572845 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0810 04:45:10.573415 (+ 570us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:45:10.574212 (+ 797us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:45:10.575133 (+ 921us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:45:10.575442 (+ 309us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:45:10.575528 (+ 86us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:45:10.576773 (+ 1245us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 04:45:10.576775 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 04:45:10.576775 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 04:45:10.576782 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 04:45:10.578278 (+ 1496us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 04:45:10.578656 (+ 378us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 04:45:10.578657 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 04:45:10.578658 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 04:45:10.578715 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 04:45:10.578907 (+ 192us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 04:45:10.578908 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 04:45:10.578908 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 04:45:10.579033 (+ 125us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 04:45:10.579134 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0810 04:45:10.579233 (+ 99us) server_negotiation.cc:299] Negotiation successful
0810 04:45:10.579295 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260810 04:45:10.587790 25213 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 04:45:10.588315 25213 catalog_manager.cc:1504] Loading token signing keys...
I20260810 04:45:10.591997 25213 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5ceb3d5c41654733b5aec508fdd6ec61: Generated new TSK 0
I20260810 04:45:10.592185 25213 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 04:45:10.597291 23197 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskemv_AM/build/release/bin/kudu with pid 25076
W20260810 04:45:10.613588 25057 connection.cc:535] client connection to 127.0.0.1:34163 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:34163): Connection reset by peer (error 104)
2026-08-10T04:45:10Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/0 (1432 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/1
Loading random data
Initializing database '/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786336863388938-23197-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 04:45:10 dist-test-slave-bqcl krb5kdc[25250](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 04:45:10 dist-test-slave-bqcl krb5kdc[25250](info): set up 2 sockets
Aug 10 04:45:10 dist-test-slave-bqcl krb5kdc[25250](info): commencing operation
krb5kdc: starting...
W20260810 04:45:11.761961 23197 mini_kdc.cc:121] Time spent starting KDC: real 1.135s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786336863388938-23197-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786336863388938-23197-0/krb5kdc/test-user.keytab.
Aug 10 04:45:11 dist-test-slave-bqcl krb5kdc[25250](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786337111, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T04:45:11Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T04:45:11Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786336863388938-23197-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786336863388938-23197-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260810 04:45:11.874380 23197 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskemv_AM/build/release/bin/kudu
/tmp/dist-test-taskemv_AM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786336863388938-23197-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786336863388938-23197-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786336863388938-23197-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786336863388938-23197-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36279
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786336863388938-23197-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:41853
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:36279
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786336863388938-23197-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786336863388938-23197-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786336863388938-23197-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 04:45:11.955161 25269 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 04:45:11.955365 25269 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 04:45:11.955387 25269 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 04:45:11.956537 25269 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 04:45:11.956579 25269 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 04:45:11.956593 25269 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 04:45:11.956624 25269 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 04:45:11.956646 25269 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 04:45:11.958153 25269 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:41853
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786336863388938-23197-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786336863388938-23197-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:36279
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786336863388938-23197-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36279
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786336863388938-23197-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25269
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786336863388938-23197-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 03:54:01 UTC on ccycloud.quasar-icijmt.root.comops.site
build id 4577009
I20260810 04:45:11.958379 25269 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 04:45:11.958575 25269 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 04:45:11.963063 25279 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 04:45:11.963075 25277 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 04:45:11.963112 25269 server_base.cc:1034] running on GCE node
W20260810 04:45:11.963184 25283 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 04:45:11.963524 25269 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 04:45:11.963797 25269 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 04:45:11.964938 25269 hybrid_clock.cc:648] HybridClock initialized: now 1786337111964919 us; error 35 us; skew 500 ppm
Aug 10 04:45:11 dist-test-slave-bqcl krb5kdc[25250](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786337111, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 04:45:11.967357 25269 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260810 04:45:11.967753 25269 webserver.cc:460] Webserver started at http://127.0.0.1:32799/ using document root <none> and password file <none>
I20260810 04:45:11.967933 25269 fs_manager.cc:359] Metadata directory not provided
I20260810 04:45:11.967972 25269 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 04:45:11.968065 25269 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 04:45:11.969022 25269 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786336863388938-23197-0/minicluster-data/master-0/data/instance:
uuid: "857d160fd8524b3dbc9ca71e10d85a09"
format_stamp: "Formatted at 2026-08-10 04:45:11 on dist-test-slave-bqcl"
server_key: "d4e92f3e304cd9a10f510bcf6711dd8a"
server_key_iv: "d555425623f124ceebf4d4ac9a2d4c00"
server_key_version: "encryptionkey@0"
I20260810 04:45:11.969309 25269 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786336863388938-23197-0/minicluster-data/master-0/wal/instance:
uuid: "857d160fd8524b3dbc9ca71e10d85a09"
format_stamp: "Formatted at 2026-08-10 04:45:11 on dist-test-slave-bqcl"
server_key: "d4e92f3e304cd9a10f510bcf6711dd8a"
server_key_iv: "d555425623f124ceebf4d4ac9a2d4c00"
server_key_version: "encryptionkey@0"
I20260810 04:45:11.970517 25269 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260810 04:45:11.971267 25295 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 04:45:11.971462 25269 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 04:45:11.971550 25269 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786336863388938-23197-0/minicluster-data/master-0/data,/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786336863388938-23197-0/minicluster-data/master-0/wal
uuid: "857d160fd8524b3dbc9ca71e10d85a09"
format_stamp: "Formatted at 2026-08-10 04:45:11 on dist-test-slave-bqcl"
server_key: "d4e92f3e304cd9a10f510bcf6711dd8a"
server_key_iv: "d555425623f124ceebf4d4ac9a2d4c00"
server_key_version: "encryptionkey@0"
I20260810 04:45:11.971607 25269 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786336863388938-23197-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786336863388938-23197-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786336863388938-23197-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 04:45:11.982239 25269 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 04:45:11.983839 25269 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 04:45:11.983940 25269 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 04:45:11.987691 25269 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:36279
I20260810 04:45:11.987735 25377 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:36279 every 8 connection(s)
I20260810 04:45:11.988056 25269 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786336863388938-23197-0/minicluster-data/master-0/data/info.pb
I20260810 04:45:11.988535 23197 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskemv_AM/build/release/bin/kudu as pid 25269
I20260810 04:45:11.988684 23197 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786336863388938-23197-0/minicluster-data/master-0/wal/instance
I20260810 04:45:11.988880 23197 external_mini_cluster.cc:1428] Setting key fec305141a66f38b257b21e54d3bf7a0
I20260810 04:45:11.989068 25381 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 04:45:11.992434 25381 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 857d160fd8524b3dbc9ca71e10d85a09: Bootstrap starting.
I20260810 04:45:11.993036 25381 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 857d160fd8524b3dbc9ca71e10d85a09: Neither blocks nor log segments found. Creating new log.
I20260810 04:45:11.993367 25381 log.cc:824] T 00000000000000000000000000000000 P 857d160fd8524b3dbc9ca71e10d85a09: Log is configured to *not* fsync() on all Append() calls
I20260810 04:45:11.994177 25381 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 857d160fd8524b3dbc9ca71e10d85a09: No bootstrap required, opened a new log
I20260810 04:45:11.994979 25387 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:45:11.989370 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56541 (local address 127.0.0.1:36279)
0810 04:45:11.989586 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0810 04:45:11.989595 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 04:45:11.989626 (+ 31us) server_negotiation.cc:407] Connection header received
0810 04:45:11.990235 (+ 609us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:45:11.990244 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:45:11.990307 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:45:11.990451 (+ 144us) server_negotiation.cc:226] Negotiated authn=SASL
0810 04:45:11.990691 (+ 240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:45:11.992000 (+ 1309us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:45:11.992845 (+ 845us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:45:11.993431 (+ 586us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:45:11.993558 (+ 127us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:45:11.994325 (+ 767us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 04:45:11.994330 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 04:45:11.994330 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 04:45:11.994521 (+ 191us) server_negotiation.cc:922] calling sasl_server_start()
0810 04:45:11.994561 (+ 40us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 04:45:11.994575 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 04:45:11.994679 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0810 04:45:11.994745 (+ 66us) server_negotiation.cc:299] Negotiation successful
0810 04:45:11.994771 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":165,"thread_start_us":78,"threads_started":1}
I20260810 04:45:11.996030 23197 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 04:45:11.996471 25381 raft_consensus.cc:348] T 00000000000000000000000000000000 P 857d160fd8524b3dbc9ca71e10d85a09 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "857d160fd8524b3dbc9ca71e10d85a09" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36279 } }
I20260810 04:45:11.996646 25381 raft_consensus.cc:374] T 00000000000000000000000000000000 P 857d160fd8524b3dbc9ca71e10d85a09 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 04:45:11.996687 25381 raft_consensus.cc:729] T 00000000000000000000000000000000 P 857d160fd8524b3dbc9ca71e10d85a09 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 857d160fd8524b3dbc9ca71e10d85a09, State: Initialized, Role: FOLLOWER
I20260810 04:45:11.996807 25381 consensus_queue.cc:260] T 00000000000000000000000000000000 P 857d160fd8524b3dbc9ca71e10d85a09 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "857d160fd8524b3dbc9ca71e10d85a09" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36279 } }
I20260810 04:45:11.996884 25381 raft_consensus.cc:388] T 00000000000000000000000000000000 P 857d160fd8524b3dbc9ca71e10d85a09 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 04:45:11.996918 25381 raft_consensus.cc:482] T 00000000000000000000000000000000 P 857d160fd8524b3dbc9ca71e10d85a09 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 04:45:11.996963 25381 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 857d160fd8524b3dbc9ca71e10d85a09 [term 0 FOLLOWER]: Advancing to term 1
I20260810 04:45:11.997632 25381 raft_consensus.cc:504] T 00000000000000000000000000000000 P 857d160fd8524b3dbc9ca71e10d85a09 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "857d160fd8524b3dbc9ca71e10d85a09" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36279 } }
I20260810 04:45:11.997750 25381 leader_election.cc:302] T 00000000000000000000000000000000 P 857d160fd8524b3dbc9ca71e10d85a09 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 857d160fd8524b3dbc9ca71e10d85a09; no voters:
I20260810 04:45:11.997912 25381 leader_election.cc:288] T 00000000000000000000000000000000 P 857d160fd8524b3dbc9ca71e10d85a09 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 04:45:11.997946 25399 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 857d160fd8524b3dbc9ca71e10d85a09 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 04:45:11.998114 25399 raft_consensus.cc:686] T 00000000000000000000000000000000 P 857d160fd8524b3dbc9ca71e10d85a09 [term 1 LEADER]: Becoming Leader. State: Replica: 857d160fd8524b3dbc9ca71e10d85a09, State: Running, Role: LEADER
I20260810 04:45:11.998190 25381 sys_catalog.cc:564] T 00000000000000000000000000000000 P 857d160fd8524b3dbc9ca71e10d85a09 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 04:45:11.998292 25399 consensus_queue.cc:237] T 00000000000000000000000000000000 P 857d160fd8524b3dbc9ca71e10d85a09 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "857d160fd8524b3dbc9ca71e10d85a09" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36279 } }
I20260810 04:45:11.998709 25399 sys_catalog.cc:455] T 00000000000000000000000000000000 P 857d160fd8524b3dbc9ca71e10d85a09 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 857d160fd8524b3dbc9ca71e10d85a09. Latest consensus state: current_term: 1 leader_uuid: "857d160fd8524b3dbc9ca71e10d85a09" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "857d160fd8524b3dbc9ca71e10d85a09" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36279 } } }
I20260810 04:45:11.998950 25399 sys_catalog.cc:458] T 00000000000000000000000000000000 P 857d160fd8524b3dbc9ca71e10d85a09 [sys.catalog]: This master's current role is: LEADER
I20260810 04:45:11.998883 25402 sys_catalog.cc:455] T 00000000000000000000000000000000 P 857d160fd8524b3dbc9ca71e10d85a09 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "857d160fd8524b3dbc9ca71e10d85a09" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "857d160fd8524b3dbc9ca71e10d85a09" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36279 } } }
I20260810 04:45:11.999396 25402 sys_catalog.cc:458] T 00000000000000000000000000000000 P 857d160fd8524b3dbc9ca71e10d85a09 [sys.catalog]: This master's current role is: LEADER
I20260810 04:45:11.999526 25410 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 04:45:11.999969 25410 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 04:45:12.000936 25387 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:45:11.997559 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57335 (local address 127.0.0.1:36279)
0810 04:45:11.997610 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0810 04:45:11.997612 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 04:45:11.997754 (+ 142us) server_negotiation.cc:407] Connection header received
0810 04:45:11.997802 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:45:11.997803 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:45:11.997812 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:45:11.997863 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0810 04:45:11.998035 (+ 172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:45:11.998734 (+ 699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:45:11.999800 (+ 1066us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:45:12.000077 (+ 277us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:45:12.000156 (+ 79us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:45:12.000696 (+ 540us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 04:45:12.000697 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 04:45:12.000697 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 04:45:12.000702 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0810 04:45:12.000715 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 04:45:12.000719 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 04:45:12.000770 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0810 04:45:12.000860 (+ 90us) server_negotiation.cc:299] Negotiation successful
0810 04:45:12.000871 (+ 11us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260810 04:45:12.001493 25410 catalog_manager.cc:1347] Generated new cluster ID: fbd8658ad928426cac9b315b8b51b97a
I20260810 04:45:12.001538 25410 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 04:45:12.011276 25410 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 04:45:12.011780 25410 catalog_manager.cc:1504] Loading token signing keys...
I20260810 04:45:12.019479 25410 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 857d160fd8524b3dbc9ca71e10d85a09: Generated new TSK 0
I20260810 04:45:12.019635 25410 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 04:45:12.027143 23197 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskemv_AM/build/release/bin/kudu with pid 25269
W20260810 04:45:12.036456 25247 connection.cc:535] client connection to 127.0.0.1:36279 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:36279): Connection reset by peer (error 104)
2026-08-10T04:45:12Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/1 (1422 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/2
Loading random data
Initializing database '/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786336863388938-23197-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 04:45:12 dist-test-slave-bqcl krb5kdc[25437](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 04:45:12 dist-test-slave-bqcl krb5kdc[25437](info): set up 2 sockets
Aug 10 04:45:12 dist-test-slave-bqcl krb5kdc[25437](info): commencing operation
krb5kdc: starting...
W20260810 04:45:13.178364 23197 mini_kdc.cc:121] Time spent starting KDC: real 1.129s user 0.003s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786336863388938-23197-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786336863388938-23197-0/krb5kdc/test-user.keytab.
Aug 10 04:45:13 dist-test-slave-bqcl krb5kdc[25437](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786337113, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T04:45:13Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T04:45:13Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786336863388938-23197-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786336863388938-23197-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260810 04:45:13.294548 23197 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskemv_AM/build/release/bin/kudu
/tmp/dist-test-taskemv_AM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786336863388938-23197-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786336863388938-23197-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786336863388938-23197-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786336863388938-23197-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36391
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786336863388938-23197-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:43977
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:36391
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786336863388938-23197-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786336863388938-23197-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786336863388938-23197-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 04:45:13.377578 25469 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 04:45:13.377815 25469 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 04:45:13.377851 25469 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 04:45:13.379071 25469 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 04:45:13.379122 25469 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 04:45:13.379144 25469 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 04:45:13.379163 25469 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 04:45:13.379184 25469 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 04:45:13.380698 25469 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:43977
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786336863388938-23197-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786336863388938-23197-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:36391
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786336863388938-23197-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36391
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786336863388938-23197-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25469
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786336863388938-23197-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 03:54:01 UTC on ccycloud.quasar-icijmt.root.comops.site
build id 4577009
I20260810 04:45:13.380962 25469 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 04:45:13.381187 25469 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 04:45:13.385857 25475 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 04:45:13.385864 25478 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 04:45:13.385872 25481 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 04:45:13.386097 25469 server_base.cc:1034] running on GCE node
I20260810 04:45:13.386379 25469 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 04:45:13.386634 25469 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 04:45:13.387778 25469 hybrid_clock.cc:648] HybridClock initialized: now 1786337113387758 us; error 33 us; skew 500 ppm
Aug 10 04:45:13 dist-test-slave-bqcl krb5kdc[25437](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786337113, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 04:45:13.390220 25469 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260810 04:45:13.390628 25469 webserver.cc:460] Webserver started at http://127.0.0.1:41093/ using document root <none> and password file <none>
I20260810 04:45:13.390836 25469 fs_manager.cc:359] Metadata directory not provided
I20260810 04:45:13.390883 25469 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 04:45:13.391004 25469 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 04:45:13.392021 25469 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786336863388938-23197-0/minicluster-data/master-0/data/instance:
uuid: "f2663b0ec97f4db4a26269264fd9b2ab"
format_stamp: "Formatted at 2026-08-10 04:45:13 on dist-test-slave-bqcl"
server_key: "9c746520fdf93c956ce20b49b3af33c5"
server_key_iv: "f66e2800535777dac1b449d3abbb537b"
server_key_version: "encryptionkey@0"
I20260810 04:45:13.392347 25469 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786336863388938-23197-0/minicluster-data/master-0/wal/instance:
uuid: "f2663b0ec97f4db4a26269264fd9b2ab"
format_stamp: "Formatted at 2026-08-10 04:45:13 on dist-test-slave-bqcl"
server_key: "9c746520fdf93c956ce20b49b3af33c5"
server_key_iv: "f66e2800535777dac1b449d3abbb537b"
server_key_version: "encryptionkey@0"
I20260810 04:45:13.393631 25469 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260810 04:45:13.394320 25490 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 04:45:13.394493 25469 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260810 04:45:13.394580 25469 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786336863388938-23197-0/minicluster-data/master-0/data,/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786336863388938-23197-0/minicluster-data/master-0/wal
uuid: "f2663b0ec97f4db4a26269264fd9b2ab"
format_stamp: "Formatted at 2026-08-10 04:45:13 on dist-test-slave-bqcl"
server_key: "9c746520fdf93c956ce20b49b3af33c5"
server_key_iv: "f66e2800535777dac1b449d3abbb537b"
server_key_version: "encryptionkey@0"
I20260810 04:45:13.394649 25469 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786336863388938-23197-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786336863388938-23197-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786336863388938-23197-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 04:45:13.397882 25469 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 04:45:13.400000 25469 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 04:45:13.400135 25469 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 04:45:13.403967 25469 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:36391
I20260810 04:45:13.403999 25586 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:36391 every 8 connection(s)
I20260810 04:45:13.404300 25469 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786336863388938-23197-0/minicluster-data/master-0/data/info.pb
I20260810 04:45:13.405267 25587 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 04:45:13.407508 25587 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f2663b0ec97f4db4a26269264fd9b2ab: Bootstrap starting.
I20260810 04:45:13.408053 25587 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f2663b0ec97f4db4a26269264fd9b2ab: Neither blocks nor log segments found. Creating new log.
I20260810 04:45:13.408339 25587 log.cc:824] T 00000000000000000000000000000000 P f2663b0ec97f4db4a26269264fd9b2ab: Log is configured to *not* fsync() on all Append() calls
I20260810 04:45:13.408991 25587 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f2663b0ec97f4db4a26269264fd9b2ab: No bootstrap required, opened a new log
I20260810 04:45:13.409044 23197 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskemv_AM/build/release/bin/kudu as pid 25469
I20260810 04:45:13.409178 23197 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786336863388938-23197-0/minicluster-data/master-0/wal/instance
I20260810 04:45:13.409358 23197 external_mini_cluster.cc:1428] Setting key b65e4f0ad7d316bf46c82163998519ef
I20260810 04:45:13.411046 25587 raft_consensus.cc:348] T 00000000000000000000000000000000 P f2663b0ec97f4db4a26269264fd9b2ab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f2663b0ec97f4db4a26269264fd9b2ab" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36391 } }
I20260810 04:45:13.411226 25587 raft_consensus.cc:374] T 00000000000000000000000000000000 P f2663b0ec97f4db4a26269264fd9b2ab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 04:45:13.411305 25587 raft_consensus.cc:729] T 00000000000000000000000000000000 P f2663b0ec97f4db4a26269264fd9b2ab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f2663b0ec97f4db4a26269264fd9b2ab, State: Initialized, Role: FOLLOWER
I20260810 04:45:13.411461 25587 consensus_queue.cc:260] T 00000000000000000000000000000000 P f2663b0ec97f4db4a26269264fd9b2ab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f2663b0ec97f4db4a26269264fd9b2ab" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36391 } }
I20260810 04:45:13.411553 25587 raft_consensus.cc:388] T 00000000000000000000000000000000 P f2663b0ec97f4db4a26269264fd9b2ab [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 04:45:13.411617 25587 raft_consensus.cc:482] T 00000000000000000000000000000000 P f2663b0ec97f4db4a26269264fd9b2ab [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 04:45:13.411674 25587 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f2663b0ec97f4db4a26269264fd9b2ab [term 0 FOLLOWER]: Advancing to term 1
I20260810 04:45:13.412268 25593 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:45:13.409855 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46421 (local address 127.0.0.1:36391)
0810 04:45:13.410041 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0810 04:45:13.410050 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 04:45:13.410084 (+ 34us) server_negotiation.cc:407] Connection header received
0810 04:45:13.411016 (+ 932us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:45:13.411025 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:45:13.411073 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:45:13.411222 (+ 149us) server_negotiation.cc:226] Negotiated authn=SASL
0810 04:45:13.411804 (+ 582us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 04:45:13.411807 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 04:45:13.411808 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 04:45:13.411812 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0810 04:45:13.411853 (+ 41us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 04:45:13.411864 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 04:45:13.411919 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0810 04:45:13.412011 (+ 92us) server_negotiation.cc:299] Negotiation successful
0810 04:45:13.412043 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":132,"thread_start_us":80,"threads_started":1}
I20260810 04:45:13.412770 25587 raft_consensus.cc:504] T 00000000000000000000000000000000 P f2663b0ec97f4db4a26269264fd9b2ab [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f2663b0ec97f4db4a26269264fd9b2ab" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36391 } }
I20260810 04:45:13.412875 25587 leader_election.cc:302] T 00000000000000000000000000000000 P f2663b0ec97f4db4a26269264fd9b2ab [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f2663b0ec97f4db4a26269264fd9b2ab; no voters:
I20260810 04:45:13.413029 25587 leader_election.cc:288] T 00000000000000000000000000000000 P f2663b0ec97f4db4a26269264fd9b2ab [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 04:45:13.413074 25597 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f2663b0ec97f4db4a26269264fd9b2ab [term 1 FOLLOWER]: Leader election won for term 1
I20260810 04:45:13.413189 23197 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 04:45:13.413226 25597 raft_consensus.cc:686] T 00000000000000000000000000000000 P f2663b0ec97f4db4a26269264fd9b2ab [term 1 LEADER]: Becoming Leader. State: Replica: f2663b0ec97f4db4a26269264fd9b2ab, State: Running, Role: LEADER
I20260810 04:45:13.413280 25587 sys_catalog.cc:564] T 00000000000000000000000000000000 P f2663b0ec97f4db4a26269264fd9b2ab [sys.catalog]: configured and running, proceeding with master startup.
I20260810 04:45:13.413340 25597 consensus_queue.cc:237] T 00000000000000000000000000000000 P f2663b0ec97f4db4a26269264fd9b2ab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f2663b0ec97f4db4a26269264fd9b2ab" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36391 } }
I20260810 04:45:13.413664 25597 sys_catalog.cc:455] T 00000000000000000000000000000000 P f2663b0ec97f4db4a26269264fd9b2ab [sys.catalog]: SysCatalogTable state changed. Reason: New leader f2663b0ec97f4db4a26269264fd9b2ab. Latest consensus state: current_term: 1 leader_uuid: "f2663b0ec97f4db4a26269264fd9b2ab" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f2663b0ec97f4db4a26269264fd9b2ab" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36391 } } }
I20260810 04:45:13.413748 25597 sys_catalog.cc:458] T 00000000000000000000000000000000 P f2663b0ec97f4db4a26269264fd9b2ab [sys.catalog]: This master's current role is: LEADER
I20260810 04:45:13.413681 25600 sys_catalog.cc:455] T 00000000000000000000000000000000 P f2663b0ec97f4db4a26269264fd9b2ab [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f2663b0ec97f4db4a26269264fd9b2ab" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f2663b0ec97f4db4a26269264fd9b2ab" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36391 } } }
I20260810 04:45:13.413882 25600 sys_catalog.cc:458] T 00000000000000000000000000000000 P f2663b0ec97f4db4a26269264fd9b2ab [sys.catalog]: This master's current role is: LEADER
I20260810 04:45:13.413951 25608 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 04:45:13.414485 25608 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 04:45:13.416024 25593 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:45:13.414765 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52289 (local address 127.0.0.1:36391)
0810 04:45:13.414797 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0810 04:45:13.414799 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 04:45:13.415057 (+ 258us) server_negotiation.cc:407] Connection header received
0810 04:45:13.415145 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:45:13.415146 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:45:13.415151 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:45:13.415178 (+ 27us) server_negotiation.cc:226] Negotiated authn=SASL
0810 04:45:13.415809 (+ 631us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 04:45:13.415810 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 04:45:13.415810 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 04:45:13.415814 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0810 04:45:13.415827 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 04:45:13.415832 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 04:45:13.415859 (+ 27us) server_negotiation.cc:1035] Waiting for connection context
0810 04:45:13.415950 (+ 91us) server_negotiation.cc:299] Negotiation successful
0810 04:45:13.415963 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260810 04:45:13.415968 25608 catalog_manager.cc:1347] Generated new cluster ID: 5cd4620e478a433492ae9a537126b209
I20260810 04:45:13.416128 25608 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 04:45:13.420827 25608 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 04:45:13.420974 25608 catalog_manager.cc:1504] Loading token signing keys...
I20260810 04:45:13.425114 25608 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f2663b0ec97f4db4a26269264fd9b2ab: Generated new TSK 0
I20260810 04:45:13.425294 25608 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 04:45:13.429462 23197 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskemv_AM/build/release/bin/kudu with pid 25469
2026-08-10T04:45:13Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/2 (1408 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/3
Loading random data
Initializing database '/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786336863388938-23197-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 04:45:13 dist-test-slave-bqcl krb5kdc[25640](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 04:45:13 dist-test-slave-bqcl krb5kdc[25640](info): set up 2 sockets
Aug 10 04:45:13 dist-test-slave-bqcl krb5kdc[25640](info): commencing operation
krb5kdc: starting...
W20260810 04:45:14.596843 23197 mini_kdc.cc:121] Time spent starting KDC: real 1.138s user 0.004s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786336863388938-23197-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786336863388938-23197-0/krb5kdc/test-user.keytab.
Aug 10 04:45:14 dist-test-slave-bqcl krb5kdc[25640](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786337114, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T04:45:14Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T04:45:14Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786336863388938-23197-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786336863388938-23197-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260810 04:45:14.712822 23197 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskemv_AM/build/release/bin/kudu
/tmp/dist-test-taskemv_AM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786336863388938-23197-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786336863388938-23197-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786336863388938-23197-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786336863388938-23197-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36807
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786336863388938-23197-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:36813
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:36807
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786336863388938-23197-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786336863388938-23197-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786336863388938-23197-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 04:45:14.793912 25668 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 04:45:14.794145 25668 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 04:45:14.794183 25668 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 04:45:14.795354 25668 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 04:45:14.795403 25668 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 04:45:14.795426 25668 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 04:45:14.795444 25668 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 04:45:14.795464 25668 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 04:45:14.796955 25668 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:36813
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786336863388938-23197-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786336863388938-23197-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:36807
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786336863388938-23197-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36807
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786336863388938-23197-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25668
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786336863388938-23197-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 03:54:01 UTC on ccycloud.quasar-icijmt.root.comops.site
build id 4577009
I20260810 04:45:14.797207 25668 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 04:45:14.797429 25668 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 04:45:14.802114 25677 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 04:45:14.802166 25681 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 04:45:14.802258 25668 server_base.cc:1034] running on GCE node
W20260810 04:45:14.802343 25678 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 04:45:14.802623 25668 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 04:45:14.802915 25668 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 04:45:14.804054 25668 hybrid_clock.cc:648] HybridClock initialized: now 1786337114804041 us; error 27 us; skew 500 ppm
Aug 10 04:45:14 dist-test-slave-bqcl krb5kdc[25640](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786337114, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 04:45:14.806584 25668 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260810 04:45:14.806998 25668 webserver.cc:460] Webserver started at http://127.0.0.1:41305/ using document root <none> and password file <none>
I20260810 04:45:14.807219 25668 fs_manager.cc:359] Metadata directory not provided
I20260810 04:45:14.807268 25668 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 04:45:14.807395 25668 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 04:45:14.808394 25668 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786336863388938-23197-0/minicluster-data/master-0/data/instance:
uuid: "843ba2335a5045aa89e3213aba8bf10b"
format_stamp: "Formatted at 2026-08-10 04:45:14 on dist-test-slave-bqcl"
server_key: "9219d9f96fec586559d6350f4e4263b2"
server_key_iv: "714e9b7211b7ae22871af5d3d6c4ff80"
server_key_version: "encryptionkey@0"
I20260810 04:45:14.808723 25668 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786336863388938-23197-0/minicluster-data/master-0/wal/instance:
uuid: "843ba2335a5045aa89e3213aba8bf10b"
format_stamp: "Formatted at 2026-08-10 04:45:14 on dist-test-slave-bqcl"
server_key: "9219d9f96fec586559d6350f4e4263b2"
server_key_iv: "714e9b7211b7ae22871af5d3d6c4ff80"
server_key_version: "encryptionkey@0"
I20260810 04:45:14.809912 25668 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260810 04:45:14.810616 25692 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 04:45:14.810761 25668 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260810 04:45:14.810844 25668 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786336863388938-23197-0/minicluster-data/master-0/data,/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786336863388938-23197-0/minicluster-data/master-0/wal
uuid: "843ba2335a5045aa89e3213aba8bf10b"
format_stamp: "Formatted at 2026-08-10 04:45:14 on dist-test-slave-bqcl"
server_key: "9219d9f96fec586559d6350f4e4263b2"
server_key_iv: "714e9b7211b7ae22871af5d3d6c4ff80"
server_key_version: "encryptionkey@0"
I20260810 04:45:14.810909 25668 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786336863388938-23197-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786336863388938-23197-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786336863388938-23197-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 04:45:14.829991 25668 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 04:45:14.832063 25668 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 04:45:14.832202 25668 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 04:45:14.836094 25668 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:36807
I20260810 04:45:14.836130 25772 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:36807 every 8 connection(s)
I20260810 04:45:14.836444 25668 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786336863388938-23197-0/minicluster-data/master-0/data/info.pb
I20260810 04:45:14.837273 25774 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 04:45:14.837425 23197 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskemv_AM/build/release/bin/kudu as pid 25668
I20260810 04:45:14.837531 23197 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786336863388938-23197-0/minicluster-data/master-0/wal/instance
I20260810 04:45:14.837731 23197 external_mini_cluster.cc:1428] Setting key b833f3d345c6724f73fc1f2564684998
I20260810 04:45:14.840163 25774 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 843ba2335a5045aa89e3213aba8bf10b: Bootstrap starting.
I20260810 04:45:14.840739 25774 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 843ba2335a5045aa89e3213aba8bf10b: Neither blocks nor log segments found. Creating new log.
I20260810 04:45:14.841055 25774 log.cc:824] T 00000000000000000000000000000000 P 843ba2335a5045aa89e3213aba8bf10b: Log is configured to *not* fsync() on all Append() calls
I20260810 04:45:14.841789 25774 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 843ba2335a5045aa89e3213aba8bf10b: No bootstrap required, opened a new log
I20260810 04:45:14.844128 25774 raft_consensus.cc:348] T 00000000000000000000000000000000 P 843ba2335a5045aa89e3213aba8bf10b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "843ba2335a5045aa89e3213aba8bf10b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36807 } }
Aug 10 04:45:14 dist-test-slave-bqcl krb5kdc[25640](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786337114, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260810 04:45:14.844259 25774 raft_consensus.cc:374] T 00000000000000000000000000000000 P 843ba2335a5045aa89e3213aba8bf10b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 04:45:14.844281 25774 raft_consensus.cc:729] T 00000000000000000000000000000000 P 843ba2335a5045aa89e3213aba8bf10b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 843ba2335a5045aa89e3213aba8bf10b, State: Initialized, Role: FOLLOWER
I20260810 04:45:14.844390 25774 consensus_queue.cc:260] T 00000000000000000000000000000000 P 843ba2335a5045aa89e3213aba8bf10b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "843ba2335a5045aa89e3213aba8bf10b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36807 } }
I20260810 04:45:14.844455 25774 raft_consensus.cc:388] T 00000000000000000000000000000000 P 843ba2335a5045aa89e3213aba8bf10b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 04:45:14.844477 25774 raft_consensus.cc:482] T 00000000000000000000000000000000 P 843ba2335a5045aa89e3213aba8bf10b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 04:45:14.844523 25774 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 843ba2335a5045aa89e3213aba8bf10b [term 0 FOLLOWER]: Advancing to term 1
I20260810 04:45:14.845204 25774 raft_consensus.cc:504] T 00000000000000000000000000000000 P 843ba2335a5045aa89e3213aba8bf10b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "843ba2335a5045aa89e3213aba8bf10b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36807 } }
I20260810 04:45:14.845333 25774 leader_election.cc:302] T 00000000000000000000000000000000 P 843ba2335a5045aa89e3213aba8bf10b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 843ba2335a5045aa89e3213aba8bf10b; no voters:
I20260810 04:45:14.845497 25774 leader_election.cc:288] T 00000000000000000000000000000000 P 843ba2335a5045aa89e3213aba8bf10b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 04:45:14.845546 25784 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 843ba2335a5045aa89e3213aba8bf10b [term 1 FOLLOWER]: Leader election won for term 1
I20260810 04:45:14.845680 25784 raft_consensus.cc:686] T 00000000000000000000000000000000 P 843ba2335a5045aa89e3213aba8bf10b [term 1 LEADER]: Becoming Leader. State: Replica: 843ba2335a5045aa89e3213aba8bf10b, State: Running, Role: LEADER
I20260810 04:45:14.845774 25774 sys_catalog.cc:564] T 00000000000000000000000000000000 P 843ba2335a5045aa89e3213aba8bf10b [sys.catalog]: configured and running, proceeding with master startup.
I20260810 04:45:14.845767 25784 consensus_queue.cc:237] T 00000000000000000000000000000000 P 843ba2335a5045aa89e3213aba8bf10b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "843ba2335a5045aa89e3213aba8bf10b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36807 } }
I20260810 04:45:14.846202 25785 sys_catalog.cc:455] T 00000000000000000000000000000000 P 843ba2335a5045aa89e3213aba8bf10b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "843ba2335a5045aa89e3213aba8bf10b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "843ba2335a5045aa89e3213aba8bf10b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36807 } } }
I20260810 04:45:14.846218 25787 sys_catalog.cc:455] T 00000000000000000000000000000000 P 843ba2335a5045aa89e3213aba8bf10b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 843ba2335a5045aa89e3213aba8bf10b. Latest consensus state: current_term: 1 leader_uuid: "843ba2335a5045aa89e3213aba8bf10b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "843ba2335a5045aa89e3213aba8bf10b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36807 } } }
I20260810 04:45:14.846289 25785 sys_catalog.cc:458] T 00000000000000000000000000000000 P 843ba2335a5045aa89e3213aba8bf10b [sys.catalog]: This master's current role is: LEADER
I20260810 04:45:14.846310 25787 sys_catalog.cc:458] T 00000000000000000000000000000000 P 843ba2335a5045aa89e3213aba8bf10b [sys.catalog]: This master's current role is: LEADER
I20260810 04:45:14.846941 25790 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 04:45:14.847437 25790 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 04:45:14.848600 25782 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:45:14.838355 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54507 (local address 127.0.0.1:36807)
0810 04:45:14.838533 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0810 04:45:14.838541 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 04:45:14.838566 (+ 25us) server_negotiation.cc:407] Connection header received
0810 04:45:14.838965 (+ 399us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:45:14.838972 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:45:14.839018 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:45:14.839128 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0810 04:45:14.839888 (+ 760us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:45:14.840789 (+ 901us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:45:14.841716 (+ 927us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:45:14.842182 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:45:14.842264 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:45:14.844776 (+ 2512us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 04:45:14.844780 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 04:45:14.844780 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 04:45:14.844792 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0810 04:45:14.846926 (+ 2134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 04:45:14.847323 (+ 397us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 04:45:14.847326 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 04:45:14.847326 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 04:45:14.847427 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 04:45:14.847697 (+ 270us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 04:45:14.847698 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 04:45:14.847699 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 04:45:14.847993 (+ 294us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 04:45:14.848150 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0810 04:45:14.848271 (+ 121us) server_negotiation.cc:299] Negotiation successful
0810 04:45:14.848402 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":122,"thread_start_us":82,"threads_started":1}
I20260810 04:45:14.849159 25790 catalog_manager.cc:1347] Generated new cluster ID: 490ae55d06b441c4a855b5ed23c71c07
I20260810 04:45:14.849212 25790 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 04:45:14.849609 23197 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 04:45:14.860847 25782 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:45:14.851145 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50847 (local address 127.0.0.1:36807)
0810 04:45:14.854223 (+ 3078us) server_negotiation.cc:206] Beginning negotiation
0810 04:45:14.854224 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0810 04:45:14.854358 (+ 134us) server_negotiation.cc:407] Connection header received
0810 04:45:14.854420 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:45:14.854421 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:45:14.854431 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:45:14.854476 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0810 04:45:14.855084 (+ 608us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:45:14.855793 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:45:14.856783 (+ 990us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:45:14.857080 (+ 297us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:45:14.857134 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:45:14.858436 (+ 1302us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 04:45:14.858438 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 04:45:14.858438 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 04:45:14.858445 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 04:45:14.859850 (+ 1405us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 04:45:14.860208 (+ 358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 04:45:14.860209 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 04:45:14.860209 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 04:45:14.860257 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 04:45:14.860419 (+ 162us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 04:45:14.860419 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 04:45:14.860420 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 04:45:14.860522 (+ 102us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 04:45:14.860609 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0810 04:45:14.860705 (+ 96us) server_negotiation.cc:299] Negotiation successful
0810 04:45:14.860765 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3057}
I20260810 04:45:14.863950 25790 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 04:45:14.864629 25790 catalog_manager.cc:1504] Loading token signing keys...
I20260810 04:45:14.869941 25790 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 843ba2335a5045aa89e3213aba8bf10b: Generated new TSK 0
I20260810 04:45:14.870193 25790 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 04:45:14.877292 23197 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskemv_AM/build/release/bin/kudu with pid 25668
2026-08-10T04:45:14Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/3 (1454 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/4
Loading random data
Initializing database '/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786336863388938-23197-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 04:45:14 dist-test-slave-bqcl krb5kdc[25825](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 04:45:14 dist-test-slave-bqcl krb5kdc[25825](info): set up 2 sockets
Aug 10 04:45:14 dist-test-slave-bqcl krb5kdc[25825](info): commencing operation
krb5kdc: starting...
W20260810 04:45:16.049736 23197 mini_kdc.cc:121] Time spent starting KDC: real 1.136s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786336863388938-23197-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786336863388938-23197-0/krb5kdc/test-user.keytab.
Aug 10 04:45:16 dist-test-slave-bqcl krb5kdc[25825](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786337116, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T04:45:16Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T04:45:16Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786336863388938-23197-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786336863388938-23197-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260810 04:45:16.165589 23197 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskemv_AM/build/release/bin/kudu
/tmp/dist-test-taskemv_AM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786336863388938-23197-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786336863388938-23197-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786336863388938-23197-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786336863388938-23197-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42787
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786336863388938-23197-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:46579
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:42787
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786336863388938-23197-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786336863388938-23197-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786336863388938-23197-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 04:45:16.245163 25848 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 04:45:16.245385 25848 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 04:45:16.245407 25848 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 04:45:16.246536 25848 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 04:45:16.246573 25848 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 04:45:16.246585 25848 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 04:45:16.246596 25848 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 04:45:16.246608 25848 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 04:45:16.247982 25848 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:46579
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786336863388938-23197-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786336863388938-23197-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:42787
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786336863388938-23197-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42787
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786336863388938-23197-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25848
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786336863388938-23197-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 03:54:01 UTC on ccycloud.quasar-icijmt.root.comops.site
build id 4577009
I20260810 04:45:16.248169 25848 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 04:45:16.248338 25848 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 04:45:16.252844 25859 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 04:45:16.252877 25854 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 04:45:16.252853 25855 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 04:45:16.253180 25848 server_base.cc:1034] running on GCE node
I20260810 04:45:16.253428 25848 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 04:45:16.253720 25848 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 04:45:16.254884 25848 hybrid_clock.cc:648] HybridClock initialized: now 1786337116254872 us; error 28 us; skew 500 ppm
Aug 10 04:45:16 dist-test-slave-bqcl krb5kdc[25825](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786337116, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 04:45:16.260931 25848 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260810 04:45:16.261427 25848 webserver.cc:460] Webserver started at http://127.0.0.1:38069/ using document root <none> and password file <none>
I20260810 04:45:16.261637 25848 fs_manager.cc:359] Metadata directory not provided
I20260810 04:45:16.261688 25848 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 04:45:16.261823 25848 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 04:45:16.262931 25848 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786336863388938-23197-0/minicluster-data/master-0/data/instance:
uuid: "25180468c4bf4bbfb9c8854e6d7e71c8"
format_stamp: "Formatted at 2026-08-10 04:45:16 on dist-test-slave-bqcl"
server_key: "7390d16051ab08fcea674deb8d2de80d"
server_key_iv: "b4005465b103a0cef70a3c4d6a6b868f"
server_key_version: "encryptionkey@0"
I20260810 04:45:16.263294 25848 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786336863388938-23197-0/minicluster-data/master-0/wal/instance:
uuid: "25180468c4bf4bbfb9c8854e6d7e71c8"
format_stamp: "Formatted at 2026-08-10 04:45:16 on dist-test-slave-bqcl"
server_key: "7390d16051ab08fcea674deb8d2de80d"
server_key_iv: "b4005465b103a0cef70a3c4d6a6b868f"
server_key_version: "encryptionkey@0"
I20260810 04:45:16.264684 25848 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260810 04:45:16.265372 25870 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 04:45:16.265545 25848 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260810 04:45:16.265630 25848 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786336863388938-23197-0/minicluster-data/master-0/data,/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786336863388938-23197-0/minicluster-data/master-0/wal
uuid: "25180468c4bf4bbfb9c8854e6d7e71c8"
format_stamp: "Formatted at 2026-08-10 04:45:16 on dist-test-slave-bqcl"
server_key: "7390d16051ab08fcea674deb8d2de80d"
server_key_iv: "b4005465b103a0cef70a3c4d6a6b868f"
server_key_version: "encryptionkey@0"
I20260810 04:45:16.265692 25848 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786336863388938-23197-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786336863388938-23197-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786336863388938-23197-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 04:45:16.288311 25848 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 04:45:16.290421 25848 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 04:45:16.290575 25848 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 04:45:16.294502 25848 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:42787
I20260810 04:45:16.294539 25962 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:42787 every 8 connection(s)
I20260810 04:45:16.294866 25848 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786336863388938-23197-0/minicluster-data/master-0/data/info.pb
I20260810 04:45:16.295693 25963 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 04:45:16.297896 25963 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 25180468c4bf4bbfb9c8854e6d7e71c8: Bootstrap starting.
I20260810 04:45:16.298484 25963 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 25180468c4bf4bbfb9c8854e6d7e71c8: Neither blocks nor log segments found. Creating new log.
I20260810 04:45:16.298728 25963 log.cc:824] T 00000000000000000000000000000000 P 25180468c4bf4bbfb9c8854e6d7e71c8: Log is configured to *not* fsync() on all Append() calls
I20260810 04:45:16.299360 25963 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 25180468c4bf4bbfb9c8854e6d7e71c8: No bootstrap required, opened a new log
I20260810 04:45:16.300117 23197 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskemv_AM/build/release/bin/kudu as pid 25848
I20260810 04:45:16.300231 23197 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786336863388938-23197-0/minicluster-data/master-0/wal/instance
I20260810 04:45:16.300391 23197 external_mini_cluster.cc:1428] Setting key 59bafb4a7b8122d6c04d67c1a707c227
I20260810 04:45:16.301147 25963 raft_consensus.cc:348] T 00000000000000000000000000000000 P 25180468c4bf4bbfb9c8854e6d7e71c8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "25180468c4bf4bbfb9c8854e6d7e71c8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42787 } }
I20260810 04:45:16.301285 25963 raft_consensus.cc:374] T 00000000000000000000000000000000 P 25180468c4bf4bbfb9c8854e6d7e71c8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 04:45:16.301316 25963 raft_consensus.cc:729] T 00000000000000000000000000000000 P 25180468c4bf4bbfb9c8854e6d7e71c8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 25180468c4bf4bbfb9c8854e6d7e71c8, State: Initialized, Role: FOLLOWER
I20260810 04:45:16.301442 25963 consensus_queue.cc:260] T 00000000000000000000000000000000 P 25180468c4bf4bbfb9c8854e6d7e71c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "25180468c4bf4bbfb9c8854e6d7e71c8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42787 } }
I20260810 04:45:16.301510 25963 raft_consensus.cc:388] T 00000000000000000000000000000000 P 25180468c4bf4bbfb9c8854e6d7e71c8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 04:45:16.301540 25963 raft_consensus.cc:482] T 00000000000000000000000000000000 P 25180468c4bf4bbfb9c8854e6d7e71c8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 04:45:16.301585 25963 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 25180468c4bf4bbfb9c8854e6d7e71c8 [term 0 FOLLOWER]: Advancing to term 1
I20260810 04:45:16.302095 25963 raft_consensus.cc:504] T 00000000000000000000000000000000 P 25180468c4bf4bbfb9c8854e6d7e71c8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "25180468c4bf4bbfb9c8854e6d7e71c8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42787 } }
I20260810 04:45:16.302213 25963 leader_election.cc:302] T 00000000000000000000000000000000 P 25180468c4bf4bbfb9c8854e6d7e71c8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 25180468c4bf4bbfb9c8854e6d7e71c8; no voters:
I20260810 04:45:16.302368 25963 leader_election.cc:288] T 00000000000000000000000000000000 P 25180468c4bf4bbfb9c8854e6d7e71c8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 04:45:16.302407 25973 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 25180468c4bf4bbfb9c8854e6d7e71c8 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 04:45:16.302508 25973 raft_consensus.cc:686] T 00000000000000000000000000000000 P 25180468c4bf4bbfb9c8854e6d7e71c8 [term 1 LEADER]: Becoming Leader. State: Replica: 25180468c4bf4bbfb9c8854e6d7e71c8, State: Running, Role: LEADER
I20260810 04:45:16.302590 25973 consensus_queue.cc:237] T 00000000000000000000000000000000 P 25180468c4bf4bbfb9c8854e6d7e71c8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "25180468c4bf4bbfb9c8854e6d7e71c8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42787 } }
I20260810 04:45:16.302625 25963 sys_catalog.cc:564] T 00000000000000000000000000000000 P 25180468c4bf4bbfb9c8854e6d7e71c8 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 04:45:16.302953 25973 sys_catalog.cc:455] T 00000000000000000000000000000000 P 25180468c4bf4bbfb9c8854e6d7e71c8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 25180468c4bf4bbfb9c8854e6d7e71c8. Latest consensus state: current_term: 1 leader_uuid: "25180468c4bf4bbfb9c8854e6d7e71c8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "25180468c4bf4bbfb9c8854e6d7e71c8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42787 } } }
I20260810 04:45:16.303023 25973 sys_catalog.cc:458] T 00000000000000000000000000000000 P 25180468c4bf4bbfb9c8854e6d7e71c8 [sys.catalog]: This master's current role is: LEADER
I20260810 04:45:16.303249 25980 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 04:45:16.303763 25980 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 04:45:16.304306 25975 sys_catalog.cc:455] T 00000000000000000000000000000000 P 25180468c4bf4bbfb9c8854e6d7e71c8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "25180468c4bf4bbfb9c8854e6d7e71c8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "25180468c4bf4bbfb9c8854e6d7e71c8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42787 } } }
I20260810 04:45:16.304389 25975 sys_catalog.cc:458] T 00000000000000000000000000000000 P 25180468c4bf4bbfb9c8854e6d7e71c8 [sys.catalog]: This master's current role is: LEADER
I20260810 04:45:16.306176 25980 catalog_manager.cc:1347] Generated new cluster ID: 3d889d08f16f43a686f53d964cb2858f
I20260810 04:45:16.306234 25980 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 04:45:16.306452 25971 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:45:16.300869 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55879 (local address 127.0.0.1:42787)
0810 04:45:16.301037 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0810 04:45:16.301046 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 04:45:16.301072 (+ 26us) server_negotiation.cc:407] Connection header received
0810 04:45:16.301167 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:45:16.301172 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:45:16.301220 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:45:16.301346 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0810 04:45:16.301732 (+ 386us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:45:16.302930 (+ 1198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:45:16.304012 (+ 1082us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:45:16.304862 (+ 850us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:45:16.304971 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:45:16.305604 (+ 633us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 04:45:16.305607 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 04:45:16.305608 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 04:45:16.305819 (+ 211us) server_negotiation.cc:922] calling sasl_server_start()
0810 04:45:16.305850 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 04:45:16.305862 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 04:45:16.305920 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0810 04:45:16.306149 (+ 229us) server_negotiation.cc:299] Negotiation successful
0810 04:45:16.306218 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":120,"thread_start_us":68,"threads_started":1}
I20260810 04:45:16.307368 23197 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 04:45:16.311949 25971 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:45:16.308906 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44737 (local address 127.0.0.1:42787)
0810 04:45:16.308939 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0810 04:45:16.308940 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0810 04:45:16.308948 (+ 8us) server_negotiation.cc:407] Connection header received
0810 04:45:16.308966 (+ 18us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:45:16.308967 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:45:16.308976 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:45:16.309036 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0810 04:45:16.309155 (+ 119us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:45:16.309838 (+ 683us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:45:16.310799 (+ 961us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:45:16.311085 (+ 286us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:45:16.311159 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:45:16.311738 (+ 579us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 04:45:16.311740 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 04:45:16.311740 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 04:45:16.311744 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0810 04:45:16.311759 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 04:45:16.311763 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 04:45:16.311820 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0810 04:45:16.311859 (+ 39us) server_negotiation.cc:299] Negotiation successful
0810 04:45:16.311881 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260810 04:45:16.330158 25980 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 04:45:16.330677 25980 catalog_manager.cc:1504] Loading token signing keys...
I20260810 04:45:16.339740 25980 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 25180468c4bf4bbfb9c8854e6d7e71c8: Generated new TSK 0
I20260810 04:45:16.339902 25980 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 04:45:16.349308 23197 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskemv_AM/build/release/bin/kudu with pid 25848
2026-08-10T04:45:16Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/4 (1465 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/5
Loading random data
Initializing database '/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786336863388938-23197-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 04:45:16 dist-test-slave-bqcl krb5kdc[26018](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 04:45:16 dist-test-slave-bqcl krb5kdc[26018](info): set up 2 sockets
Aug 10 04:45:16 dist-test-slave-bqcl krb5kdc[26018](info): commencing operation
krb5kdc: starting...
W20260810 04:45:17.510787 23197 mini_kdc.cc:121] Time spent starting KDC: real 1.131s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786336863388938-23197-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786336863388938-23197-0/krb5kdc/test-user.keytab.
Aug 10 04:45:17 dist-test-slave-bqcl krb5kdc[26018](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786337117, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T04:45:17Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T04:45:17Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786336863388938-23197-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786336863388938-23197-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260810 04:45:17.624373 23197 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskemv_AM/build/release/bin/kudu
/tmp/dist-test-taskemv_AM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786336863388938-23197-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786336863388938-23197-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786336863388938-23197-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786336863388938-23197-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33115
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786336863388938-23197-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:45435
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:33115
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786336863388938-23197-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786336863388938-23197-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786336863388938-23197-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 04:45:17.704557 26036 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 04:45:17.704759 26036 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 04:45:17.704790 26036 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 04:45:17.705905 26036 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 04:45:17.705946 26036 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 04:45:17.705965 26036 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 04:45:17.705982 26036 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 04:45:17.705996 26036 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 04:45:17.707466 26036 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:45435
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786336863388938-23197-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786336863388938-23197-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:33115
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786336863388938-23197-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33115
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786336863388938-23197-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26036
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786336863388938-23197-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 03:54:01 UTC on ccycloud.quasar-icijmt.root.comops.site
build id 4577009
I20260810 04:45:17.707705 26036 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 04:45:17.707911 26036 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 04:45:17.712363 26045 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 04:45:17.712513 26041 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 04:45:17.712467 26050 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 04:45:17.712585 26036 server_base.cc:1034] running on GCE node
I20260810 04:45:17.712893 26036 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 04:45:17.713167 26036 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 04:45:17.714301 26036 hybrid_clock.cc:648] HybridClock initialized: now 1786337117714281 us; error 36 us; skew 500 ppm
Aug 10 04:45:17 dist-test-slave-bqcl krb5kdc[26018](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786337117, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 04:45:17.716679 26036 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260810 04:45:17.717084 26036 webserver.cc:460] Webserver started at http://127.0.0.1:32845/ using document root <none> and password file <none>
I20260810 04:45:17.717283 26036 fs_manager.cc:359] Metadata directory not provided
I20260810 04:45:17.717321 26036 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 04:45:17.717414 26036 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 04:45:17.718462 26036 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786336863388938-23197-0/minicluster-data/master-0/data/instance:
uuid: "10845d99be8f428fa039ec3c8aabc1f6"
format_stamp: "Formatted at 2026-08-10 04:45:17 on dist-test-slave-bqcl"
server_key: "7a9b208b124eb42d5793cbe1cd2747a2"
server_key_iv: "997d047f849bd177883359cce2898f89"
server_key_version: "encryptionkey@0"
I20260810 04:45:17.718741 26036 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786336863388938-23197-0/minicluster-data/master-0/wal/instance:
uuid: "10845d99be8f428fa039ec3c8aabc1f6"
format_stamp: "Formatted at 2026-08-10 04:45:17 on dist-test-slave-bqcl"
server_key: "7a9b208b124eb42d5793cbe1cd2747a2"
server_key_iv: "997d047f849bd177883359cce2898f89"
server_key_version: "encryptionkey@0"
I20260810 04:45:17.719898 26036 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260810 04:45:17.720574 26059 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 04:45:17.720739 26036 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260810 04:45:17.720865 26036 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786336863388938-23197-0/minicluster-data/master-0/data,/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786336863388938-23197-0/minicluster-data/master-0/wal
uuid: "10845d99be8f428fa039ec3c8aabc1f6"
format_stamp: "Formatted at 2026-08-10 04:45:17 on dist-test-slave-bqcl"
server_key: "7a9b208b124eb42d5793cbe1cd2747a2"
server_key_iv: "997d047f849bd177883359cce2898f89"
server_key_version: "encryptionkey@0"
I20260810 04:45:17.720927 26036 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786336863388938-23197-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786336863388938-23197-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786336863388938-23197-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 04:45:17.723989 26036 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 04:45:17.725633 26036 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 04:45:17.725734 26036 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 04:45:17.729435 26036 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:33115
I20260810 04:45:17.729481 26141 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:33115 every 8 connection(s)
I20260810 04:45:17.729758 26036 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786336863388938-23197-0/minicluster-data/master-0/data/info.pb
I20260810 04:45:17.730712 26142 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 04:45:17.732781 26142 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 10845d99be8f428fa039ec3c8aabc1f6: Bootstrap starting.
I20260810 04:45:17.733310 26142 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 10845d99be8f428fa039ec3c8aabc1f6: Neither blocks nor log segments found. Creating new log.
I20260810 04:45:17.733570 26142 log.cc:824] T 00000000000000000000000000000000 P 10845d99be8f428fa039ec3c8aabc1f6: Log is configured to *not* fsync() on all Append() calls
I20260810 04:45:17.734241 26142 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 10845d99be8f428fa039ec3c8aabc1f6: No bootstrap required, opened a new log
I20260810 04:45:17.735751 26142 raft_consensus.cc:348] T 00000000000000000000000000000000 P 10845d99be8f428fa039ec3c8aabc1f6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "10845d99be8f428fa039ec3c8aabc1f6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33115 } }
I20260810 04:45:17.735879 26142 raft_consensus.cc:374] T 00000000000000000000000000000000 P 10845d99be8f428fa039ec3c8aabc1f6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 04:45:17.735913 26142 raft_consensus.cc:729] T 00000000000000000000000000000000 P 10845d99be8f428fa039ec3c8aabc1f6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 10845d99be8f428fa039ec3c8aabc1f6, State: Initialized, Role: FOLLOWER
I20260810 04:45:17.736043 26142 consensus_queue.cc:260] T 00000000000000000000000000000000 P 10845d99be8f428fa039ec3c8aabc1f6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "10845d99be8f428fa039ec3c8aabc1f6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33115 } }
I20260810 04:45:17.736101 26142 raft_consensus.cc:388] T 00000000000000000000000000000000 P 10845d99be8f428fa039ec3c8aabc1f6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 04:45:17.736122 26142 raft_consensus.cc:482] T 00000000000000000000000000000000 P 10845d99be8f428fa039ec3c8aabc1f6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 04:45:17.736176 26142 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 10845d99be8f428fa039ec3c8aabc1f6 [term 0 FOLLOWER]: Advancing to term 1
I20260810 04:45:17.736660 26142 raft_consensus.cc:504] T 00000000000000000000000000000000 P 10845d99be8f428fa039ec3c8aabc1f6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "10845d99be8f428fa039ec3c8aabc1f6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33115 } }
I20260810 04:45:17.736752 26142 leader_election.cc:302] T 00000000000000000000000000000000 P 10845d99be8f428fa039ec3c8aabc1f6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 10845d99be8f428fa039ec3c8aabc1f6; no voters:
I20260810 04:45:17.736905 26142 leader_election.cc:288] T 00000000000000000000000000000000 P 10845d99be8f428fa039ec3c8aabc1f6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 04:45:17.736934 26148 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 10845d99be8f428fa039ec3c8aabc1f6 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 04:45:17.737058 26148 raft_consensus.cc:686] T 00000000000000000000000000000000 P 10845d99be8f428fa039ec3c8aabc1f6 [term 1 LEADER]: Becoming Leader. State: Replica: 10845d99be8f428fa039ec3c8aabc1f6, State: Running, Role: LEADER
I20260810 04:45:17.737138 26142 sys_catalog.cc:564] T 00000000000000000000000000000000 P 10845d99be8f428fa039ec3c8aabc1f6 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 04:45:17.737176 26148 consensus_queue.cc:237] T 00000000000000000000000000000000 P 10845d99be8f428fa039ec3c8aabc1f6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "10845d99be8f428fa039ec3c8aabc1f6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33115 } }
I20260810 04:45:17.737521 26153 sys_catalog.cc:455] T 00000000000000000000000000000000 P 10845d99be8f428fa039ec3c8aabc1f6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 10845d99be8f428fa039ec3c8aabc1f6. Latest consensus state: current_term: 1 leader_uuid: "10845d99be8f428fa039ec3c8aabc1f6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "10845d99be8f428fa039ec3c8aabc1f6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33115 } } }
I20260810 04:45:17.737627 26151 sys_catalog.cc:455] T 00000000000000000000000000000000 P 10845d99be8f428fa039ec3c8aabc1f6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "10845d99be8f428fa039ec3c8aabc1f6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "10845d99be8f428fa039ec3c8aabc1f6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33115 } } }
I20260810 04:45:17.737666 26153 sys_catalog.cc:458] T 00000000000000000000000000000000 P 10845d99be8f428fa039ec3c8aabc1f6 [sys.catalog]: This master's current role is: LEADER
I20260810 04:45:17.737710 26151 sys_catalog.cc:458] T 00000000000000000000000000000000 P 10845d99be8f428fa039ec3c8aabc1f6 [sys.catalog]: This master's current role is: LEADER
I20260810 04:45:17.738246 26167 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 04:45:17.738360 23197 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskemv_AM/build/release/bin/kudu as pid 26036
I20260810 04:45:17.738480 23197 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786336863388938-23197-0/minicluster-data/master-0/wal/instance
I20260810 04:45:17.738626 23197 external_mini_cluster.cc:1428] Setting key 50b10aa138649e077db9e1cbe70d6d88
I20260810 04:45:17.738785 26167 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 04:45:17.740238 26167 catalog_manager.cc:1347] Generated new cluster ID: b2ddfe0ca7e54dcaa07885c58acaff72
I20260810 04:45:17.740280 26167 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 04:45:17.744038 26180 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:45:17.739048 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34491 (local address 127.0.0.1:33115)
0810 04:45:17.742439 (+ 3391us) server_negotiation.cc:206] Beginning negotiation
0810 04:45:17.742448 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 04:45:17.742477 (+ 29us) server_negotiation.cc:407] Connection header received
0810 04:45:17.742583 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:45:17.742589 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:45:17.742617 (+ 28us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:45:17.742818 (+ 201us) server_negotiation.cc:226] Negotiated authn=SASL
0810 04:45:17.743627 (+ 809us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 04:45:17.743630 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 04:45:17.743631 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 04:45:17.743635 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0810 04:45:17.743679 (+ 44us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 04:45:17.743692 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 04:45:17.743746 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0810 04:45:17.743820 (+ 74us) server_negotiation.cc:299] Negotiation successful
0810 04:45:17.743857 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3339,"thread_start_us":81,"threads_started":1}
I20260810 04:45:17.744937 23197 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 04:45:17.747550 26180 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:45:17.746491 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48691 (local address 127.0.0.1:33115)
0810 04:45:17.746529 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0810 04:45:17.746531 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 04:45:17.746701 (+ 170us) server_negotiation.cc:407] Connection header received
0810 04:45:17.746763 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:45:17.746764 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:45:17.746780 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:45:17.746831 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0810 04:45:17.747348 (+ 517us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 04:45:17.747349 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 04:45:17.747349 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 04:45:17.747353 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0810 04:45:17.747367 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 04:45:17.747371 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 04:45:17.747426 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0810 04:45:17.747480 (+ 54us) server_negotiation.cc:299] Negotiation successful
0810 04:45:17.747491 (+ 11us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260810 04:45:17.748734 26167 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 04:45:17.748873 26167 catalog_manager.cc:1504] Loading token signing keys...
I20260810 04:45:17.752430 26167 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 10845d99be8f428fa039ec3c8aabc1f6: Generated new TSK 0
I20260810 04:45:17.752579 26167 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 04:45:17.756317 23197 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskemv_AM/build/release/bin/kudu with pid 26036
2026-08-10T04:45:17Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/5 (1425 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/6
Loading random data
Initializing database '/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786336863388938-23197-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 04:45:17 dist-test-slave-bqcl krb5kdc[26196](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 04:45:17 dist-test-slave-bqcl krb5kdc[26196](info): set up 2 sockets
Aug 10 04:45:17 dist-test-slave-bqcl krb5kdc[26196](info): commencing operation
krb5kdc: starting...
W20260810 04:45:18.958669 23197 mini_kdc.cc:121] Time spent starting KDC: real 1.153s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786336863388938-23197-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786336863388938-23197-0/krb5kdc/test-user.keytab.
Aug 10 04:45:19 dist-test-slave-bqcl krb5kdc[26196](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786337119, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T04:45:19Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T04:45:19Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786336863388938-23197-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786336863388938-23197-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260810 04:45:19.072520 23197 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskemv_AM/build/release/bin/kudu
/tmp/dist-test-taskemv_AM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786336863388938-23197-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786336863388938-23197-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786336863388938-23197-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786336863388938-23197-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39863
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786336863388938-23197-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:44603
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:39863
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786336863388938-23197-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786336863388938-23197-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786336863388938-23197-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 04:45:19.153056 26220 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 04:45:19.153270 26220 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 04:45:19.153292 26220 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 04:45:19.154505 26220 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 04:45:19.154544 26220 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 04:45:19.154558 26220 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 04:45:19.154572 26220 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 04:45:19.154594 26220 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 04:45:19.156085 26220 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:44603
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786336863388938-23197-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786336863388938-23197-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:39863
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786336863388938-23197-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39863
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786336863388938-23197-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26220
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786336863388938-23197-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 03:54:01 UTC on ccycloud.quasar-icijmt.root.comops.site
build id 4577009
I20260810 04:45:19.156312 26220 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 04:45:19.156500 26220 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 04:45:19.161056 26240 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 04:45:19.161073 26230 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 04:45:19.161069 26233 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 04:45:19.161273 26220 server_base.cc:1034] running on GCE node
I20260810 04:45:19.161516 26220 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 04:45:19.161800 26220 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 04:45:19.162931 26220 hybrid_clock.cc:648] HybridClock initialized: now 1786337119162917 us; error 30 us; skew 500 ppm
Aug 10 04:45:19 dist-test-slave-bqcl krb5kdc[26196](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786337119, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 04:45:19.165282 26220 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260810 04:45:19.165704 26220 webserver.cc:460] Webserver started at http://127.0.0.1:42475/ using document root <none> and password file <none>
I20260810 04:45:19.165915 26220 fs_manager.cc:359] Metadata directory not provided
I20260810 04:45:19.165962 26220 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 04:45:19.166078 26220 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 04:45:19.167088 26220 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786336863388938-23197-0/minicluster-data/master-0/data/instance:
uuid: "b1a52cb187544d5a9c72b878f36ed42b"
format_stamp: "Formatted at 2026-08-10 04:45:19 on dist-test-slave-bqcl"
server_key: "6eb9e9511a3e9703b45db9ed1f72cc9b"
server_key_iv: "82bad072432a9c1e45d2fbede590a2ae"
server_key_version: "encryptionkey@0"
I20260810 04:45:19.167419 26220 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786336863388938-23197-0/minicluster-data/master-0/wal/instance:
uuid: "b1a52cb187544d5a9c72b878f36ed42b"
format_stamp: "Formatted at 2026-08-10 04:45:19 on dist-test-slave-bqcl"
server_key: "6eb9e9511a3e9703b45db9ed1f72cc9b"
server_key_iv: "82bad072432a9c1e45d2fbede590a2ae"
server_key_version: "encryptionkey@0"
I20260810 04:45:19.168673 26220 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260810 04:45:19.169381 26255 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 04:45:19.169523 26220 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260810 04:45:19.169607 26220 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786336863388938-23197-0/minicluster-data/master-0/data,/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786336863388938-23197-0/minicluster-data/master-0/wal
uuid: "b1a52cb187544d5a9c72b878f36ed42b"
format_stamp: "Formatted at 2026-08-10 04:45:19 on dist-test-slave-bqcl"
server_key: "6eb9e9511a3e9703b45db9ed1f72cc9b"
server_key_iv: "82bad072432a9c1e45d2fbede590a2ae"
server_key_version: "encryptionkey@0"
I20260810 04:45:19.169673 26220 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786336863388938-23197-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786336863388938-23197-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786336863388938-23197-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 04:45:19.186110 26220 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 04:45:19.187872 26220 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 04:45:19.188004 26220 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 04:45:19.191838 26220 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:39863
I20260810 04:45:19.191871 26348 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:39863 every 8 connection(s)
I20260810 04:45:19.192183 26220 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786336863388938-23197-0/minicluster-data/master-0/data/info.pb
I20260810 04:45:19.193010 26353 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 04:45:19.195230 26353 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b1a52cb187544d5a9c72b878f36ed42b: Bootstrap starting.
I20260810 04:45:19.195791 26353 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b1a52cb187544d5a9c72b878f36ed42b: Neither blocks nor log segments found. Creating new log.
I20260810 04:45:19.196058 26353 log.cc:824] T 00000000000000000000000000000000 P b1a52cb187544d5a9c72b878f36ed42b: Log is configured to *not* fsync() on all Append() calls
I20260810 04:45:19.196682 26353 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b1a52cb187544d5a9c72b878f36ed42b: No bootstrap required, opened a new log
I20260810 04:45:19.196776 23197 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskemv_AM/build/release/bin/kudu as pid 26220
I20260810 04:45:19.196898 23197 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786336863388938-23197-0/minicluster-data/master-0/wal/instance
I20260810 04:45:19.197063 23197 external_mini_cluster.cc:1428] Setting key 4493c37b3014bd299e7793c73558e6b1
I20260810 04:45:19.198655 26353 raft_consensus.cc:348] T 00000000000000000000000000000000 P b1a52cb187544d5a9c72b878f36ed42b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b1a52cb187544d5a9c72b878f36ed42b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39863 } }
I20260810 04:45:19.198788 26353 raft_consensus.cc:374] T 00000000000000000000000000000000 P b1a52cb187544d5a9c72b878f36ed42b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 04:45:19.198822 26353 raft_consensus.cc:729] T 00000000000000000000000000000000 P b1a52cb187544d5a9c72b878f36ed42b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b1a52cb187544d5a9c72b878f36ed42b, State: Initialized, Role: FOLLOWER
I20260810 04:45:19.198941 26353 consensus_queue.cc:260] T 00000000000000000000000000000000 P b1a52cb187544d5a9c72b878f36ed42b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b1a52cb187544d5a9c72b878f36ed42b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39863 } }
I20260810 04:45:19.199016 26353 raft_consensus.cc:388] T 00000000000000000000000000000000 P b1a52cb187544d5a9c72b878f36ed42b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 04:45:19.199060 26353 raft_consensus.cc:482] T 00000000000000000000000000000000 P b1a52cb187544d5a9c72b878f36ed42b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 04:45:19.199154 26353 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b1a52cb187544d5a9c72b878f36ed42b [term 0 FOLLOWER]: Advancing to term 1
I20260810 04:45:19.199819 26353 raft_consensus.cc:504] T 00000000000000000000000000000000 P b1a52cb187544d5a9c72b878f36ed42b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b1a52cb187544d5a9c72b878f36ed42b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39863 } }
I20260810 04:45:19.199937 26353 leader_election.cc:302] T 00000000000000000000000000000000 P b1a52cb187544d5a9c72b878f36ed42b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b1a52cb187544d5a9c72b878f36ed42b; no voters:
I20260810 04:45:19.200107 26353 leader_election.cc:288] T 00000000000000000000000000000000 P b1a52cb187544d5a9c72b878f36ed42b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 04:45:19.200158 26366 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b1a52cb187544d5a9c72b878f36ed42b [term 1 FOLLOWER]: Leader election won for term 1
I20260810 04:45:19.200353 26353 sys_catalog.cc:564] T 00000000000000000000000000000000 P b1a52cb187544d5a9c72b878f36ed42b [sys.catalog]: configured and running, proceeding with master startup.
I20260810 04:45:19.200367 26366 raft_consensus.cc:686] T 00000000000000000000000000000000 P b1a52cb187544d5a9c72b878f36ed42b [term 1 LEADER]: Becoming Leader. State: Replica: b1a52cb187544d5a9c72b878f36ed42b, State: Running, Role: LEADER
I20260810 04:45:19.200525 26366 consensus_queue.cc:237] T 00000000000000000000000000000000 P b1a52cb187544d5a9c72b878f36ed42b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b1a52cb187544d5a9c72b878f36ed42b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39863 } }
I20260810 04:45:19.200878 26368 sys_catalog.cc:455] T 00000000000000000000000000000000 P b1a52cb187544d5a9c72b878f36ed42b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b1a52cb187544d5a9c72b878f36ed42b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b1a52cb187544d5a9c72b878f36ed42b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39863 } } }
I20260810 04:45:19.200948 26368 sys_catalog.cc:458] T 00000000000000000000000000000000 P b1a52cb187544d5a9c72b878f36ed42b [sys.catalog]: This master's current role is: LEADER
I20260810 04:45:19.201138 26369 sys_catalog.cc:455] T 00000000000000000000000000000000 P b1a52cb187544d5a9c72b878f36ed42b [sys.catalog]: SysCatalogTable state changed. Reason: New leader b1a52cb187544d5a9c72b878f36ed42b. Latest consensus state: current_term: 1 leader_uuid: "b1a52cb187544d5a9c72b878f36ed42b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b1a52cb187544d5a9c72b878f36ed42b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39863 } } }
I20260810 04:45:19.201246 26369 sys_catalog.cc:458] T 00000000000000000000000000000000 P b1a52cb187544d5a9c72b878f36ed42b [sys.catalog]: This master's current role is: LEADER
I20260810 04:45:19.201646 26380 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 04:45:19.202176 26380 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 04:45:19.203707 26380 catalog_manager.cc:1347] Generated new cluster ID: 7891c1368d4d46c4ad62502ea1b6e8aa
I20260810 04:45:19.203765 26380 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 10 04:45:19 dist-test-slave-bqcl krb5kdc[26196](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786337119, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260810 04:45:19.211091 26362 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:45:19.197508 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.178:48049 (local address 127.0.0.1:39863)
0810 04:45:19.197665 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0810 04:45:19.197672 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 04:45:19.197698 (+ 26us) server_negotiation.cc:407] Connection header received
0810 04:45:19.198829 (+ 1131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:45:19.198838 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:45:19.198906 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:45:19.199052 (+ 146us) server_negotiation.cc:226] Negotiated authn=SASL
0810 04:45:19.199804 (+ 752us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:45:19.201126 (+ 1322us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:45:19.202190 (+ 1064us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:45:19.202824 (+ 634us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:45:19.202929 (+ 105us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:45:19.208114 (+ 5185us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 04:45:19.208118 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 04:45:19.208118 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 04:45:19.208129 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0810 04:45:19.209610 (+ 1481us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 04:45:19.209972 (+ 362us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 04:45:19.209974 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 04:45:19.209974 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 04:45:19.210080 (+ 106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 04:45:19.210303 (+ 223us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 04:45:19.210304 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 04:45:19.210304 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 04:45:19.210521 (+ 217us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 04:45:19.210664 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0810 04:45:19.210804 (+ 140us) server_negotiation.cc:299] Negotiation successful
0810 04:45:19.210899 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":103,"thread_start_us":66,"threads_started":1}
I20260810 04:45:19.211970 23197 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 04:45:19.218700 26362 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:45:19.213208 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.178:44173 (local address 127.0.0.1:39863)
0810 04:45:19.213242 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0810 04:45:19.213243 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0810 04:45:19.213377 (+ 134us) server_negotiation.cc:407] Connection header received
0810 04:45:19.213427 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:45:19.213428 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:45:19.213437 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:45:19.213481 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0810 04:45:19.213937 (+ 456us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:45:19.214484 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:45:19.215135 (+ 651us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:45:19.215354 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:45:19.215407 (+ 53us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:45:19.216440 (+ 1033us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 04:45:19.216442 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 04:45:19.216442 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 04:45:19.216448 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0810 04:45:19.217633 (+ 1185us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 04:45:19.217958 (+ 325us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 04:45:19.217959 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 04:45:19.217959 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 04:45:19.218009 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 04:45:19.218233 (+ 224us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 04:45:19.218234 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 04:45:19.218235 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 04:45:19.218340 (+ 105us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 04:45:19.218429 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0810 04:45:19.218555 (+ 126us) server_negotiation.cc:299] Negotiation successful
0810 04:45:19.218611 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260810 04:45:19.223388 26380 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 04:45:19.224063 26380 catalog_manager.cc:1504] Loading token signing keys...
I20260810 04:45:19.232910 26380 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b1a52cb187544d5a9c72b878f36ed42b: Generated new TSK 0
I20260810 04:45:19.233081 26380 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 04:45:19.240309 23197 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskemv_AM/build/release/bin/kudu with pid 26220
W20260810 04:45:19.259979 26191 connection.cc:535] client connection to 127.0.0.1:39863 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:39863): Connection reset by peer (error 104)
2026-08-10T04:45:19Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/6 (1467 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/7
Loading random data
Initializing database '/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786336863388938-23197-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 04:45:19 dist-test-slave-bqcl krb5kdc[26408](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 04:45:19 dist-test-slave-bqcl krb5kdc[26408](info): set up 2 sockets
Aug 10 04:45:19 dist-test-slave-bqcl krb5kdc[26408](info): commencing operation
krb5kdc: starting...
W20260810 04:45:20.398241 23197 mini_kdc.cc:121] Time spent starting KDC: real 1.125s user 0.002s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786336863388938-23197-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786336863388938-23197-0/krb5kdc/test-user.keytab.
Aug 10 04:45:20 dist-test-slave-bqcl krb5kdc[26408](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786337120, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T04:45:20Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T04:45:20Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786336863388938-23197-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786336863388938-23197-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260810 04:45:20.509691 23197 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskemv_AM/build/release/bin/kudu
/tmp/dist-test-taskemv_AM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786336863388938-23197-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786336863388938-23197-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786336863388938-23197-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786336863388938-23197-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44701
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786336863388938-23197-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:39261
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:44701
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786336863388938-23197-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786336863388938-23197-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786336863388938-23197-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 04:45:20.587630 26425 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 04:45:20.587826 26425 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 04:45:20.587847 26425 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 04:45:20.588905 26425 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 04:45:20.588940 26425 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 04:45:20.588953 26425 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 04:45:20.588963 26425 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 04:45:20.588973 26425 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 04:45:20.590379 26425 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:39261
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786336863388938-23197-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786336863388938-23197-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:44701
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786336863388938-23197-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44701
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786336863388938-23197-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26425
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786336863388938-23197-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 03:54:01 UTC on ccycloud.quasar-icijmt.root.comops.site
build id 4577009
I20260810 04:45:20.590564 26425 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 04:45:20.590732 26425 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 04:45:20.595120 26433 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 04:45:20.595134 26432 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 04:45:20.595182 26436 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 04:45:20.595355 26425 server_base.cc:1034] running on GCE node
I20260810 04:45:20.595584 26425 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 04:45:20.595860 26425 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 04:45:20.596997 26425 hybrid_clock.cc:648] HybridClock initialized: now 1786337120596977 us; error 33 us; skew 500 ppm
Aug 10 04:45:20 dist-test-slave-bqcl krb5kdc[26408](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786337120, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 04:45:20.599402 26425 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260810 04:45:20.599821 26425 webserver.cc:460] Webserver started at http://127.0.0.1:35291/ using document root <none> and password file <none>
I20260810 04:45:20.600026 26425 fs_manager.cc:359] Metadata directory not provided
I20260810 04:45:20.600073 26425 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 04:45:20.600207 26425 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 04:45:20.601173 26425 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786336863388938-23197-0/minicluster-data/master-0/data/instance:
uuid: "b17942dbdc8b4b5fafddb13012dd88d5"
format_stamp: "Formatted at 2026-08-10 04:45:20 on dist-test-slave-bqcl"
server_key: "f695a50d10839009a20ed6328e5bac03"
server_key_iv: "a031e3d808f754ffdf123b177d655db6"
server_key_version: "encryptionkey@0"
I20260810 04:45:20.601497 26425 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786336863388938-23197-0/minicluster-data/master-0/wal/instance:
uuid: "b17942dbdc8b4b5fafddb13012dd88d5"
format_stamp: "Formatted at 2026-08-10 04:45:20 on dist-test-slave-bqcl"
server_key: "f695a50d10839009a20ed6328e5bac03"
server_key_iv: "a031e3d808f754ffdf123b177d655db6"
server_key_version: "encryptionkey@0"
I20260810 04:45:20.602777 26425 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260810 04:45:20.603531 26452 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 04:45:20.603689 26425 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 04:45:20.603772 26425 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786336863388938-23197-0/minicluster-data/master-0/data,/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786336863388938-23197-0/minicluster-data/master-0/wal
uuid: "b17942dbdc8b4b5fafddb13012dd88d5"
format_stamp: "Formatted at 2026-08-10 04:45:20 on dist-test-slave-bqcl"
server_key: "f695a50d10839009a20ed6328e5bac03"
server_key_iv: "a031e3d808f754ffdf123b177d655db6"
server_key_version: "encryptionkey@0"
I20260810 04:45:20.603830 26425 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786336863388938-23197-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786336863388938-23197-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786336863388938-23197-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 04:45:20.614512 26425 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 04:45:20.616430 26425 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 04:45:20.616556 26425 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 04:45:20.620296 26425 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:44701
I20260810 04:45:20.620338 26548 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:44701 every 8 connection(s)
I20260810 04:45:20.620637 26425 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786336863388938-23197-0/minicluster-data/master-0/data/info.pb
I20260810 04:45:20.621423 26549 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 04:45:20.623677 23197 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskemv_AM/build/release/bin/kudu as pid 26425
I20260810 04:45:20.623685 26549 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b17942dbdc8b4b5fafddb13012dd88d5: Bootstrap starting.
I20260810 04:45:20.623801 23197 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786336863388938-23197-0/minicluster-data/master-0/wal/instance
I20260810 04:45:20.623978 23197 external_mini_cluster.cc:1428] Setting key dcbf8f273aa9ba238824fc18a4718629
I20260810 04:45:20.624343 26549 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b17942dbdc8b4b5fafddb13012dd88d5: Neither blocks nor log segments found. Creating new log.
I20260810 04:45:20.624672 26549 log.cc:824] T 00000000000000000000000000000000 P b17942dbdc8b4b5fafddb13012dd88d5: Log is configured to *not* fsync() on all Append() calls
I20260810 04:45:20.625403 26549 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b17942dbdc8b4b5fafddb13012dd88d5: No bootstrap required, opened a new log
I20260810 04:45:20.627635 26549 raft_consensus.cc:348] T 00000000000000000000000000000000 P b17942dbdc8b4b5fafddb13012dd88d5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b17942dbdc8b4b5fafddb13012dd88d5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44701 } }
I20260810 04:45:20.627764 26549 raft_consensus.cc:374] T 00000000000000000000000000000000 P b17942dbdc8b4b5fafddb13012dd88d5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 04:45:20.627784 26549 raft_consensus.cc:729] T 00000000000000000000000000000000 P b17942dbdc8b4b5fafddb13012dd88d5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b17942dbdc8b4b5fafddb13012dd88d5, State: Initialized, Role: FOLLOWER
I20260810 04:45:20.627895 26549 consensus_queue.cc:260] T 00000000000000000000000000000000 P b17942dbdc8b4b5fafddb13012dd88d5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b17942dbdc8b4b5fafddb13012dd88d5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44701 } }
I20260810 04:45:20.627969 26549 raft_consensus.cc:388] T 00000000000000000000000000000000 P b17942dbdc8b4b5fafddb13012dd88d5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 04:45:20.627996 26549 raft_consensus.cc:482] T 00000000000000000000000000000000 P b17942dbdc8b4b5fafddb13012dd88d5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 04:45:20.628044 26549 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b17942dbdc8b4b5fafddb13012dd88d5 [term 0 FOLLOWER]: Advancing to term 1
I20260810 04:45:20.628713 26549 raft_consensus.cc:504] T 00000000000000000000000000000000 P b17942dbdc8b4b5fafddb13012dd88d5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b17942dbdc8b4b5fafddb13012dd88d5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44701 } }
I20260810 04:45:20.628806 26549 leader_election.cc:302] T 00000000000000000000000000000000 P b17942dbdc8b4b5fafddb13012dd88d5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b17942dbdc8b4b5fafddb13012dd88d5; no voters:
I20260810 04:45:20.628988 26549 leader_election.cc:288] T 00000000000000000000000000000000 P b17942dbdc8b4b5fafddb13012dd88d5 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 04:45:20.629040 26554 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b17942dbdc8b4b5fafddb13012dd88d5 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 04:45:20.629192 26554 raft_consensus.cc:686] T 00000000000000000000000000000000 P b17942dbdc8b4b5fafddb13012dd88d5 [term 1 LEADER]: Becoming Leader. State: Replica: b17942dbdc8b4b5fafddb13012dd88d5, State: Running, Role: LEADER
I20260810 04:45:20.629243 26549 sys_catalog.cc:564] T 00000000000000000000000000000000 P b17942dbdc8b4b5fafddb13012dd88d5 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 04:45:20.629382 26554 consensus_queue.cc:237] T 00000000000000000000000000000000 P b17942dbdc8b4b5fafddb13012dd88d5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b17942dbdc8b4b5fafddb13012dd88d5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44701 } }
I20260810 04:45:20.629760 26556 sys_catalog.cc:455] T 00000000000000000000000000000000 P b17942dbdc8b4b5fafddb13012dd88d5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b17942dbdc8b4b5fafddb13012dd88d5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b17942dbdc8b4b5fafddb13012dd88d5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44701 } } }
I20260810 04:45:20.629837 26556 sys_catalog.cc:458] T 00000000000000000000000000000000 P b17942dbdc8b4b5fafddb13012dd88d5 [sys.catalog]: This master's current role is: LEADER
I20260810 04:45:20.629841 26552 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:45:20.624458 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.178:57617 (local address 127.0.0.1:44701)
0810 04:45:20.624636 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0810 04:45:20.624645 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 04:45:20.624678 (+ 33us) server_negotiation.cc:407] Connection header received
0810 04:45:20.625175 (+ 497us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:45:20.625184 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:45:20.625263 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:45:20.625411 (+ 148us) server_negotiation.cc:226] Negotiated authn=SASL
0810 04:45:20.625622 (+ 211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:45:20.627077 (+ 1455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:45:20.627841 (+ 764us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:45:20.628421 (+ 580us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:45:20.628542 (+ 121us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:45:20.629135 (+ 593us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 04:45:20.629140 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 04:45:20.629140 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 04:45:20.629346 (+ 206us) server_negotiation.cc:922] calling sasl_server_start()
0810 04:45:20.629384 (+ 38us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 04:45:20.629399 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 04:45:20.629479 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0810 04:45:20.629611 (+ 132us) server_negotiation.cc:299] Negotiation successful
0810 04:45:20.629640 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":125,"thread_start_us":73,"threads_started":1}
I20260810 04:45:20.630424 26557 sys_catalog.cc:455] T 00000000000000000000000000000000 P b17942dbdc8b4b5fafddb13012dd88d5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b17942dbdc8b4b5fafddb13012dd88d5. Latest consensus state: current_term: 1 leader_uuid: "b17942dbdc8b4b5fafddb13012dd88d5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b17942dbdc8b4b5fafddb13012dd88d5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44701 } } }
I20260810 04:45:20.630548 26564 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 04:45:20.630563 26557 sys_catalog.cc:458] T 00000000000000000000000000000000 P b17942dbdc8b4b5fafddb13012dd88d5 [sys.catalog]: This master's current role is: LEADER
I20260810 04:45:20.631035 26564 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 04:45:20.631201 23197 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 04:45:20.635401 26552 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:45:20.632602 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.178:33949 (local address 127.0.0.1:44701)
0810 04:45:20.632641 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0810 04:45:20.632642 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0810 04:45:20.632734 (+ 92us) server_negotiation.cc:407] Connection header received
0810 04:45:20.632794 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:45:20.632794 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:45:20.632802 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:45:20.632845 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0810 04:45:20.632976 (+ 131us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:45:20.633418 (+ 442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:45:20.634419 (+ 1001us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:45:20.634625 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:45:20.634674 (+ 49us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:45:20.635207 (+ 533us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 04:45:20.635208 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 04:45:20.635208 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 04:45:20.635212 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0810 04:45:20.635223 (+ 11us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 04:45:20.635227 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 04:45:20.635267 (+ 40us) server_negotiation.cc:1035] Waiting for connection context
0810 04:45:20.635332 (+ 65us) server_negotiation.cc:299] Negotiation successful
0810 04:45:20.635340 (+ 8us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260810 04:45:20.638866 26564 catalog_manager.cc:1347] Generated new cluster ID: 69ce581f531c441e9c5a6b22418c464e
I20260810 04:45:20.638922 26564 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 04:45:20.671314 26564 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 04:45:20.672062 26564 catalog_manager.cc:1504] Loading token signing keys...
I20260810 04:45:20.674685 26564 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b17942dbdc8b4b5fafddb13012dd88d5: Generated new TSK 0
I20260810 04:45:20.674853 26564 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 04:45:20.677906 23197 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskemv_AM/build/release/bin/kudu with pid 26425
W20260810 04:45:20.702944 26406 connection.cc:535] client connection to 127.0.0.1:44701 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:44701): Connection reset by peer (error 104)
2026-08-10T04:45:20Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/7 (1442 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/8
Loading random data
Initializing database '/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786336863388938-23197-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 04:45:20 dist-test-slave-bqcl krb5kdc[26613](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 04:45:20 dist-test-slave-bqcl krb5kdc[26613](info): set up 2 sockets
Aug 10 04:45:20 dist-test-slave-bqcl krb5kdc[26613](info): commencing operation
krb5kdc: starting...
W20260810 04:45:21.859458 23197 mini_kdc.cc:121] Time spent starting KDC: real 1.143s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786336863388938-23197-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786336863388938-23197-0/krb5kdc/test-user.keytab.
Aug 10 04:45:21 dist-test-slave-bqcl krb5kdc[26613](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786337121, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T04:45:21Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T04:45:21Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786336863388938-23197-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786336863388938-23197-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260810 04:45:21.973156 23197 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskemv_AM/build/release/bin/kudu
/tmp/dist-test-taskemv_AM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786336863388938-23197-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786336863388938-23197-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786336863388938-23197-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786336863388938-23197-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44699
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786336863388938-23197-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:38929
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:44699
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786336863388938-23197-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786336863388938-23197-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786336863388938-23197-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 04:45:22.052172 26630 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 04:45:22.052361 26630 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 04:45:22.052390 26630 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 04:45:22.053498 26630 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 04:45:22.053542 26630 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 04:45:22.053562 26630 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 04:45:22.053578 26630 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 04:45:22.053593 26630 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 04:45:22.055038 26630 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:38929
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786336863388938-23197-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786336863388938-23197-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:44699
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786336863388938-23197-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44699
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786336863388938-23197-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26630
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786336863388938-23197-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 03:54:01 UTC on ccycloud.quasar-icijmt.root.comops.site
build id 4577009
I20260810 04:45:22.055271 26630 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 04:45:22.055472 26630 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 04:45:22.059948 26639 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 04:45:22.059983 26642 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 04:45:22.060075 26630 server_base.cc:1034] running on GCE node
W20260810 04:45:22.060034 26638 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 04:45:22.060366 26630 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 04:45:22.060629 26630 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 04:45:22.061766 26630 hybrid_clock.cc:648] HybridClock initialized: now 1786337122061759 us; error 28 us; skew 500 ppm
Aug 10 04:45:22 dist-test-slave-bqcl krb5kdc[26613](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786337122, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 04:45:22.066398 26630 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260810 04:45:22.066767 26630 webserver.cc:460] Webserver started at http://127.0.0.1:42349/ using document root <none> and password file <none>
I20260810 04:45:22.066951 26630 fs_manager.cc:359] Metadata directory not provided
I20260810 04:45:22.066991 26630 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 04:45:22.067080 26630 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 04:45:22.068010 26630 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786336863388938-23197-0/minicluster-data/master-0/data/instance:
uuid: "4e6b46926910485c894a21c27ea53171"
format_stamp: "Formatted at 2026-08-10 04:45:22 on dist-test-slave-bqcl"
server_key: "9977c46854d52e4f669bc3c496fd6da4"
server_key_iv: "dcb78a7b70b8183458f5acc4143ead8d"
server_key_version: "encryptionkey@0"
I20260810 04:45:22.068292 26630 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786336863388938-23197-0/minicluster-data/master-0/wal/instance:
uuid: "4e6b46926910485c894a21c27ea53171"
format_stamp: "Formatted at 2026-08-10 04:45:22 on dist-test-slave-bqcl"
server_key: "9977c46854d52e4f669bc3c496fd6da4"
server_key_iv: "dcb78a7b70b8183458f5acc4143ead8d"
server_key_version: "encryptionkey@0"
I20260810 04:45:22.069429 26630 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260810 04:45:22.070029 26651 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 04:45:22.070187 26630 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260810 04:45:22.070256 26630 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786336863388938-23197-0/minicluster-data/master-0/data,/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786336863388938-23197-0/minicluster-data/master-0/wal
uuid: "4e6b46926910485c894a21c27ea53171"
format_stamp: "Formatted at 2026-08-10 04:45:22 on dist-test-slave-bqcl"
server_key: "9977c46854d52e4f669bc3c496fd6da4"
server_key_iv: "dcb78a7b70b8183458f5acc4143ead8d"
server_key_version: "encryptionkey@0"
I20260810 04:45:22.070322 26630 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786336863388938-23197-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786336863388938-23197-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786336863388938-23197-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 04:45:22.073352 26630 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 04:45:22.075083 26630 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 04:45:22.075196 26630 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 04:45:22.078939 26630 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:44699
I20260810 04:45:22.078970 26736 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:44699 every 8 connection(s)
I20260810 04:45:22.079252 26630 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786336863388938-23197-0/minicluster-data/master-0/data/info.pb
I20260810 04:45:22.080138 26738 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 04:45:22.082253 26738 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4e6b46926910485c894a21c27ea53171: Bootstrap starting.
I20260810 04:45:22.082758 26738 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4e6b46926910485c894a21c27ea53171: Neither blocks nor log segments found. Creating new log.
I20260810 04:45:22.082998 26738 log.cc:824] T 00000000000000000000000000000000 P 4e6b46926910485c894a21c27ea53171: Log is configured to *not* fsync() on all Append() calls
I20260810 04:45:22.083621 26738 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4e6b46926910485c894a21c27ea53171: No bootstrap required, opened a new log
I20260810 04:45:22.085114 26738 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4e6b46926910485c894a21c27ea53171 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4e6b46926910485c894a21c27ea53171" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44699 } }
I20260810 04:45:22.085258 26738 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4e6b46926910485c894a21c27ea53171 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 04:45:22.085285 26738 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4e6b46926910485c894a21c27ea53171 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4e6b46926910485c894a21c27ea53171, State: Initialized, Role: FOLLOWER
I20260810 04:45:22.085378 26738 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4e6b46926910485c894a21c27ea53171 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4e6b46926910485c894a21c27ea53171" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44699 } }
I20260810 04:45:22.085426 26738 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4e6b46926910485c894a21c27ea53171 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 04:45:22.085444 26738 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4e6b46926910485c894a21c27ea53171 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 04:45:22.085474 26738 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4e6b46926910485c894a21c27ea53171 [term 0 FOLLOWER]: Advancing to term 1
I20260810 04:45:22.085920 26738 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4e6b46926910485c894a21c27ea53171 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4e6b46926910485c894a21c27ea53171" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44699 } }
I20260810 04:45:22.086004 26738 leader_election.cc:302] T 00000000000000000000000000000000 P 4e6b46926910485c894a21c27ea53171 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4e6b46926910485c894a21c27ea53171; no voters:
I20260810 04:45:22.086115 26738 leader_election.cc:288] T 00000000000000000000000000000000 P 4e6b46926910485c894a21c27ea53171 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 04:45:22.086172 26744 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4e6b46926910485c894a21c27ea53171 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 04:45:22.086323 26744 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4e6b46926910485c894a21c27ea53171 [term 1 LEADER]: Becoming Leader. State: Replica: 4e6b46926910485c894a21c27ea53171, State: Running, Role: LEADER
I20260810 04:45:22.086376 26738 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4e6b46926910485c894a21c27ea53171 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 04:45:22.086431 26744 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4e6b46926910485c894a21c27ea53171 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4e6b46926910485c894a21c27ea53171" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44699 } }
I20260810 04:45:22.086809 26748 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4e6b46926910485c894a21c27ea53171 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4e6b46926910485c894a21c27ea53171" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4e6b46926910485c894a21c27ea53171" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44699 } } }
I20260810 04:45:22.086947 26748 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4e6b46926910485c894a21c27ea53171 [sys.catalog]: This master's current role is: LEADER
I20260810 04:45:22.086819 26751 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4e6b46926910485c894a21c27ea53171 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4e6b46926910485c894a21c27ea53171. Latest consensus state: current_term: 1 leader_uuid: "4e6b46926910485c894a21c27ea53171" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4e6b46926910485c894a21c27ea53171" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44699 } } }
I20260810 04:45:22.087128 26751 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4e6b46926910485c894a21c27ea53171 [sys.catalog]: This master's current role is: LEADER
I20260810 04:45:22.087314 23197 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskemv_AM/build/release/bin/kudu as pid 26630
I20260810 04:45:22.087435 23197 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786336863388938-23197-0/minicluster-data/master-0/wal/instance
I20260810 04:45:22.087584 23197 external_mini_cluster.cc:1428] Setting key b35dee427eff04654cb1e9eebcd7478e
I20260810 04:45:22.087591 26758 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 04:45:22.088096 26758 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 04:45:22.089689 26758 catalog_manager.cc:1347] Generated new cluster ID: 56cfa4be5b8d478b944c095f02a6ccb2
I20260810 04:45:22.089741 26758 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 04:45:22.090050 26773 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:45:22.087988 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.178:60547 (local address 127.0.0.1:44699)
0810 04:45:22.088189 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0810 04:45:22.088198 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 04:45:22.088245 (+ 47us) server_negotiation.cc:407] Connection header received
0810 04:45:22.088360 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:45:22.088368 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:45:22.088411 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:45:22.088537 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0810 04:45:22.089463 (+ 926us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 04:45:22.089467 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 04:45:22.089467 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 04:45:22.089471 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0810 04:45:22.089512 (+ 41us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 04:45:22.089522 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 04:45:22.089574 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0810 04:45:22.089847 (+ 273us) server_negotiation.cc:299] Negotiation successful
0810 04:45:22.089890 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":157,"thread_start_us":84,"threads_started":1}
I20260810 04:45:22.090905 23197 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 04:45:22.093329 26773 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:45:22.092298 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.178:32939 (local address 127.0.0.1:44699)
0810 04:45:22.092326 (+ 28us) server_negotiation.cc:206] Beginning negotiation
0810 04:45:22.092327 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0810 04:45:22.092455 (+ 128us) server_negotiation.cc:407] Connection header received
0810 04:45:22.092511 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:45:22.092512 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:45:22.092517 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:45:22.092581 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0810 04:45:22.093129 (+ 548us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 04:45:22.093130 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 04:45:22.093131 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 04:45:22.093135 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0810 04:45:22.093152 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 04:45:22.093156 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 04:45:22.093217 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0810 04:45:22.093259 (+ 42us) server_negotiation.cc:299] Negotiation successful
0810 04:45:22.093271 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13}
I20260810 04:45:22.097800 26758 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 04:45:22.097929 26758 catalog_manager.cc:1504] Loading token signing keys...
I20260810 04:45:22.107319 26758 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4e6b46926910485c894a21c27ea53171: Generated new TSK 0
I20260810 04:45:22.107457 26758 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 04:45:22.111296 23197 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskemv_AM/build/release/bin/kudu with pid 26630
2026-08-10T04:45:22Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/8 (1421 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/9
Loading random data
Initializing database '/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786336863388938-23197-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 04:45:22 dist-test-slave-bqcl krb5kdc[26799](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 04:45:22 dist-test-slave-bqcl krb5kdc[26799](info): set up 2 sockets
Aug 10 04:45:22 dist-test-slave-bqcl krb5kdc[26799](info): commencing operation
krb5kdc: starting...
W20260810 04:45:23.263989 23197 mini_kdc.cc:121] Time spent starting KDC: real 1.126s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786336863388938-23197-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786336863388938-23197-0/krb5kdc/test-user.keytab.
Aug 10 04:45:23 dist-test-slave-bqcl krb5kdc[26799](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786337123, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T04:45:23Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T04:45:23Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786336863388938-23197-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786336863388938-23197-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260810 04:45:23.379076 23197 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskemv_AM/build/release/bin/kudu
/tmp/dist-test-taskemv_AM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786336863388938-23197-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786336863388938-23197-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786336863388938-23197-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786336863388938-23197-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39479
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786336863388938-23197-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:45521
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:39479
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786336863388938-23197-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786336863388938-23197-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786336863388938-23197-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 04:45:23.459311 26821 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 04:45:23.459510 26821 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 04:45:23.459542 26821 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 04:45:23.460649 26821 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 04:45:23.460698 26821 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 04:45:23.460717 26821 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 04:45:23.460733 26821 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 04:45:23.460748 26821 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 04:45:23.462196 26821 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:45521
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786336863388938-23197-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786336863388938-23197-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:39479
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786336863388938-23197-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39479
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786336863388938-23197-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26821
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786336863388938-23197-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 03:54:01 UTC on ccycloud.quasar-icijmt.root.comops.site
build id 4577009
I20260810 04:45:23.462431 26821 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 04:45:23.462633 26821 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 04:45:23.467195 26829 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 04:45:23.467229 26828 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 04:45:23.467334 26821 server_base.cc:1034] running on GCE node
W20260810 04:45:23.467195 26832 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 04:45:23.467690 26821 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 04:45:23.467957 26821 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 04:45:23.469115 26821 hybrid_clock.cc:648] HybridClock initialized: now 1786337123469100 us; error 29 us; skew 500 ppm
Aug 10 04:45:23 dist-test-slave-bqcl krb5kdc[26799](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786337123, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 04:45:23.471685 26821 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260810 04:45:23.472138 26821 webserver.cc:460] Webserver started at http://127.0.0.1:38659/ using document root <none> and password file <none>
I20260810 04:45:23.472370 26821 fs_manager.cc:359] Metadata directory not provided
I20260810 04:45:23.472420 26821 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 04:45:23.472570 26821 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 04:45:23.473701 26821 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786336863388938-23197-0/minicluster-data/master-0/data/instance:
uuid: "a56dcd9698014f0c99a4dbe288e06859"
format_stamp: "Formatted at 2026-08-10 04:45:23 on dist-test-slave-bqcl"
server_key: "72c4ee41eb2e6cf24434944124916b1a"
server_key_iv: "dfb2636d905eebb1283a942e037df486"
server_key_version: "encryptionkey@0"
I20260810 04:45:23.474068 26821 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786336863388938-23197-0/minicluster-data/master-0/wal/instance:
uuid: "a56dcd9698014f0c99a4dbe288e06859"
format_stamp: "Formatted at 2026-08-10 04:45:23 on dist-test-slave-bqcl"
server_key: "72c4ee41eb2e6cf24434944124916b1a"
server_key_iv: "dfb2636d905eebb1283a942e037df486"
server_key_version: "encryptionkey@0"
I20260810 04:45:23.475448 26821 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260810 04:45:23.476167 26843 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 04:45:23.476336 26821 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260810 04:45:23.476413 26821 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786336863388938-23197-0/minicluster-data/master-0/data,/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786336863388938-23197-0/minicluster-data/master-0/wal
uuid: "a56dcd9698014f0c99a4dbe288e06859"
format_stamp: "Formatted at 2026-08-10 04:45:23 on dist-test-slave-bqcl"
server_key: "72c4ee41eb2e6cf24434944124916b1a"
server_key_iv: "dfb2636d905eebb1283a942e037df486"
server_key_version: "encryptionkey@0"
I20260810 04:45:23.476468 26821 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786336863388938-23197-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786336863388938-23197-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786336863388938-23197-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 04:45:23.490530 26821 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 04:45:23.492578 26821 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 04:45:23.492697 26821 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 04:45:23.496534 26821 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:39479
I20260810 04:45:23.496644 26931 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:39479 every 8 connection(s)
I20260810 04:45:23.496908 26821 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786336863388938-23197-0/minicluster-data/master-0/data/info.pb
I20260810 04:45:23.497816 26932 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 04:45:23.500279 26932 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a56dcd9698014f0c99a4dbe288e06859: Bootstrap starting.
I20260810 04:45:23.500881 26932 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a56dcd9698014f0c99a4dbe288e06859: Neither blocks nor log segments found. Creating new log.
I20260810 04:45:23.501184 26932 log.cc:824] T 00000000000000000000000000000000 P a56dcd9698014f0c99a4dbe288e06859: Log is configured to *not* fsync() on all Append() calls
I20260810 04:45:23.501899 26932 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a56dcd9698014f0c99a4dbe288e06859: No bootstrap required, opened a new log
I20260810 04:45:23.503515 23197 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskemv_AM/build/release/bin/kudu as pid 26821
I20260810 04:45:23.503607 23197 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786336863388938-23197-0/minicluster-data/master-0/wal/instance
I20260810 04:45:23.503607 26932 raft_consensus.cc:348] T 00000000000000000000000000000000 P a56dcd9698014f0c99a4dbe288e06859 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a56dcd9698014f0c99a4dbe288e06859" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39479 } }
I20260810 04:45:23.503751 23197 external_mini_cluster.cc:1428] Setting key 58eec46bc10446d86e1ebe6b0ebb4130
I20260810 04:45:23.503762 26932 raft_consensus.cc:374] T 00000000000000000000000000000000 P a56dcd9698014f0c99a4dbe288e06859 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 04:45:23.503798 26932 raft_consensus.cc:729] T 00000000000000000000000000000000 P a56dcd9698014f0c99a4dbe288e06859 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a56dcd9698014f0c99a4dbe288e06859, State: Initialized, Role: FOLLOWER
I20260810 04:45:23.503942 26932 consensus_queue.cc:260] T 00000000000000000000000000000000 P a56dcd9698014f0c99a4dbe288e06859 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a56dcd9698014f0c99a4dbe288e06859" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39479 } }
I20260810 04:45:23.504027 26932 raft_consensus.cc:388] T 00000000000000000000000000000000 P a56dcd9698014f0c99a4dbe288e06859 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 04:45:23.504068 26932 raft_consensus.cc:482] T 00000000000000000000000000000000 P a56dcd9698014f0c99a4dbe288e06859 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 04:45:23.504122 26932 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a56dcd9698014f0c99a4dbe288e06859 [term 0 FOLLOWER]: Advancing to term 1
I20260810 04:45:23.505115 26932 raft_consensus.cc:504] T 00000000000000000000000000000000 P a56dcd9698014f0c99a4dbe288e06859 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a56dcd9698014f0c99a4dbe288e06859" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39479 } }
I20260810 04:45:23.505242 26932 leader_election.cc:302] T 00000000000000000000000000000000 P a56dcd9698014f0c99a4dbe288e06859 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a56dcd9698014f0c99a4dbe288e06859; no voters:
I20260810 04:45:23.505409 26932 leader_election.cc:288] T 00000000000000000000000000000000 P a56dcd9698014f0c99a4dbe288e06859 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 04:45:23.505439 26939 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a56dcd9698014f0c99a4dbe288e06859 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 04:45:23.505597 26939 raft_consensus.cc:686] T 00000000000000000000000000000000 P a56dcd9698014f0c99a4dbe288e06859 [term 1 LEADER]: Becoming Leader. State: Replica: a56dcd9698014f0c99a4dbe288e06859, State: Running, Role: LEADER
I20260810 04:45:23.505671 26932 sys_catalog.cc:564] T 00000000000000000000000000000000 P a56dcd9698014f0c99a4dbe288e06859 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 04:45:23.505754 26939 consensus_queue.cc:237] T 00000000000000000000000000000000 P a56dcd9698014f0c99a4dbe288e06859 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a56dcd9698014f0c99a4dbe288e06859" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39479 } }
I20260810 04:45:23.506083 26939 sys_catalog.cc:455] T 00000000000000000000000000000000 P a56dcd9698014f0c99a4dbe288e06859 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a56dcd9698014f0c99a4dbe288e06859. Latest consensus state: current_term: 1 leader_uuid: "a56dcd9698014f0c99a4dbe288e06859" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a56dcd9698014f0c99a4dbe288e06859" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39479 } } }
I20260810 04:45:23.506337 26939 sys_catalog.cc:458] T 00000000000000000000000000000000 P a56dcd9698014f0c99a4dbe288e06859 [sys.catalog]: This master's current role is: LEADER
I20260810 04:45:23.506261 26942 sys_catalog.cc:455] T 00000000000000000000000000000000 P a56dcd9698014f0c99a4dbe288e06859 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a56dcd9698014f0c99a4dbe288e06859" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a56dcd9698014f0c99a4dbe288e06859" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39479 } } }
I20260810 04:45:23.506709 26942 sys_catalog.cc:458] T 00000000000000000000000000000000 P a56dcd9698014f0c99a4dbe288e06859 [sys.catalog]: This master's current role is: LEADER
I20260810 04:45:23.506875 26954 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 04:45:23.507344 26954 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 04:45:23.509199 26954 catalog_manager.cc:1347] Generated new cluster ID: 72dbdee7b2cc4929b0396020a7879ffd
I20260810 04:45:23.509250 26954 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 10 04:45:23 dist-test-slave-bqcl krb5kdc[26799](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786337123, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260810 04:45:23.513988 26954 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 04:45:23.514618 26954 catalog_manager.cc:1504] Loading token signing keys...
I20260810 04:45:23.515362 26938 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:45:23.504226 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.178:54939 (local address 127.0.0.1:39479)
0810 04:45:23.504388 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0810 04:45:23.504398 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 04:45:23.504575 (+ 177us) server_negotiation.cc:407] Connection header received
0810 04:45:23.504725 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:45:23.504731 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:45:23.504775 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:45:23.504890 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0810 04:45:23.505737 (+ 847us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:45:23.507245 (+ 1508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:45:23.508132 (+ 887us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:45:23.508739 (+ 607us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:45:23.508844 (+ 105us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:45:23.511705 (+ 2861us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 04:45:23.511709 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 04:45:23.511710 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 04:45:23.511723 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0810 04:45:23.513632 (+ 1909us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 04:45:23.514118 (+ 486us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 04:45:23.514137 (+ 19us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 04:45:23.514137 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 04:45:23.514246 (+ 109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 04:45:23.514503 (+ 257us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 04:45:23.514504 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 04:45:23.514504 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 04:45:23.514779 (+ 275us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 04:45:23.514932 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0810 04:45:23.515069 (+ 137us) server_negotiation.cc:299] Negotiation successful
0810 04:45:23.515163 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":111,"thread_start_us":78,"threads_started":1}
I20260810 04:45:23.516315 23197 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 04:45:23.518913 26954 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a56dcd9698014f0c99a4dbe288e06859: Generated new TSK 0
I20260810 04:45:23.519069 26954 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 04:45:23.524032 26938 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:45:23.517913 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.178:45961 (local address 127.0.0.1:39479)
0810 04:45:23.517957 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0810 04:45:23.517959 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 04:45:23.518026 (+ 67us) server_negotiation.cc:407] Connection header received
0810 04:45:23.518087 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:45:23.518087 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:45:23.518096 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:45:23.518177 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0810 04:45:23.518762 (+ 585us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:45:23.519409 (+ 647us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:45:23.520419 (+ 1010us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:45:23.520623 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:45:23.520667 (+ 44us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:45:23.521869 (+ 1202us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 04:45:23.521870 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 04:45:23.521871 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 04:45:23.521876 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0810 04:45:23.523071 (+ 1195us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 04:45:23.523434 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 04:45:23.523435 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 04:45:23.523435 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 04:45:23.523477 (+ 42us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 04:45:23.523660 (+ 183us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 04:45:23.523661 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 04:45:23.523661 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 04:45:23.523740 (+ 79us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 04:45:23.523817 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0810 04:45:23.523936 (+ 119us) server_negotiation.cc:299] Negotiation successful
0810 04:45:23.523978 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260810 04:45:23.525943 23197 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskemv_AM/build/release/bin/kudu with pid 26821
W20260810 04:45:23.556068 26796 connection.cc:535] client connection to 127.0.0.1:39479 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:39479): Connection reset by peer (error 104)
2026-08-10T04:45:23Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/9 (1431 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/10
Loading random data
Initializing database '/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786336863388938-23197-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 04:45:23 dist-test-slave-bqcl krb5kdc[26983](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 04:45:23 dist-test-slave-bqcl krb5kdc[26983](info): set up 2 sockets
Aug 10 04:45:23 dist-test-slave-bqcl krb5kdc[26983](info): commencing operation
krb5kdc: starting...
W20260810 04:45:24.712283 23197 mini_kdc.cc:121] Time spent starting KDC: real 1.143s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786336863388938-23197-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786336863388938-23197-0/krb5kdc/test-user.keytab.
Aug 10 04:45:24 dist-test-slave-bqcl krb5kdc[26983](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786337124, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T04:45:24Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T04:45:24Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786336863388938-23197-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786336863388938-23197-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260810 04:45:24.821657 23197 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskemv_AM/build/release/bin/kudu
/tmp/dist-test-taskemv_AM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786336863388938-23197-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786336863388938-23197-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786336863388938-23197-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786336863388938-23197-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36155
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786336863388938-23197-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:33801
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:36155
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786336863388938-23197-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786336863388938-23197-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786336863388938-23197-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 04:45:24.899459 27003 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 04:45:24.899641 27003 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 04:45:24.899662 27003 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 04:45:24.900723 27003 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 04:45:24.900756 27003 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 04:45:24.900768 27003 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 04:45:24.900779 27003 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 04:45:24.900790 27003 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 04:45:24.902187 27003 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:33801
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786336863388938-23197-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786336863388938-23197-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:36155
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786336863388938-23197-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36155
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786336863388938-23197-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27003
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786336863388938-23197-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 03:54:01 UTC on ccycloud.quasar-icijmt.root.comops.site
build id 4577009
I20260810 04:45:24.902362 27003 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 04:45:24.902521 27003 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 04:45:24.907061 27013 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 04:45:24.907153 27003 server_base.cc:1034] running on GCE node
W20260810 04:45:24.907061 27016 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 04:45:24.907111 27010 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 04:45:24.907502 27003 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 04:45:24.907771 27003 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 04:45:24.908921 27003 hybrid_clock.cc:648] HybridClock initialized: now 1786337124908901 us; error 33 us; skew 500 ppm
Aug 10 04:45:24 dist-test-slave-bqcl krb5kdc[26983](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786337124, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 04:45:24.911253 27003 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260810 04:45:24.911687 27003 webserver.cc:460] Webserver started at http://127.0.0.1:39253/ using document root <none> and password file <none>
I20260810 04:45:24.911886 27003 fs_manager.cc:359] Metadata directory not provided
I20260810 04:45:24.911934 27003 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 04:45:24.912066 27003 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 04:45:24.913025 27003 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786336863388938-23197-0/minicluster-data/master-0/data/instance:
uuid: "b5a8db67ca64480db6e843931000d5e6"
format_stamp: "Formatted at 2026-08-10 04:45:24 on dist-test-slave-bqcl"
server_key: "26a3d2610fac9723996054e06e89aa85"
server_key_iv: "5377b4d461874a2c24711e2915ed2ea7"
server_key_version: "encryptionkey@0"
I20260810 04:45:24.913331 27003 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786336863388938-23197-0/minicluster-data/master-0/wal/instance:
uuid: "b5a8db67ca64480db6e843931000d5e6"
format_stamp: "Formatted at 2026-08-10 04:45:24 on dist-test-slave-bqcl"
server_key: "26a3d2610fac9723996054e06e89aa85"
server_key_iv: "5377b4d461874a2c24711e2915ed2ea7"
server_key_version: "encryptionkey@0"
I20260810 04:45:24.914580 27003 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260810 04:45:24.915227 27029 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 04:45:24.915377 27003 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260810 04:45:24.915454 27003 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786336863388938-23197-0/minicluster-data/master-0/data,/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786336863388938-23197-0/minicluster-data/master-0/wal
uuid: "b5a8db67ca64480db6e843931000d5e6"
format_stamp: "Formatted at 2026-08-10 04:45:24 on dist-test-slave-bqcl"
server_key: "26a3d2610fac9723996054e06e89aa85"
server_key_iv: "5377b4d461874a2c24711e2915ed2ea7"
server_key_version: "encryptionkey@0"
I20260810 04:45:24.915508 27003 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786336863388938-23197-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786336863388938-23197-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786336863388938-23197-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 04:45:24.931243 27003 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 04:45:24.933074 27003 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 04:45:24.933178 27003 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 04:45:24.937170 27003 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:36155
I20260810 04:45:24.937206 27124 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:36155 every 8 connection(s)
I20260810 04:45:24.937508 27003 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786336863388938-23197-0/minicluster-data/master-0/data/info.pb
I20260810 04:45:24.938372 27125 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 04:45:24.940572 27125 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b5a8db67ca64480db6e843931000d5e6: Bootstrap starting.
I20260810 04:45:24.941107 27125 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b5a8db67ca64480db6e843931000d5e6: Neither blocks nor log segments found. Creating new log.
I20260810 04:45:24.941339 27125 log.cc:824] T 00000000000000000000000000000000 P b5a8db67ca64480db6e843931000d5e6: Log is configured to *not* fsync() on all Append() calls
I20260810 04:45:24.941982 27125 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b5a8db67ca64480db6e843931000d5e6: No bootstrap required, opened a new log
I20260810 04:45:24.943643 27125 raft_consensus.cc:348] T 00000000000000000000000000000000 P b5a8db67ca64480db6e843931000d5e6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b5a8db67ca64480db6e843931000d5e6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36155 } }
I20260810 04:45:24.943778 27125 raft_consensus.cc:374] T 00000000000000000000000000000000 P b5a8db67ca64480db6e843931000d5e6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 04:45:24.943799 27125 raft_consensus.cc:729] T 00000000000000000000000000000000 P b5a8db67ca64480db6e843931000d5e6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b5a8db67ca64480db6e843931000d5e6, State: Initialized, Role: FOLLOWER
I20260810 04:45:24.943899 27125 consensus_queue.cc:260] T 00000000000000000000000000000000 P b5a8db67ca64480db6e843931000d5e6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b5a8db67ca64480db6e843931000d5e6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36155 } }
I20260810 04:45:24.943953 27125 raft_consensus.cc:388] T 00000000000000000000000000000000 P b5a8db67ca64480db6e843931000d5e6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 04:45:24.943971 27125 raft_consensus.cc:482] T 00000000000000000000000000000000 P b5a8db67ca64480db6e843931000d5e6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 04:45:24.944000 27125 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b5a8db67ca64480db6e843931000d5e6 [term 0 FOLLOWER]: Advancing to term 1
I20260810 04:45:24.944455 27125 raft_consensus.cc:504] T 00000000000000000000000000000000 P b5a8db67ca64480db6e843931000d5e6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b5a8db67ca64480db6e843931000d5e6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36155 } }
I20260810 04:45:24.944537 27125 leader_election.cc:302] T 00000000000000000000000000000000 P b5a8db67ca64480db6e843931000d5e6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b5a8db67ca64480db6e843931000d5e6; no voters:
I20260810 04:45:24.944662 27125 leader_election.cc:288] T 00000000000000000000000000000000 P b5a8db67ca64480db6e843931000d5e6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 04:45:24.944706 27128 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b5a8db67ca64480db6e843931000d5e6 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 04:45:24.944859 27128 raft_consensus.cc:686] T 00000000000000000000000000000000 P b5a8db67ca64480db6e843931000d5e6 [term 1 LEADER]: Becoming Leader. State: Replica: b5a8db67ca64480db6e843931000d5e6, State: Running, Role: LEADER
I20260810 04:45:24.944901 27125 sys_catalog.cc:564] T 00000000000000000000000000000000 P b5a8db67ca64480db6e843931000d5e6 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 04:45:24.944973 27128 consensus_queue.cc:237] T 00000000000000000000000000000000 P b5a8db67ca64480db6e843931000d5e6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b5a8db67ca64480db6e843931000d5e6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36155 } }
I20260810 04:45:24.945616 27129 sys_catalog.cc:455] T 00000000000000000000000000000000 P b5a8db67ca64480db6e843931000d5e6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b5a8db67ca64480db6e843931000d5e6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b5a8db67ca64480db6e843931000d5e6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36155 } } }
I20260810 04:45:24.945657 23197 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskemv_AM/build/release/bin/kudu as pid 27003
I20260810 04:45:24.945709 27129 sys_catalog.cc:458] T 00000000000000000000000000000000 P b5a8db67ca64480db6e843931000d5e6 [sys.catalog]: This master's current role is: LEADER
I20260810 04:45:24.945763 23197 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786336863388938-23197-0/minicluster-data/master-0/wal/instance
I20260810 04:45:24.945909 27141 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 04:45:24.945945 23197 external_mini_cluster.cc:1428] Setting key 0c89f84b2586bd09b34a7eca44a380af
I20260810 04:45:24.946035 27130 sys_catalog.cc:455] T 00000000000000000000000000000000 P b5a8db67ca64480db6e843931000d5e6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b5a8db67ca64480db6e843931000d5e6. Latest consensus state: current_term: 1 leader_uuid: "b5a8db67ca64480db6e843931000d5e6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b5a8db67ca64480db6e843931000d5e6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36155 } } }
I20260810 04:45:24.946097 27130 sys_catalog.cc:458] T 00000000000000000000000000000000 P b5a8db67ca64480db6e843931000d5e6 [sys.catalog]: This master's current role is: LEADER
I20260810 04:45:24.946444 27141 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 04:45:24.949877 27141 catalog_manager.cc:1347] Generated new cluster ID: 01f703876a9e4c54ab6c1c28c5d94e1d
I20260810 04:45:24.949939 27141 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 04:45:24.951882 27153 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:45:24.946474 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.178:41725 (local address 127.0.0.1:36155)
0810 04:45:24.946803 (+ 329us) server_negotiation.cc:206] Beginning negotiation
0810 04:45:24.946813 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 04:45:24.946847 (+ 34us) server_negotiation.cc:407] Connection header received
0810 04:45:24.947164 (+ 317us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:45:24.947174 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:45:24.947240 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:45:24.947386 (+ 146us) server_negotiation.cc:226] Negotiated authn=SASL
0810 04:45:24.947640 (+ 254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:45:24.949008 (+ 1368us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:45:24.950080 (+ 1072us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:45:24.950579 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:45:24.950685 (+ 106us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:45:24.951295 (+ 610us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 04:45:24.951298 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 04:45:24.951299 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 04:45:24.951464 (+ 165us) server_negotiation.cc:922] calling sasl_server_start()
0810 04:45:24.951496 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 04:45:24.951515 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 04:45:24.951586 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0810 04:45:24.951679 (+ 93us) server_negotiation.cc:299] Negotiation successful
0810 04:45:24.951710 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":263,"thread_start_us":73,"threads_started":1}
I20260810 04:45:24.952723 23197 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 04:45:24.957067 27153 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:45:24.954435 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.178:49073 (local address 127.0.0.1:36155)
0810 04:45:24.954473 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0810 04:45:24.954475 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 04:45:24.954629 (+ 154us) server_negotiation.cc:407] Connection header received
0810 04:45:24.954685 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:45:24.954685 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:45:24.954697 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:45:24.954742 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0810 04:45:24.954863 (+ 121us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:45:24.955327 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:45:24.956014 (+ 687us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:45:24.956229 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:45:24.956282 (+ 53us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:45:24.956839 (+ 557us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 04:45:24.956840 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 04:45:24.956840 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 04:45:24.956844 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0810 04:45:24.956858 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 04:45:24.956862 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 04:45:24.956911 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0810 04:45:24.956979 (+ 68us) server_negotiation.cc:299] Negotiation successful
0810 04:45:24.956989 (+ 10us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260810 04:45:24.957165 27141 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 04:45:24.957818 27141 catalog_manager.cc:1504] Loading token signing keys...
I20260810 04:45:24.969758 27141 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b5a8db67ca64480db6e843931000d5e6: Generated new TSK 0
I20260810 04:45:24.969928 27141 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 04:45:24.976316 23197 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskemv_AM/build/release/bin/kudu with pid 27003
W20260810 04:45:24.994645 26980 connection.cc:535] client connection to 127.0.0.1:36155 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:36155): Connection reset by peer (error 104)
2026-08-10T04:45:24Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/10 (1438 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/11
Loading random data
Initializing database '/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786336863388938-23197-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 04:45:25 dist-test-slave-bqcl krb5kdc[27172](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 04:45:25 dist-test-slave-bqcl krb5kdc[27172](info): set up 2 sockets
Aug 10 04:45:25 dist-test-slave-bqcl krb5kdc[27172](info): commencing operation
krb5kdc: starting...
W20260810 04:45:26.147948 23197 mini_kdc.cc:121] Time spent starting KDC: real 1.140s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786336863388938-23197-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786336863388938-23197-0/krb5kdc/test-user.keytab.
Aug 10 04:45:26 dist-test-slave-bqcl krb5kdc[27172](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786337126, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T04:45:26Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T04:45:26Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786336863388938-23197-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786336863388938-23197-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260810 04:45:26.265925 23197 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskemv_AM/build/release/bin/kudu
/tmp/dist-test-taskemv_AM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786336863388938-23197-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786336863388938-23197-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786336863388938-23197-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786336863388938-23197-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44799
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786336863388938-23197-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:36383
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:44799
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786336863388938-23197-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786336863388938-23197-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786336863388938-23197-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 04:45:26.349802 27194 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 04:45:26.349989 27194 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 04:45:26.350019 27194 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 04:45:26.351168 27194 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 04:45:26.351210 27194 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 04:45:26.351223 27194 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 04:45:26.351238 27194 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 04:45:26.351260 27194 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 04:45:26.352702 27194 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:36383
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786336863388938-23197-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786336863388938-23197-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:44799
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786336863388938-23197-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44799
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786336863388938-23197-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27194
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786336863388938-23197-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 03:54:01 UTC on ccycloud.quasar-icijmt.root.comops.site
build id 4577009
I20260810 04:45:26.352916 27194 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 04:45:26.353116 27194 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 04:45:26.357867 27203 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 04:45:26.357892 27210 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 04:45:26.357888 27205 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 04:45:26.357918 27194 server_base.cc:1034] running on GCE node
I20260810 04:45:26.358299 27194 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 04:45:26.358569 27194 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 04:45:26.359711 27194 hybrid_clock.cc:648] HybridClock initialized: now 1786337126359692 us; error 30 us; skew 500 ppm
Aug 10 04:45:26 dist-test-slave-bqcl krb5kdc[27172](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786337126, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 04:45:26.362031 27194 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260810 04:45:26.362442 27194 webserver.cc:460] Webserver started at http://127.0.0.1:35359/ using document root <none> and password file <none>
I20260810 04:45:26.362607 27194 fs_manager.cc:359] Metadata directory not provided
I20260810 04:45:26.362651 27194 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 04:45:26.362739 27194 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 04:45:26.363672 27194 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786336863388938-23197-0/minicluster-data/master-0/data/instance:
uuid: "137bafd3b8ac4bfabb3067af8791b323"
format_stamp: "Formatted at 2026-08-10 04:45:26 on dist-test-slave-bqcl"
server_key: "a47ed6e83671c2d662d99258052e1ebd"
server_key_iv: "88743e3017ac3eb07f2f256818016037"
server_key_version: "encryptionkey@0"
I20260810 04:45:26.363952 27194 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786336863388938-23197-0/minicluster-data/master-0/wal/instance:
uuid: "137bafd3b8ac4bfabb3067af8791b323"
format_stamp: "Formatted at 2026-08-10 04:45:26 on dist-test-slave-bqcl"
server_key: "a47ed6e83671c2d662d99258052e1ebd"
server_key_iv: "88743e3017ac3eb07f2f256818016037"
server_key_version: "encryptionkey@0"
I20260810 04:45:26.365072 27194 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260810 04:45:26.365693 27220 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 04:45:26.365859 27194 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260810 04:45:26.365942 27194 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786336863388938-23197-0/minicluster-data/master-0/data,/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786336863388938-23197-0/minicluster-data/master-0/wal
uuid: "137bafd3b8ac4bfabb3067af8791b323"
format_stamp: "Formatted at 2026-08-10 04:45:26 on dist-test-slave-bqcl"
server_key: "a47ed6e83671c2d662d99258052e1ebd"
server_key_iv: "88743e3017ac3eb07f2f256818016037"
server_key_version: "encryptionkey@0"
I20260810 04:45:26.365993 27194 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786336863388938-23197-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786336863388938-23197-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786336863388938-23197-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 04:45:26.369065 27194 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 04:45:26.370780 27194 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 04:45:26.370880 27194 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 04:45:26.374478 27194 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:44799
I20260810 04:45:26.374507 27315 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:44799 every 8 connection(s)
I20260810 04:45:26.374794 27194 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786336863388938-23197-0/minicluster-data/master-0/data/info.pb
I20260810 04:45:26.375690 27316 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 04:45:26.377676 27316 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 137bafd3b8ac4bfabb3067af8791b323: Bootstrap starting.
I20260810 04:45:26.378213 27316 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 137bafd3b8ac4bfabb3067af8791b323: Neither blocks nor log segments found. Creating new log.
I20260810 04:45:26.378463 27316 log.cc:824] T 00000000000000000000000000000000 P 137bafd3b8ac4bfabb3067af8791b323: Log is configured to *not* fsync() on all Append() calls
I20260810 04:45:26.379071 27316 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 137bafd3b8ac4bfabb3067af8791b323: No bootstrap required, opened a new log
I20260810 04:45:26.380141 23197 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskemv_AM/build/release/bin/kudu as pid 27194
I20260810 04:45:26.380241 23197 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786336863388938-23197-0/minicluster-data/master-0/wal/instance
I20260810 04:45:26.380410 23197 external_mini_cluster.cc:1428] Setting key 8e54fcc21c5be8fc48f3b8722f043497
I20260810 04:45:26.380695 27316 raft_consensus.cc:348] T 00000000000000000000000000000000 P 137bafd3b8ac4bfabb3067af8791b323 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "137bafd3b8ac4bfabb3067af8791b323" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44799 } }
I20260810 04:45:26.380865 27316 raft_consensus.cc:374] T 00000000000000000000000000000000 P 137bafd3b8ac4bfabb3067af8791b323 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 04:45:26.380934 27316 raft_consensus.cc:729] T 00000000000000000000000000000000 P 137bafd3b8ac4bfabb3067af8791b323 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 137bafd3b8ac4bfabb3067af8791b323, State: Initialized, Role: FOLLOWER
I20260810 04:45:26.381074 27316 consensus_queue.cc:260] T 00000000000000000000000000000000 P 137bafd3b8ac4bfabb3067af8791b323 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "137bafd3b8ac4bfabb3067af8791b323" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44799 } }
I20260810 04:45:26.381160 27316 raft_consensus.cc:388] T 00000000000000000000000000000000 P 137bafd3b8ac4bfabb3067af8791b323 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 04:45:26.381192 27316 raft_consensus.cc:482] T 00000000000000000000000000000000 P 137bafd3b8ac4bfabb3067af8791b323 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 04:45:26.381235 27316 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 137bafd3b8ac4bfabb3067af8791b323 [term 0 FOLLOWER]: Advancing to term 1
I20260810 04:45:26.381887 27316 raft_consensus.cc:504] T 00000000000000000000000000000000 P 137bafd3b8ac4bfabb3067af8791b323 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "137bafd3b8ac4bfabb3067af8791b323" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44799 } }
I20260810 04:45:26.381990 27316 leader_election.cc:302] T 00000000000000000000000000000000 P 137bafd3b8ac4bfabb3067af8791b323 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 137bafd3b8ac4bfabb3067af8791b323; no voters:
I20260810 04:45:26.382169 27316 leader_election.cc:288] T 00000000000000000000000000000000 P 137bafd3b8ac4bfabb3067af8791b323 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 04:45:26.382215 27324 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 137bafd3b8ac4bfabb3067af8791b323 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 04:45:26.382472 27316 sys_catalog.cc:564] T 00000000000000000000000000000000 P 137bafd3b8ac4bfabb3067af8791b323 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 04:45:26.382434 27324 raft_consensus.cc:686] T 00000000000000000000000000000000 P 137bafd3b8ac4bfabb3067af8791b323 [term 1 LEADER]: Becoming Leader. State: Replica: 137bafd3b8ac4bfabb3067af8791b323, State: Running, Role: LEADER
I20260810 04:45:26.382645 27324 consensus_queue.cc:237] T 00000000000000000000000000000000 P 137bafd3b8ac4bfabb3067af8791b323 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "137bafd3b8ac4bfabb3067af8791b323" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44799 } }
I20260810 04:45:26.382654 27323 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:45:26.380891 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.178:50721 (local address 127.0.0.1:44799)
0810 04:45:26.381116 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0810 04:45:26.381125 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 04:45:26.381173 (+ 48us) server_negotiation.cc:407] Connection header received
0810 04:45:26.381281 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:45:26.381288 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:45:26.381317 (+ 29us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:45:26.381455 (+ 138us) server_negotiation.cc:226] Negotiated authn=SASL
0810 04:45:26.382053 (+ 598us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 04:45:26.382057 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 04:45:26.382057 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 04:45:26.382061 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0810 04:45:26.382101 (+ 40us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 04:45:26.382117 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 04:45:26.382218 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0810 04:45:26.382302 (+ 84us) server_negotiation.cc:299] Negotiation successful
0810 04:45:26.382436 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":165,"thread_start_us":112,"threads_started":1}
I20260810 04:45:26.383025 27327 sys_catalog.cc:455] T 00000000000000000000000000000000 P 137bafd3b8ac4bfabb3067af8791b323 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 137bafd3b8ac4bfabb3067af8791b323. Latest consensus state: current_term: 1 leader_uuid: "137bafd3b8ac4bfabb3067af8791b323" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "137bafd3b8ac4bfabb3067af8791b323" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44799 } } }
I20260810 04:45:26.383030 27328 sys_catalog.cc:455] T 00000000000000000000000000000000 P 137bafd3b8ac4bfabb3067af8791b323 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "137bafd3b8ac4bfabb3067af8791b323" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "137bafd3b8ac4bfabb3067af8791b323" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44799 } } }
I20260810 04:45:26.383117 27327 sys_catalog.cc:458] T 00000000000000000000000000000000 P 137bafd3b8ac4bfabb3067af8791b323 [sys.catalog]: This master's current role is: LEADER
I20260810 04:45:26.383131 27328 sys_catalog.cc:458] T 00000000000000000000000000000000 P 137bafd3b8ac4bfabb3067af8791b323 [sys.catalog]: This master's current role is: LEADER
I20260810 04:45:26.383378 27333 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 04:45:26.383612 23197 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 04:45:26.383909 27333 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 04:45:26.386179 27323 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:45:26.385088 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.178:49613 (local address 127.0.0.1:44799)
0810 04:45:26.385115 (+ 27us) server_negotiation.cc:206] Beginning negotiation
0810 04:45:26.385117 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 04:45:26.385318 (+ 201us) server_negotiation.cc:407] Connection header received
0810 04:45:26.385392 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:45:26.385393 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:45:26.385406 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:45:26.385451 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0810 04:45:26.385949 (+ 498us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 04:45:26.385950 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 04:45:26.385950 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 04:45:26.385953 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0810 04:45:26.385964 (+ 11us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 04:45:26.385968 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 04:45:26.386010 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0810 04:45:26.386072 (+ 62us) server_negotiation.cc:299] Negotiation successful
0810 04:45:26.386082 (+ 10us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13}
I20260810 04:45:26.390689 27333 catalog_manager.cc:1347] Generated new cluster ID: b74a91e720a0422c925bb5384a174b5f
I20260810 04:45:26.390738 27333 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 04:45:26.400167 27333 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 04:45:26.400305 27333 catalog_manager.cc:1504] Loading token signing keys...
I20260810 04:45:26.412433 27333 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 137bafd3b8ac4bfabb3067af8791b323: Generated new TSK 0
I20260810 04:45:26.412562 27333 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 04:45:26.416481 23197 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskemv_AM/build/release/bin/kudu with pid 27194
2026-08-10T04:45:26Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/11 (1437 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/12
Loading random data
Initializing database '/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786336863388938-23197-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 04:45:26 dist-test-slave-bqcl krb5kdc[27364](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 04:45:26 dist-test-slave-bqcl krb5kdc[27364](info): set up 2 sockets
Aug 10 04:45:26 dist-test-slave-bqcl krb5kdc[27364](info): commencing operation
krb5kdc: starting...
W20260810 04:45:27.574735 23197 mini_kdc.cc:121] Time spent starting KDC: real 1.128s user 0.005s sys 0.001s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786336863388938-23197-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786336863388938-23197-0/krb5kdc/test-user.keytab.
Aug 10 04:45:27 dist-test-slave-bqcl krb5kdc[27364](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786337127, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T04:45:27Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T04:45:27Z Disabled control of system clock
WARNING: no policy specified for kudu/127.22.167.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.167.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.167.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786336863388938-23197-0/krb5kdc/kudu_127.22.167.126.keytab.
Entry for principal kudu/127.22.167.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786336863388938-23197-0/krb5kdc/kudu_127.22.167.126.keytab.
I20260810 04:45:27.685809 23197 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskemv_AM/build/release/bin/kudu
/tmp/dist-test-taskemv_AM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786336863388938-23197-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786336863388938-23197-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786336863388938-23197-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786336863388938-23197-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.167.126:44727
--webserver_interface=127.22.167.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786336863388938-23197-0/krb5kdc/kudu_127.22.167.126.keytab
--principal=kudu/127.22.167.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.167.84:34479
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.167.126:44727
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786336863388938-23197-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786336863388938-23197-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786336863388938-23197-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 04:45:27.764348 27384 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 04:45:27.764530 27384 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 04:45:27.764552 27384 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 04:45:27.765620 27384 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 04:45:27.765657 27384 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 04:45:27.765671 27384 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 04:45:27.765681 27384 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 04:45:27.765692 27384 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 04:45:27.767139 27384 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.167.84:34479
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786336863388938-23197-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786336863388938-23197-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.22.167.126:44727
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786336863388938-23197-0/krb5kdc/kudu_127.22.167.126.keytab
--principal=kudu/127.22.167.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.167.126:44727
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786336863388938-23197-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.167.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27384
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786336863388938-23197-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 03:54:01 UTC on ccycloud.quasar-icijmt.root.comops.site
build id 4577009
I20260810 04:45:27.767321 27384 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 04:45:27.767475 27384 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260810 04:45:27.771957 27384 server_base.cc:1034] running on GCE node
W20260810 04:45:27.771929 27399 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 04:45:27.771934 27396 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 04:45:27.771915 27402 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 04:45:27.772442 27384 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 04:45:27.772725 27384 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 04:45:27.773882 27384 hybrid_clock.cc:648] HybridClock initialized: now 1786337127773864 us; error 29 us; skew 500 ppm
Aug 10 04:45:27 dist-test-slave-bqcl krb5kdc[27364](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786337127, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.167.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 04:45:27.779333 27384 init.cc:377] Logged in from keytab as kudu/127.22.167.126@KRBTEST.COM (short username kudu)
I20260810 04:45:27.779769 27384 webserver.cc:460] Webserver started at http://127.22.167.126:40591/ using document root <none> and password file <none>
I20260810 04:45:27.779978 27384 fs_manager.cc:359] Metadata directory not provided
I20260810 04:45:27.780032 27384 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 04:45:27.780158 27384 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 04:45:27.781149 27384 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786336863388938-23197-0/minicluster-data/master-0/data/instance:
uuid: "ccbd801c0415477b8200e40bfb76f747"
format_stamp: "Formatted at 2026-08-10 04:45:27 on dist-test-slave-bqcl"
server_key: "5b56e220f5d5d915e9cd45b5a73af9c9"
server_key_iv: "a624b2b81c153e330fd0c78e78dcca2c"
server_key_version: "encryptionkey@0"
I20260810 04:45:27.781471 27384 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786336863388938-23197-0/minicluster-data/master-0/wal/instance:
uuid: "ccbd801c0415477b8200e40bfb76f747"
format_stamp: "Formatted at 2026-08-10 04:45:27 on dist-test-slave-bqcl"
server_key: "5b56e220f5d5d915e9cd45b5a73af9c9"
server_key_iv: "a624b2b81c153e330fd0c78e78dcca2c"
server_key_version: "encryptionkey@0"
I20260810 04:45:27.782728 27384 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260810 04:45:27.783437 27415 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 04:45:27.783595 27384 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260810 04:45:27.783679 27384 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786336863388938-23197-0/minicluster-data/master-0/data,/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786336863388938-23197-0/minicluster-data/master-0/wal
uuid: "ccbd801c0415477b8200e40bfb76f747"
format_stamp: "Formatted at 2026-08-10 04:45:27 on dist-test-slave-bqcl"
server_key: "5b56e220f5d5d915e9cd45b5a73af9c9"
server_key_iv: "a624b2b81c153e330fd0c78e78dcca2c"
server_key_version: "encryptionkey@0"
I20260810 04:45:27.783742 27384 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786336863388938-23197-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786336863388938-23197-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786336863388938-23197-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 04:45:27.794574 27384 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 04:45:27.796484 27384 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 04:45:27.796612 27384 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 04:45:27.800454 27384 rpc_server.cc:307] RPC server started. Bound to: 127.22.167.126:44727
I20260810 04:45:27.800498 27495 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.167.126:44727 every 8 connection(s)
I20260810 04:45:27.800798 27384 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786336863388938-23197-0/minicluster-data/master-0/data/info.pb
I20260810 04:45:27.801596 27496 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 04:45:27.803752 27496 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ccbd801c0415477b8200e40bfb76f747: Bootstrap starting.
I20260810 04:45:27.804297 27496 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ccbd801c0415477b8200e40bfb76f747: Neither blocks nor log segments found. Creating new log.
I20260810 04:45:27.804560 27496 log.cc:824] T 00000000000000000000000000000000 P ccbd801c0415477b8200e40bfb76f747: Log is configured to *not* fsync() on all Append() calls
I20260810 04:45:27.805279 27496 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ccbd801c0415477b8200e40bfb76f747: No bootstrap required, opened a new log
I20260810 04:45:27.806914 27496 raft_consensus.cc:348] T 00000000000000000000000000000000 P ccbd801c0415477b8200e40bfb76f747 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ccbd801c0415477b8200e40bfb76f747" member_type: VOTER last_known_addr { host: "127.22.167.126" port: 44727 } }
I20260810 04:45:27.807044 27496 raft_consensus.cc:374] T 00000000000000000000000000000000 P ccbd801c0415477b8200e40bfb76f747 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 04:45:27.807065 27496 raft_consensus.cc:729] T 00000000000000000000000000000000 P ccbd801c0415477b8200e40bfb76f747 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ccbd801c0415477b8200e40bfb76f747, State: Initialized, Role: FOLLOWER
I20260810 04:45:27.807196 27496 consensus_queue.cc:260] T 00000000000000000000000000000000 P ccbd801c0415477b8200e40bfb76f747 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ccbd801c0415477b8200e40bfb76f747" member_type: VOTER last_known_addr { host: "127.22.167.126" port: 44727 } }
I20260810 04:45:27.807258 27496 raft_consensus.cc:388] T 00000000000000000000000000000000 P ccbd801c0415477b8200e40bfb76f747 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 04:45:27.807278 27496 raft_consensus.cc:482] T 00000000000000000000000000000000 P ccbd801c0415477b8200e40bfb76f747 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 04:45:27.807312 27496 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ccbd801c0415477b8200e40bfb76f747 [term 0 FOLLOWER]: Advancing to term 1
I20260810 04:45:27.808092 27496 raft_consensus.cc:504] T 00000000000000000000000000000000 P ccbd801c0415477b8200e40bfb76f747 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ccbd801c0415477b8200e40bfb76f747" member_type: VOTER last_known_addr { host: "127.22.167.126" port: 44727 } }
I20260810 04:45:27.808187 27496 leader_election.cc:302] T 00000000000000000000000000000000 P ccbd801c0415477b8200e40bfb76f747 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ccbd801c0415477b8200e40bfb76f747; no voters:
I20260810 04:45:27.808333 27496 leader_election.cc:288] T 00000000000000000000000000000000 P ccbd801c0415477b8200e40bfb76f747 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 04:45:27.808374 27500 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ccbd801c0415477b8200e40bfb76f747 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 04:45:27.808555 27500 raft_consensus.cc:686] T 00000000000000000000000000000000 P ccbd801c0415477b8200e40bfb76f747 [term 1 LEADER]: Becoming Leader. State: Replica: ccbd801c0415477b8200e40bfb76f747, State: Running, Role: LEADER
I20260810 04:45:27.808562 27496 sys_catalog.cc:564] T 00000000000000000000000000000000 P ccbd801c0415477b8200e40bfb76f747 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 04:45:27.808655 27500 consensus_queue.cc:237] T 00000000000000000000000000000000 P ccbd801c0415477b8200e40bfb76f747 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ccbd801c0415477b8200e40bfb76f747" member_type: VOTER last_known_addr { host: "127.22.167.126" port: 44727 } }
I20260810 04:45:27.809026 27501 sys_catalog.cc:455] T 00000000000000000000000000000000 P ccbd801c0415477b8200e40bfb76f747 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ccbd801c0415477b8200e40bfb76f747" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ccbd801c0415477b8200e40bfb76f747" member_type: VOTER last_known_addr { host: "127.22.167.126" port: 44727 } } }
I20260810 04:45:27.809239 27501 sys_catalog.cc:458] T 00000000000000000000000000000000 P ccbd801c0415477b8200e40bfb76f747 [sys.catalog]: This master's current role is: LEADER
I20260810 04:45:27.809177 27502 sys_catalog.cc:455] T 00000000000000000000000000000000 P ccbd801c0415477b8200e40bfb76f747 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ccbd801c0415477b8200e40bfb76f747. Latest consensus state: current_term: 1 leader_uuid: "ccbd801c0415477b8200e40bfb76f747" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ccbd801c0415477b8200e40bfb76f747" member_type: VOTER last_known_addr { host: "127.22.167.126" port: 44727 } } }
I20260810 04:45:27.809405 27502 sys_catalog.cc:458] T 00000000000000000000000000000000 P ccbd801c0415477b8200e40bfb76f747 [sys.catalog]: This master's current role is: LEADER
I20260810 04:45:27.809623 27512 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 04:45:27.809911 23197 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskemv_AM/build/release/bin/kudu as pid 27384
I20260810 04:45:27.810030 23197 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786336863388938-23197-0/minicluster-data/master-0/wal/instance
I20260810 04:45:27.810110 27512 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 04:45:27.810225 23197 external_mini_cluster.cc:1428] Setting key 717cc80adffff33fc3e76f9f8d10d3e3
I20260810 04:45:27.811704 27512 catalog_manager.cc:1347] Generated new cluster ID: bbaa8bbe632b48089ba57511f132b487
I20260810 04:45:27.811746 27512 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 10 04:45:27 dist-test-slave-bqcl krb5kdc[27364](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786337127, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.167.126@KRBTEST.COM
I20260810 04:45:27.822700 27512 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 04:45:27.823236 27512 catalog_manager.cc:1504] Loading token signing keys...
I20260810 04:45:27.824569 27533 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:45:27.810647 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55225 (local address 127.22.167.126:44727)
0810 04:45:27.814226 (+ 3579us) server_negotiation.cc:206] Beginning negotiation
0810 04:45:27.814235 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 04:45:27.814267 (+ 32us) server_negotiation.cc:407] Connection header received
0810 04:45:27.814371 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:45:27.814377 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:45:27.814436 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:45:27.814564 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0810 04:45:27.815303 (+ 739us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:45:27.816637 (+ 1334us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:45:27.817691 (+ 1054us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:45:27.818326 (+ 635us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:45:27.818421 (+ 95us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:45:27.821052 (+ 2631us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 04:45:27.821057 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 04:45:27.821058 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 04:45:27.821069 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0810 04:45:27.822924 (+ 1855us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 04:45:27.823390 (+ 466us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 04:45:27.823393 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 04:45:27.823394 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 04:45:27.823491 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 04:45:27.823743 (+ 252us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 04:45:27.823744 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 04:45:27.823744 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 04:45:27.823990 (+ 246us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 04:45:27.824124 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0810 04:45:27.824250 (+ 126us) server_negotiation.cc:299] Negotiation successful
0810 04:45:27.824345 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3529,"thread_start_us":92,"threads_started":1}
I20260810 04:45:27.825414 27512 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ccbd801c0415477b8200e40bfb76f747: Generated new TSK 0
I20260810 04:45:27.825507 23197 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 04:45:27.825628 27512 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 04:45:27.832813 27533 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:45:27.826852 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48779 (local address 127.22.167.126:44727)
0810 04:45:27.826896 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0810 04:45:27.826897 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0810 04:45:27.826995 (+ 98us) server_negotiation.cc:407] Connection header received
0810 04:45:27.827041 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:45:27.827042 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:45:27.827049 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:45:27.827096 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0810 04:45:27.827662 (+ 566us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:45:27.828147 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:45:27.829154 (+ 1007us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:45:27.829348 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:45:27.829391 (+ 43us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:45:27.830608 (+ 1217us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 04:45:27.830609 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 04:45:27.830609 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 04:45:27.830615 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0810 04:45:27.831805 (+ 1190us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 04:45:27.832175 (+ 370us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 04:45:27.832176 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 04:45:27.832176 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 04:45:27.832228 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 04:45:27.832411 (+ 183us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 04:45:27.832411 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 04:45:27.832412 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 04:45:27.832496 (+ 84us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 04:45:27.832608 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0810 04:45:27.832712 (+ 104us) server_negotiation.cc:299] Negotiation successful
0810 04:45:27.832759 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260810 04:45:27.834772 23197 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskemv_AM/build/release/bin/kudu with pid 27384
W20260810 04:45:27.848307 27362 connection.cc:535] client connection to 127.22.167.126:44727 recv error: Network error: failed to read from TLS socket (remote: 127.22.167.126:44727): Connection reset by peer (error 104)
2026-08-10T04:45:27Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/12 (1415 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/13
Loading random data
Initializing database '/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786336863388938-23197-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 04:45:27 dist-test-slave-bqcl krb5kdc[27556](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 04:45:27 dist-test-slave-bqcl krb5kdc[27556](info): set up 2 sockets
Aug 10 04:45:27 dist-test-slave-bqcl krb5kdc[27556](info): commencing operation
krb5kdc: starting...
W20260810 04:45:28.989568 23197 mini_kdc.cc:121] Time spent starting KDC: real 1.128s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786336863388938-23197-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786336863388938-23197-0/krb5kdc/test-user.keytab.
Aug 10 04:45:29 dist-test-slave-bqcl krb5kdc[27556](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786337129, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T04:45:29Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T04:45:29Z Disabled control of system clock
WARNING: no policy specified for kudu/127.22.167.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.167.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.167.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786336863388938-23197-0/krb5kdc/kudu_127.22.167.126.keytab.
Entry for principal kudu/127.22.167.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786336863388938-23197-0/krb5kdc/kudu_127.22.167.126.keytab.
I20260810 04:45:29.103070 23197 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskemv_AM/build/release/bin/kudu
/tmp/dist-test-taskemv_AM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786336863388938-23197-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786336863388938-23197-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786336863388938-23197-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786336863388938-23197-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.167.126:45241
--webserver_interface=127.22.167.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786336863388938-23197-0/krb5kdc/kudu_127.22.167.126.keytab
--principal=kudu/127.22.167.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.167.84:41651
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.167.126:45241
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786336863388938-23197-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786336863388938-23197-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786336863388938-23197-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 04:45:29.182626 27576 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 04:45:29.182825 27576 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 04:45:29.182854 27576 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 04:45:29.183998 27576 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 04:45:29.184043 27576 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 04:45:29.184070 27576 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 04:45:29.184087 27576 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 04:45:29.184103 27576 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 04:45:29.185567 27576 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.167.84:41651
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786336863388938-23197-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786336863388938-23197-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.22.167.126:45241
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786336863388938-23197-0/krb5kdc/kudu_127.22.167.126.keytab
--principal=kudu/127.22.167.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.167.126:45241
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786336863388938-23197-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.167.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27576
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786336863388938-23197-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 03:54:01 UTC on ccycloud.quasar-icijmt.root.comops.site
build id 4577009
I20260810 04:45:29.185798 27576 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 04:45:29.185992 27576 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 04:45:29.190388 27586 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 04:45:29.190395 27587 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 04:45:29.190615 27576 server_base.cc:1034] running on GCE node
W20260810 04:45:29.190429 27590 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 04:45:29.190938 27576 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 04:45:29.191215 27576 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 04:45:29.192366 27576 hybrid_clock.cc:648] HybridClock initialized: now 1786337129192351 us; error 32 us; skew 500 ppm
Aug 10 04:45:29 dist-test-slave-bqcl krb5kdc[27556](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786337129, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.167.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 04:45:29.194677 27576 init.cc:377] Logged in from keytab as kudu/127.22.167.126@KRBTEST.COM (short username kudu)
I20260810 04:45:29.195055 27576 webserver.cc:460] Webserver started at http://127.22.167.126:35065/ using document root <none> and password file <none>
I20260810 04:45:29.195240 27576 fs_manager.cc:359] Metadata directory not provided
I20260810 04:45:29.195286 27576 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 04:45:29.195408 27576 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 04:45:29.196386 27576 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786336863388938-23197-0/minicluster-data/master-0/data/instance:
uuid: "3d01196dafe140fea5803e9dfe7535c1"
format_stamp: "Formatted at 2026-08-10 04:45:29 on dist-test-slave-bqcl"
server_key: "2f2a8b4002dc781a9862bedc00978a07"
server_key_iv: "5ec227e9aa6c75fb57959e4dbe484831"
server_key_version: "encryptionkey@0"
I20260810 04:45:29.196709 27576 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786336863388938-23197-0/minicluster-data/master-0/wal/instance:
uuid: "3d01196dafe140fea5803e9dfe7535c1"
format_stamp: "Formatted at 2026-08-10 04:45:29 on dist-test-slave-bqcl"
server_key: "2f2a8b4002dc781a9862bedc00978a07"
server_key_iv: "5ec227e9aa6c75fb57959e4dbe484831"
server_key_version: "encryptionkey@0"
I20260810 04:45:29.197960 27576 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260810 04:45:29.198680 27600 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 04:45:29.198849 27576 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260810 04:45:29.198930 27576 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786336863388938-23197-0/minicluster-data/master-0/data,/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786336863388938-23197-0/minicluster-data/master-0/wal
uuid: "3d01196dafe140fea5803e9dfe7535c1"
format_stamp: "Formatted at 2026-08-10 04:45:29 on dist-test-slave-bqcl"
server_key: "2f2a8b4002dc781a9862bedc00978a07"
server_key_iv: "5ec227e9aa6c75fb57959e4dbe484831"
server_key_version: "encryptionkey@0"
I20260810 04:45:29.198995 27576 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786336863388938-23197-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786336863388938-23197-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786336863388938-23197-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 04:45:29.217835 27576 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 04:45:29.219451 27576 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 04:45:29.219580 27576 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 04:45:29.223245 27576 rpc_server.cc:307] RPC server started. Bound to: 127.22.167.126:45241
I20260810 04:45:29.223278 27704 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.167.126:45241 every 8 connection(s)
I20260810 04:45:29.223572 27576 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786336863388938-23197-0/minicluster-data/master-0/data/info.pb
I20260810 04:45:29.224359 27705 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 04:45:29.226526 27705 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3d01196dafe140fea5803e9dfe7535c1: Bootstrap starting.
I20260810 04:45:29.227077 27705 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3d01196dafe140fea5803e9dfe7535c1: Neither blocks nor log segments found. Creating new log.
I20260810 04:45:29.227296 23197 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskemv_AM/build/release/bin/kudu as pid 27576
I20260810 04:45:29.227321 27705 log.cc:824] T 00000000000000000000000000000000 P 3d01196dafe140fea5803e9dfe7535c1: Log is configured to *not* fsync() on all Append() calls
I20260810 04:45:29.227398 23197 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786336863388938-23197-0/minicluster-data/master-0/wal/instance
I20260810 04:45:29.227550 23197 external_mini_cluster.cc:1428] Setting key 0500a16a28f65230b24894f62abda02d
I20260810 04:45:29.228014 27705 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3d01196dafe140fea5803e9dfe7535c1: No bootstrap required, opened a new log
I20260810 04:45:29.230394 27705 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3d01196dafe140fea5803e9dfe7535c1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3d01196dafe140fea5803e9dfe7535c1" member_type: VOTER last_known_addr { host: "127.22.167.126" port: 45241 } }
I20260810 04:45:29.230554 27705 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3d01196dafe140fea5803e9dfe7535c1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 04:45:29.230595 27705 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3d01196dafe140fea5803e9dfe7535c1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3d01196dafe140fea5803e9dfe7535c1, State: Initialized, Role: FOLLOWER
I20260810 04:45:29.230706 27705 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3d01196dafe140fea5803e9dfe7535c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3d01196dafe140fea5803e9dfe7535c1" member_type: VOTER last_known_addr { host: "127.22.167.126" port: 45241 } }
I20260810 04:45:29.230773 27705 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3d01196dafe140fea5803e9dfe7535c1 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 04:45:29.230806 27705 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3d01196dafe140fea5803e9dfe7535c1 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 04:45:29.230851 27705 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3d01196dafe140fea5803e9dfe7535c1 [term 0 FOLLOWER]: Advancing to term 1
I20260810 04:45:29.231349 27705 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3d01196dafe140fea5803e9dfe7535c1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3d01196dafe140fea5803e9dfe7535c1" member_type: VOTER last_known_addr { host: "127.22.167.126" port: 45241 } }
I20260810 04:45:29.231457 27705 leader_election.cc:302] T 00000000000000000000000000000000 P 3d01196dafe140fea5803e9dfe7535c1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3d01196dafe140fea5803e9dfe7535c1; no voters:
I20260810 04:45:29.231612 27705 leader_election.cc:288] T 00000000000000000000000000000000 P 3d01196dafe140fea5803e9dfe7535c1 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 04:45:29.231649 27710 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3d01196dafe140fea5803e9dfe7535c1 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 04:45:29.231818 27710 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3d01196dafe140fea5803e9dfe7535c1 [term 1 LEADER]: Becoming Leader. State: Replica: 3d01196dafe140fea5803e9dfe7535c1, State: Running, Role: LEADER
I20260810 04:45:29.231901 27705 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3d01196dafe140fea5803e9dfe7535c1 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 04:45:29.231986 27710 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3d01196dafe140fea5803e9dfe7535c1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3d01196dafe140fea5803e9dfe7535c1" member_type: VOTER last_known_addr { host: "127.22.167.126" port: 45241 } }
I20260810 04:45:29.232703 27709 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:45:29.227974 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37745 (local address 127.22.167.126:45241)
0810 04:45:29.228125 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0810 04:45:29.228132 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 04:45:29.228158 (+ 26us) server_negotiation.cc:407] Connection header received
0810 04:45:29.228547 (+ 389us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:45:29.228554 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:45:29.228596 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:45:29.228706 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0810 04:45:29.228925 (+ 219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:45:29.229780 (+ 855us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:45:29.230880 (+ 1100us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:45:29.231295 (+ 415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:45:29.231371 (+ 76us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:45:29.232063 (+ 692us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 04:45:29.232067 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 04:45:29.232068 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 04:45:29.232258 (+ 190us) server_negotiation.cc:922] calling sasl_server_start()
0810 04:45:29.232296 (+ 38us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 04:45:29.232312 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 04:45:29.232392 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0810 04:45:29.232473 (+ 81us) server_negotiation.cc:299] Negotiation successful
0810 04:45:29.232499 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":102,"thread_start_us":63,"threads_started":1}
I20260810 04:45:29.232963 27711 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3d01196dafe140fea5803e9dfe7535c1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3d01196dafe140fea5803e9dfe7535c1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3d01196dafe140fea5803e9dfe7535c1" member_type: VOTER last_known_addr { host: "127.22.167.126" port: 45241 } } }
I20260810 04:45:29.233039 27711 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3d01196dafe140fea5803e9dfe7535c1 [sys.catalog]: This master's current role is: LEADER
I20260810 04:45:29.233223 27714 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3d01196dafe140fea5803e9dfe7535c1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3d01196dafe140fea5803e9dfe7535c1. Latest consensus state: current_term: 1 leader_uuid: "3d01196dafe140fea5803e9dfe7535c1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3d01196dafe140fea5803e9dfe7535c1" member_type: VOTER last_known_addr { host: "127.22.167.126" port: 45241 } } }
I20260810 04:45:29.233238 27736 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 04:45:29.233350 27714 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3d01196dafe140fea5803e9dfe7535c1 [sys.catalog]: This master's current role is: LEADER
I20260810 04:45:29.233830 27736 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 04:45:29.234025 23197 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 04:45:29.235353 27736 catalog_manager.cc:1347] Generated new cluster ID: e20f790abf114a36a650e49e20ada05d
I20260810 04:45:29.235402 27736 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 04:45:29.238404 27709 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:45:29.235372 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33437 (local address 127.22.167.126:45241)
0810 04:45:29.235402 (+ 30us) server_negotiation.cc:206] Beginning negotiation
0810 04:45:29.235404 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 04:45:29.235657 (+ 253us) server_negotiation.cc:407] Connection header received
0810 04:45:29.235744 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:45:29.235745 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:45:29.235754 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:45:29.235781 (+ 27us) server_negotiation.cc:226] Negotiated authn=SASL
0810 04:45:29.235974 (+ 193us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:45:29.236684 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:45:29.237354 (+ 670us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:45:29.237562 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:45:29.237613 (+ 51us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:45:29.238187 (+ 574us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 04:45:29.238188 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 04:45:29.238188 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 04:45:29.238192 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0810 04:45:29.238205 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 04:45:29.238209 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 04:45:29.238249 (+ 40us) server_negotiation.cc:1035] Waiting for connection context
0810 04:45:29.238327 (+ 78us) server_negotiation.cc:299] Negotiation successful
0810 04:45:29.238335 (+ 8us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15}
I20260810 04:45:29.257423 27736 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 04:45:29.257928 27736 catalog_manager.cc:1504] Loading token signing keys...
I20260810 04:45:29.262674 27736 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3d01196dafe140fea5803e9dfe7535c1: Generated new TSK 0
I20260810 04:45:29.262846 27736 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 04:45:29.267705 23197 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskemv_AM/build/release/bin/kudu with pid 27576
W20260810 04:45:29.297402 27551 connection.cc:535] client connection to 127.22.167.126:45241 recv error: Network error: failed to read from TLS socket (remote: 127.22.167.126:45241): Connection reset by peer (error 104)
2026-08-10T04:45:29Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/13 (1449 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/14
Loading random data
Initializing database '/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786336863388938-23197-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 04:45:29 dist-test-slave-bqcl krb5kdc[27757](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 04:45:29 dist-test-slave-bqcl krb5kdc[27757](info): set up 2 sockets
Aug 10 04:45:29 dist-test-slave-bqcl krb5kdc[27757](info): commencing operation
krb5kdc: starting...
W20260810 04:45:30.432700 23197 mini_kdc.cc:121] Time spent starting KDC: real 1.122s user 0.002s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786336863388938-23197-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786336863388938-23197-0/krb5kdc/test-user.keytab.
Aug 10 04:45:30 dist-test-slave-bqcl krb5kdc[27757](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786337130, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T04:45:30Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T04:45:30Z Disabled control of system clock
WARNING: no policy specified for kudu/127.22.167.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.167.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.167.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786336863388938-23197-0/krb5kdc/kudu_127.22.167.126.keytab.
Entry for principal kudu/127.22.167.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786336863388938-23197-0/krb5kdc/kudu_127.22.167.126.keytab.
I20260810 04:45:30.544181 23197 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskemv_AM/build/release/bin/kudu
/tmp/dist-test-taskemv_AM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786336863388938-23197-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786336863388938-23197-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786336863388938-23197-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786336863388938-23197-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.167.126:36175
--webserver_interface=127.22.167.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786336863388938-23197-0/krb5kdc/kudu_127.22.167.126.keytab
--principal=kudu/127.22.167.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.167.84:35785
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.167.126:36175
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786336863388938-23197-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786336863388938-23197-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786336863388938-23197-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 04:45:30.622288 27781 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 04:45:30.622475 27781 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 04:45:30.622500 27781 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 04:45:30.623600 27781 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 04:45:30.623641 27781 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 04:45:30.623653 27781 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 04:45:30.623667 27781 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 04:45:30.623689 27781 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 04:45:30.625105 27781 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.167.84:35785
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786336863388938-23197-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786336863388938-23197-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.22.167.126:36175
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786336863388938-23197-0/krb5kdc/kudu_127.22.167.126.keytab
--principal=kudu/127.22.167.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.167.126:36175
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786336863388938-23197-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.167.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27781
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786336863388938-23197-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 03:54:01 UTC on ccycloud.quasar-icijmt.root.comops.site
build id 4577009
I20260810 04:45:30.625326 27781 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 04:45:30.625520 27781 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 04:45:30.629982 27790 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 04:45:30.630002 27781 server_base.cc:1034] running on GCE node
W20260810 04:45:30.630003 27791 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 04:45:30.629980 27796 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 04:45:30.630416 27781 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 04:45:30.630712 27781 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 04:45:30.631860 27781 hybrid_clock.cc:648] HybridClock initialized: now 1786337130631842 us; error 36 us; skew 500 ppm
Aug 10 04:45:30 dist-test-slave-bqcl krb5kdc[27757](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786337130, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.167.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 04:45:30.634110 27781 init.cc:377] Logged in from keytab as kudu/127.22.167.126@KRBTEST.COM (short username kudu)
I20260810 04:45:30.634531 27781 webserver.cc:460] Webserver started at http://127.22.167.126:46483/ using document root <none> and password file <none>
I20260810 04:45:30.634698 27781 fs_manager.cc:359] Metadata directory not provided
I20260810 04:45:30.634735 27781 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 04:45:30.634824 27781 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 04:45:30.635746 27781 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786336863388938-23197-0/minicluster-data/master-0/data/instance:
uuid: "6c74e6beb01d404c9645ab2b1f8d1ed9"
format_stamp: "Formatted at 2026-08-10 04:45:30 on dist-test-slave-bqcl"
server_key: "56adfe35200061ec4ea4e790274aea0a"
server_key_iv: "9efa5f59b7d57e5aeb8b18eeebf43484"
server_key_version: "encryptionkey@0"
I20260810 04:45:30.636023 27781 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786336863388938-23197-0/minicluster-data/master-0/wal/instance:
uuid: "6c74e6beb01d404c9645ab2b1f8d1ed9"
format_stamp: "Formatted at 2026-08-10 04:45:30 on dist-test-slave-bqcl"
server_key: "56adfe35200061ec4ea4e790274aea0a"
server_key_iv: "9efa5f59b7d57e5aeb8b18eeebf43484"
server_key_version: "encryptionkey@0"
I20260810 04:45:30.637197 27781 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260810 04:45:30.637890 27812 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 04:45:30.638052 27781 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260810 04:45:30.638155 27781 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786336863388938-23197-0/minicluster-data/master-0/data,/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786336863388938-23197-0/minicluster-data/master-0/wal
uuid: "6c74e6beb01d404c9645ab2b1f8d1ed9"
format_stamp: "Formatted at 2026-08-10 04:45:30 on dist-test-slave-bqcl"
server_key: "56adfe35200061ec4ea4e790274aea0a"
server_key_iv: "9efa5f59b7d57e5aeb8b18eeebf43484"
server_key_version: "encryptionkey@0"
I20260810 04:45:30.638228 27781 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786336863388938-23197-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786336863388938-23197-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786336863388938-23197-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 04:45:30.641328 27781 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 04:45:30.643114 27781 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 04:45:30.643236 27781 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 04:45:30.647097 27781 rpc_server.cc:307] RPC server started. Bound to: 127.22.167.126:36175
I20260810 04:45:30.647133 27905 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.167.126:36175 every 8 connection(s)
I20260810 04:45:30.647410 27781 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786336863388938-23197-0/minicluster-data/master-0/data/info.pb
I20260810 04:45:30.648074 23197 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskemv_AM/build/release/bin/kudu as pid 27781
I20260810 04:45:30.648197 23197 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786336863388938-23197-0/minicluster-data/master-0/wal/instance
I20260810 04:45:30.648343 27907 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 04:45:30.648370 23197 external_mini_cluster.cc:1428] Setting key 7c87d41f0a2a4bc6648ecdba0d60c020
I20260810 04:45:30.650897 27911 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:45:30.648813 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34677 (local address 127.22.167.126:36175)
0810 04:45:30.649003 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0810 04:45:30.649012 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 04:45:30.649046 (+ 34us) server_negotiation.cc:407] Connection header received
0810 04:45:30.649751 (+ 705us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:45:30.649760 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:45:30.649794 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:45:30.649941 (+ 147us) server_negotiation.cc:226] Negotiated authn=SASL
0810 04:45:30.650505 (+ 564us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 04:45:30.650508 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 04:45:30.650509 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 04:45:30.650513 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0810 04:45:30.650554 (+ 41us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 04:45:30.650567 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 04:45:30.650656 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0810 04:45:30.650687 (+ 31us) server_negotiation.cc:299] Negotiation successful
0810 04:45:30.650723 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":138,"thread_start_us":95,"threads_started":1}
I20260810 04:45:30.651494 27907 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6c74e6beb01d404c9645ab2b1f8d1ed9: Bootstrap starting.
I20260810 04:45:30.651803 23197 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 04:45:30.652115 27907 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6c74e6beb01d404c9645ab2b1f8d1ed9: Neither blocks nor log segments found. Creating new log.
I20260810 04:45:30.652364 27907 log.cc:824] T 00000000000000000000000000000000 P 6c74e6beb01d404c9645ab2b1f8d1ed9: Log is configured to *not* fsync() on all Append() calls
I20260810 04:45:30.653020 27907 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6c74e6beb01d404c9645ab2b1f8d1ed9: No bootstrap required, opened a new log
I20260810 04:45:30.653937 27911 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:45:30.653068 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50379 (local address 127.22.167.126:36175)
0810 04:45:30.653119 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0810 04:45:30.653121 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 04:45:30.653210 (+ 89us) server_negotiation.cc:407] Connection header received
0810 04:45:30.653253 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:45:30.653254 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:45:30.653258 (+ 4us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:45:30.653326 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0810 04:45:30.653761 (+ 435us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 04:45:30.653762 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 04:45:30.653762 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 04:45:30.653766 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0810 04:45:30.653778 (+ 12us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 04:45:30.653782 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 04:45:30.653839 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0810 04:45:30.653871 (+ 32us) server_negotiation.cc:299] Negotiation successful
0810 04:45:30.653881 (+ 10us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260810 04:45:30.655064 27907 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6c74e6beb01d404c9645ab2b1f8d1ed9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6c74e6beb01d404c9645ab2b1f8d1ed9" member_type: VOTER last_known_addr { host: "127.22.167.126" port: 36175 } }
I20260810 04:45:30.655200 27907 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6c74e6beb01d404c9645ab2b1f8d1ed9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 04:45:30.655218 27907 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6c74e6beb01d404c9645ab2b1f8d1ed9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6c74e6beb01d404c9645ab2b1f8d1ed9, State: Initialized, Role: FOLLOWER
I20260810 04:45:30.655294 27907 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6c74e6beb01d404c9645ab2b1f8d1ed9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6c74e6beb01d404c9645ab2b1f8d1ed9" member_type: VOTER last_known_addr { host: "127.22.167.126" port: 36175 } }
I20260810 04:45:30.655349 27907 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6c74e6beb01d404c9645ab2b1f8d1ed9 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 04:45:30.655373 27907 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6c74e6beb01d404c9645ab2b1f8d1ed9 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 04:45:30.655401 27907 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6c74e6beb01d404c9645ab2b1f8d1ed9 [term 0 FOLLOWER]: Advancing to term 1
I20260810 04:45:30.655858 27907 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6c74e6beb01d404c9645ab2b1f8d1ed9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6c74e6beb01d404c9645ab2b1f8d1ed9" member_type: VOTER last_known_addr { host: "127.22.167.126" port: 36175 } }
I20260810 04:45:30.655942 27907 leader_election.cc:302] T 00000000000000000000000000000000 P 6c74e6beb01d404c9645ab2b1f8d1ed9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6c74e6beb01d404c9645ab2b1f8d1ed9; no voters:
I20260810 04:45:30.656054 27907 leader_election.cc:288] T 00000000000000000000000000000000 P 6c74e6beb01d404c9645ab2b1f8d1ed9 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 04:45:30.656105 27927 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6c74e6beb01d404c9645ab2b1f8d1ed9 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 04:45:30.656252 27907 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6c74e6beb01d404c9645ab2b1f8d1ed9 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 04:45:30.656261 27927 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6c74e6beb01d404c9645ab2b1f8d1ed9 [term 1 LEADER]: Becoming Leader. State: Replica: 6c74e6beb01d404c9645ab2b1f8d1ed9, State: Running, Role: LEADER
I20260810 04:45:30.656363 27927 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6c74e6beb01d404c9645ab2b1f8d1ed9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6c74e6beb01d404c9645ab2b1f8d1ed9" member_type: VOTER last_known_addr { host: "127.22.167.126" port: 36175 } }
I20260810 04:45:30.656740 27928 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6c74e6beb01d404c9645ab2b1f8d1ed9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6c74e6beb01d404c9645ab2b1f8d1ed9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6c74e6beb01d404c9645ab2b1f8d1ed9" member_type: VOTER last_known_addr { host: "127.22.167.126" port: 36175 } } }
I20260810 04:45:30.656740 27930 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6c74e6beb01d404c9645ab2b1f8d1ed9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6c74e6beb01d404c9645ab2b1f8d1ed9. Latest consensus state: current_term: 1 leader_uuid: "6c74e6beb01d404c9645ab2b1f8d1ed9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6c74e6beb01d404c9645ab2b1f8d1ed9" member_type: VOTER last_known_addr { host: "127.22.167.126" port: 36175 } } }
I20260810 04:45:30.656845 27928 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6c74e6beb01d404c9645ab2b1f8d1ed9 [sys.catalog]: This master's current role is: LEADER
I20260810 04:45:30.656865 27930 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6c74e6beb01d404c9645ab2b1f8d1ed9 [sys.catalog]: This master's current role is: LEADER
I20260810 04:45:30.657049 27939 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 04:45:30.657531 27939 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 04:45:30.659075 27939 catalog_manager.cc:1347] Generated new cluster ID: b1e9d705b5394049a7a93e1d7445e841
I20260810 04:45:30.659122 27939 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 04:45:30.669432 27939 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 04:45:30.669569 27939 catalog_manager.cc:1504] Loading token signing keys...
I20260810 04:45:30.672125 27939 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6c74e6beb01d404c9645ab2b1f8d1ed9: Generated new TSK 0
I20260810 04:45:30.672274 27939 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 04:45:30.674336 23197 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskemv_AM/build/release/bin/kudu with pid 27781
2026-08-10T04:45:30Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/14 (1390 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/15
Loading random data
Initializing database '/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786336863388938-23197-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 04:45:30 dist-test-slave-bqcl krb5kdc[27961](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 04:45:30 dist-test-slave-bqcl krb5kdc[27961](info): set up 2 sockets
Aug 10 04:45:30 dist-test-slave-bqcl krb5kdc[27961](info): commencing operation
krb5kdc: starting...
W20260810 04:45:31.838452 23197 mini_kdc.cc:121] Time spent starting KDC: real 1.137s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786336863388938-23197-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786336863388938-23197-0/krb5kdc/test-user.keytab.
Aug 10 04:45:31 dist-test-slave-bqcl krb5kdc[27961](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786337131, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T04:45:31Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T04:45:31Z Disabled control of system clock
WARNING: no policy specified for kudu/127.22.167.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.167.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.167.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786336863388938-23197-0/krb5kdc/kudu_127.22.167.126.keytab.
Entry for principal kudu/127.22.167.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786336863388938-23197-0/krb5kdc/kudu_127.22.167.126.keytab.
I20260810 04:45:31.950827 23197 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskemv_AM/build/release/bin/kudu
/tmp/dist-test-taskemv_AM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786336863388938-23197-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786336863388938-23197-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786336863388938-23197-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786336863388938-23197-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.167.126:40475
--webserver_interface=127.22.167.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786336863388938-23197-0/krb5kdc/kudu_127.22.167.126.keytab
--principal=kudu/127.22.167.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.167.84:34843
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.167.126:40475
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786336863388938-23197-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786336863388938-23197-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786336863388938-23197-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 04:45:32.031315 27981 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 04:45:32.031505 27981 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 04:45:32.031536 27981 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 04:45:32.032629 27981 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 04:45:32.032671 27981 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 04:45:32.032691 27981 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 04:45:32.032707 27981 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 04:45:32.032722 27981 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 04:45:32.034176 27981 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.167.84:34843
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786336863388938-23197-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786336863388938-23197-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.22.167.126:40475
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786336863388938-23197-0/krb5kdc/kudu_127.22.167.126.keytab
--principal=kudu/127.22.167.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.167.126:40475
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786336863388938-23197-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.167.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27981
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786336863388938-23197-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 03:54:01 UTC on ccycloud.quasar-icijmt.root.comops.site
build id 4577009
I20260810 04:45:32.034396 27981 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 04:45:32.034590 27981 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 04:45:32.038892 27991 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 04:45:32.038956 27989 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 04:45:32.038899 27988 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 04:45:32.039278 27981 server_base.cc:1034] running on GCE node
I20260810 04:45:32.039507 27981 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 04:45:32.039770 27981 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 04:45:32.040922 27981 hybrid_clock.cc:648] HybridClock initialized: now 1786337132040907 us; error 33 us; skew 500 ppm
Aug 10 04:45:32 dist-test-slave-bqcl krb5kdc[27961](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786337132, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.167.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 04:45:32.043299 27981 init.cc:377] Logged in from keytab as kudu/127.22.167.126@KRBTEST.COM (short username kudu)
I20260810 04:45:32.043695 27981 webserver.cc:460] Webserver started at http://127.22.167.126:36033/ using document root <none> and password file <none>
I20260810 04:45:32.043879 27981 fs_manager.cc:359] Metadata directory not provided
I20260810 04:45:32.043915 27981 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 04:45:32.044003 27981 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 04:45:32.044920 27981 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786336863388938-23197-0/minicluster-data/master-0/data/instance:
uuid: "285f6f0d15f14b2b9fc52d919c4dbcd5"
format_stamp: "Formatted at 2026-08-10 04:45:32 on dist-test-slave-bqcl"
server_key: "d4228dfec255efc6146a8ab0a14041f0"
server_key_iv: "ffb0a9f1fa26173f9355ce7f4077b71a"
server_key_version: "encryptionkey@0"
I20260810 04:45:32.045200 27981 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786336863388938-23197-0/minicluster-data/master-0/wal/instance:
uuid: "285f6f0d15f14b2b9fc52d919c4dbcd5"
format_stamp: "Formatted at 2026-08-10 04:45:32 on dist-test-slave-bqcl"
server_key: "d4228dfec255efc6146a8ab0a14041f0"
server_key_iv: "ffb0a9f1fa26173f9355ce7f4077b71a"
server_key_version: "encryptionkey@0"
I20260810 04:45:32.046399 27981 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260810 04:45:32.047082 28003 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 04:45:32.047245 27981 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260810 04:45:32.047326 27981 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786336863388938-23197-0/minicluster-data/master-0/data,/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786336863388938-23197-0/minicluster-data/master-0/wal
uuid: "285f6f0d15f14b2b9fc52d919c4dbcd5"
format_stamp: "Formatted at 2026-08-10 04:45:32 on dist-test-slave-bqcl"
server_key: "d4228dfec255efc6146a8ab0a14041f0"
server_key_iv: "ffb0a9f1fa26173f9355ce7f4077b71a"
server_key_version: "encryptionkey@0"
I20260810 04:45:32.047379 27981 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786336863388938-23197-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786336863388938-23197-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786336863388938-23197-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 04:45:32.061213 27981 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 04:45:32.063019 27981 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 04:45:32.063139 27981 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 04:45:32.066856 27981 rpc_server.cc:307] RPC server started. Bound to: 127.22.167.126:40475
I20260810 04:45:32.066922 28091 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.167.126:40475 every 8 connection(s)
I20260810 04:45:32.067214 27981 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786336863388938-23197-0/minicluster-data/master-0/data/info.pb
I20260810 04:45:32.068017 28094 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 04:45:32.070206 28094 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 285f6f0d15f14b2b9fc52d919c4dbcd5: Bootstrap starting.
I20260810 04:45:32.070756 28094 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 285f6f0d15f14b2b9fc52d919c4dbcd5: Neither blocks nor log segments found. Creating new log.
I20260810 04:45:32.071012 28094 log.cc:824] T 00000000000000000000000000000000 P 285f6f0d15f14b2b9fc52d919c4dbcd5: Log is configured to *not* fsync() on all Append() calls
I20260810 04:45:32.071650 28094 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 285f6f0d15f14b2b9fc52d919c4dbcd5: No bootstrap required, opened a new log
I20260810 04:45:32.073184 28094 raft_consensus.cc:348] T 00000000000000000000000000000000 P 285f6f0d15f14b2b9fc52d919c4dbcd5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "285f6f0d15f14b2b9fc52d919c4dbcd5" member_type: VOTER last_known_addr { host: "127.22.167.126" port: 40475 } }
I20260810 04:45:32.073311 28094 raft_consensus.cc:374] T 00000000000000000000000000000000 P 285f6f0d15f14b2b9fc52d919c4dbcd5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 04:45:32.073342 28094 raft_consensus.cc:729] T 00000000000000000000000000000000 P 285f6f0d15f14b2b9fc52d919c4dbcd5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 285f6f0d15f14b2b9fc52d919c4dbcd5, State: Initialized, Role: FOLLOWER
I20260810 04:45:32.073468 28094 consensus_queue.cc:260] T 00000000000000000000000000000000 P 285f6f0d15f14b2b9fc52d919c4dbcd5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "285f6f0d15f14b2b9fc52d919c4dbcd5" member_type: VOTER last_known_addr { host: "127.22.167.126" port: 40475 } }
I20260810 04:45:32.073546 28094 raft_consensus.cc:388] T 00000000000000000000000000000000 P 285f6f0d15f14b2b9fc52d919c4dbcd5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 04:45:32.073578 28094 raft_consensus.cc:482] T 00000000000000000000000000000000 P 285f6f0d15f14b2b9fc52d919c4dbcd5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 04:45:32.073626 28094 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 285f6f0d15f14b2b9fc52d919c4dbcd5 [term 0 FOLLOWER]: Advancing to term 1
I20260810 04:45:32.074437 28094 raft_consensus.cc:504] T 00000000000000000000000000000000 P 285f6f0d15f14b2b9fc52d919c4dbcd5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "285f6f0d15f14b2b9fc52d919c4dbcd5" member_type: VOTER last_known_addr { host: "127.22.167.126" port: 40475 } }
I20260810 04:45:32.074530 28094 leader_election.cc:302] T 00000000000000000000000000000000 P 285f6f0d15f14b2b9fc52d919c4dbcd5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 285f6f0d15f14b2b9fc52d919c4dbcd5; no voters:
I20260810 04:45:32.074689 28094 leader_election.cc:288] T 00000000000000000000000000000000 P 285f6f0d15f14b2b9fc52d919c4dbcd5 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 04:45:32.074723 28097 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 285f6f0d15f14b2b9fc52d919c4dbcd5 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 04:45:32.074860 28097 raft_consensus.cc:686] T 00000000000000000000000000000000 P 285f6f0d15f14b2b9fc52d919c4dbcd5 [term 1 LEADER]: Becoming Leader. State: Replica: 285f6f0d15f14b2b9fc52d919c4dbcd5, State: Running, Role: LEADER
I20260810 04:45:32.074919 28094 sys_catalog.cc:564] T 00000000000000000000000000000000 P 285f6f0d15f14b2b9fc52d919c4dbcd5 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 04:45:32.074965 28097 consensus_queue.cc:237] T 00000000000000000000000000000000 P 285f6f0d15f14b2b9fc52d919c4dbcd5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "285f6f0d15f14b2b9fc52d919c4dbcd5" member_type: VOTER last_known_addr { host: "127.22.167.126" port: 40475 } }
I20260810 04:45:32.075122 23197 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskemv_AM/build/release/bin/kudu as pid 27981
I20260810 04:45:32.075242 23197 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786336863388938-23197-0/minicluster-data/master-0/wal/instance
I20260810 04:45:32.075430 23197 external_mini_cluster.cc:1428] Setting key fe08a7d4e87fc5ec3e40a09a8b6a6bda
I20260810 04:45:32.075981 28099 sys_catalog.cc:455] T 00000000000000000000000000000000 P 285f6f0d15f14b2b9fc52d919c4dbcd5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "285f6f0d15f14b2b9fc52d919c4dbcd5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "285f6f0d15f14b2b9fc52d919c4dbcd5" member_type: VOTER last_known_addr { host: "127.22.167.126" port: 40475 } } }
I20260810 04:45:32.076077 28099 sys_catalog.cc:458] T 00000000000000000000000000000000 P 285f6f0d15f14b2b9fc52d919c4dbcd5 [sys.catalog]: This master's current role is: LEADER
I20260810 04:45:32.076289 28102 sys_catalog.cc:455] T 00000000000000000000000000000000 P 285f6f0d15f14b2b9fc52d919c4dbcd5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 285f6f0d15f14b2b9fc52d919c4dbcd5. Latest consensus state: current_term: 1 leader_uuid: "285f6f0d15f14b2b9fc52d919c4dbcd5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "285f6f0d15f14b2b9fc52d919c4dbcd5" member_type: VOTER last_known_addr { host: "127.22.167.126" port: 40475 } } }
I20260810 04:45:32.076597 28102 sys_catalog.cc:458] T 00000000000000000000000000000000 P 285f6f0d15f14b2b9fc52d919c4dbcd5 [sys.catalog]: This master's current role is: LEADER
W20260810 04:45:32.076779 28134 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 285f6f0d15f14b2b9fc52d919c4dbcd5: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260810 04:45:32.076818 28134 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260810 04:45:32.077338 28133 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 04:45:32.077526 28133 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 04:45:32.079006 28133 catalog_manager.cc:1347] Generated new cluster ID: 5d20eb34697940fc8068e5c8d1ad5d2c
I20260810 04:45:32.079059 28133 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 10 04:45:32 dist-test-slave-bqcl krb5kdc[27961](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786337131, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.167.126@KRBTEST.COM
I20260810 04:45:32.085438 28128 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:45:32.076042 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45767 (local address 127.22.167.126:40475)
0810 04:45:32.076211 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0810 04:45:32.076220 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 04:45:32.076254 (+ 34us) server_negotiation.cc:407] Connection header received
0810 04:45:32.076371 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:45:32.076378 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:45:32.076447 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:45:32.076604 (+ 157us) server_negotiation.cc:226] Negotiated authn=SASL
0810 04:45:32.077336 (+ 732us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:45:32.078476 (+ 1140us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:45:32.079415 (+ 939us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:45:32.079827 (+ 412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:45:32.079918 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:45:32.082051 (+ 2133us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 04:45:32.082055 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 04:45:32.082056 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 04:45:32.082067 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0810 04:45:32.084067 (+ 2000us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 04:45:32.084382 (+ 315us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 04:45:32.084384 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 04:45:32.084385 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 04:45:32.084483 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 04:45:32.084665 (+ 182us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 04:45:32.084666 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 04:45:32.084667 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 04:45:32.084896 (+ 229us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 04:45:32.085057 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0810 04:45:32.085123 (+ 66us) server_negotiation.cc:299] Negotiation successful
0810 04:45:32.085266 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":123,"thread_start_us":70,"threads_started":1}
I20260810 04:45:32.086081 28133 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 04:45:32.086392 23197 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 04:45:32.086706 28133 catalog_manager.cc:1504] Loading token signing keys...
I20260810 04:45:32.093008 28128 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:45:32.087783 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50491 (local address 127.22.167.126:40475)
0810 04:45:32.087816 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0810 04:45:32.087818 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 04:45:32.087931 (+ 113us) server_negotiation.cc:407] Connection header received
0810 04:45:32.087983 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:45:32.087984 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:45:32.087993 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:45:32.088040 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0810 04:45:32.088488 (+ 448us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:45:32.088969 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:45:32.089628 (+ 659us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:45:32.089828 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:45:32.089880 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:45:32.090914 (+ 1034us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 04:45:32.090915 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 04:45:32.090915 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 04:45:32.090921 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0810 04:45:32.092095 (+ 1174us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 04:45:32.092381 (+ 286us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 04:45:32.092382 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 04:45:32.092382 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 04:45:32.092434 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 04:45:32.092590 (+ 156us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 04:45:32.092591 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 04:45:32.092591 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 04:45:32.092686 (+ 95us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 04:45:32.092777 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0810 04:45:32.092860 (+ 83us) server_negotiation.cc:299] Negotiation successful
0810 04:45:32.092928 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260810 04:45:32.100244 28133 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 285f6f0d15f14b2b9fc52d919c4dbcd5: Generated new TSK 0
I20260810 04:45:32.100409 28133 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 04:45:32.102370 23197 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskemv_AM/build/release/bin/kudu with pid 27981
2026-08-10T04:45:32Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/15 (1426 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/16
Loading random data
Initializing database '/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786336863388938-23197-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 04:45:32 dist-test-slave-bqcl krb5kdc[28151](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 04:45:32 dist-test-slave-bqcl krb5kdc[28151](info): set up 2 sockets
Aug 10 04:45:32 dist-test-slave-bqcl krb5kdc[28151](info): commencing operation
krb5kdc: starting...
W20260810 04:45:33.268302 23197 mini_kdc.cc:121] Time spent starting KDC: real 1.140s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786336863388938-23197-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786336863388938-23197-0/krb5kdc/test-user.keytab.
Aug 10 04:45:33 dist-test-slave-bqcl krb5kdc[28151](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786337133, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T04:45:33Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T04:45:33Z Disabled control of system clock
WARNING: no policy specified for kudu/127.22.167.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.167.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.167.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786336863388938-23197-0/krb5kdc/kudu_127.22.167.126.keytab.
Entry for principal kudu/127.22.167.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786336863388938-23197-0/krb5kdc/kudu_127.22.167.126.keytab.
I20260810 04:45:33.378584 23197 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskemv_AM/build/release/bin/kudu
/tmp/dist-test-taskemv_AM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786336863388938-23197-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786336863388938-23197-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786336863388938-23197-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786336863388938-23197-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.167.126:44153
--webserver_interface=127.22.167.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786336863388938-23197-0/krb5kdc/kudu_127.22.167.126.keytab
--principal=kudu/127.22.167.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.167.84:44225
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.167.126:44153
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786336863388938-23197-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786336863388938-23197-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786336863388938-23197-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 04:45:33.456732 28171 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 04:45:33.456913 28171 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 04:45:33.456934 28171 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 04:45:33.458065 28171 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 04:45:33.458105 28171 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 04:45:33.458117 28171 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 04:45:33.458158 28171 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 04:45:33.458184 28171 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 04:45:33.459578 28171 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.167.84:44225
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786336863388938-23197-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786336863388938-23197-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.22.167.126:44153
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786336863388938-23197-0/krb5kdc/kudu_127.22.167.126.keytab
--principal=kudu/127.22.167.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.167.126:44153
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786336863388938-23197-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.167.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28171
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786336863388938-23197-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 03:54:01 UTC on ccycloud.quasar-icijmt.root.comops.site
build id 4577009
I20260810 04:45:33.459787 28171 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 04:45:33.459978 28171 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 04:45:33.464375 28188 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 04:45:33.464479 28171 server_base.cc:1034] running on GCE node
W20260810 04:45:33.464474 28193 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 04:45:33.464397 28187 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 04:45:33.464874 28171 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 04:45:33.465114 28171 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 04:45:33.466259 28171 hybrid_clock.cc:648] HybridClock initialized: now 1786337133466239 us; error 35 us; skew 500 ppm
Aug 10 04:45:33 dist-test-slave-bqcl krb5kdc[28151](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786337133, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.167.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 04:45:33.471567 28171 init.cc:377] Logged in from keytab as kudu/127.22.167.126@KRBTEST.COM (short username kudu)
I20260810 04:45:33.471966 28171 webserver.cc:460] Webserver started at http://127.22.167.126:40417/ using document root <none> and password file <none>
I20260810 04:45:33.472162 28171 fs_manager.cc:359] Metadata directory not provided
I20260810 04:45:33.472205 28171 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 04:45:33.472296 28171 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 04:45:33.473268 28171 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786336863388938-23197-0/minicluster-data/master-0/data/instance:
uuid: "9e62171503174f2bbbdeba77d97e3ea9"
format_stamp: "Formatted at 2026-08-10 04:45:33 on dist-test-slave-bqcl"
server_key: "0e9a16d160540feef505eda86447745f"
server_key_iv: "a512df862ba1653f1c28d66cad3b0e4b"
server_key_version: "encryptionkey@0"
I20260810 04:45:33.473562 28171 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786336863388938-23197-0/minicluster-data/master-0/wal/instance:
uuid: "9e62171503174f2bbbdeba77d97e3ea9"
format_stamp: "Formatted at 2026-08-10 04:45:33 on dist-test-slave-bqcl"
server_key: "0e9a16d160540feef505eda86447745f"
server_key_iv: "a512df862ba1653f1c28d66cad3b0e4b"
server_key_version: "encryptionkey@0"
I20260810 04:45:33.474812 28171 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260810 04:45:33.475467 28202 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 04:45:33.475641 28171 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260810 04:45:33.475729 28171 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786336863388938-23197-0/minicluster-data/master-0/data,/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786336863388938-23197-0/minicluster-data/master-0/wal
uuid: "9e62171503174f2bbbdeba77d97e3ea9"
format_stamp: "Formatted at 2026-08-10 04:45:33 on dist-test-slave-bqcl"
server_key: "0e9a16d160540feef505eda86447745f"
server_key_iv: "a512df862ba1653f1c28d66cad3b0e4b"
server_key_version: "encryptionkey@0"
I20260810 04:45:33.475783 28171 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786336863388938-23197-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786336863388938-23197-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786336863388938-23197-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 04:45:33.490061 28171 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 04:45:33.491768 28171 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 04:45:33.491880 28171 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 04:45:33.495731 28171 rpc_server.cc:307] RPC server started. Bound to: 127.22.167.126:44153
I20260810 04:45:33.495769 28297 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.167.126:44153 every 8 connection(s)
I20260810 04:45:33.496076 28171 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786336863388938-23197-0/minicluster-data/master-0/data/info.pb
I20260810 04:45:33.496877 28298 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 04:45:33.499116 28298 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9e62171503174f2bbbdeba77d97e3ea9: Bootstrap starting.
I20260810 04:45:33.499647 28298 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9e62171503174f2bbbdeba77d97e3ea9: Neither blocks nor log segments found. Creating new log.
I20260810 04:45:33.499909 28298 log.cc:824] T 00000000000000000000000000000000 P 9e62171503174f2bbbdeba77d97e3ea9: Log is configured to *not* fsync() on all Append() calls
I20260810 04:45:33.500586 28298 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9e62171503174f2bbbdeba77d97e3ea9: No bootstrap required, opened a new log
I20260810 04:45:33.502197 28298 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9e62171503174f2bbbdeba77d97e3ea9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9e62171503174f2bbbdeba77d97e3ea9" member_type: VOTER last_known_addr { host: "127.22.167.126" port: 44153 } }
I20260810 04:45:33.502333 28298 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9e62171503174f2bbbdeba77d97e3ea9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 04:45:33.502355 28298 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9e62171503174f2bbbdeba77d97e3ea9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e62171503174f2bbbdeba77d97e3ea9, State: Initialized, Role: FOLLOWER
I20260810 04:45:33.502475 28298 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9e62171503174f2bbbdeba77d97e3ea9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9e62171503174f2bbbdeba77d97e3ea9" member_type: VOTER last_known_addr { host: "127.22.167.126" port: 44153 } }
I20260810 04:45:33.502537 28298 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9e62171503174f2bbbdeba77d97e3ea9 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 04:45:33.502558 28298 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9e62171503174f2bbbdeba77d97e3ea9 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 04:45:33.502570 23197 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskemv_AM/build/release/bin/kudu as pid 28171
I20260810 04:45:33.502604 28298 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9e62171503174f2bbbdeba77d97e3ea9 [term 0 FOLLOWER]: Advancing to term 1
I20260810 04:45:33.502694 23197 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786336863388938-23197-0/minicluster-data/master-0/wal/instance
I20260810 04:45:33.502873 23197 external_mini_cluster.cc:1428] Setting key 24b03cfb4a7e25c4df2fc7824e6d5e75
I20260810 04:45:33.503589 28298 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9e62171503174f2bbbdeba77d97e3ea9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9e62171503174f2bbbdeba77d97e3ea9" member_type: VOTER last_known_addr { host: "127.22.167.126" port: 44153 } }
I20260810 04:45:33.503696 28298 leader_election.cc:302] T 00000000000000000000000000000000 P 9e62171503174f2bbbdeba77d97e3ea9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9e62171503174f2bbbdeba77d97e3ea9; no voters:
I20260810 04:45:33.503849 28298 leader_election.cc:288] T 00000000000000000000000000000000 P 9e62171503174f2bbbdeba77d97e3ea9 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 04:45:33.503887 28308 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9e62171503174f2bbbdeba77d97e3ea9 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 04:45:33.504074 28308 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9e62171503174f2bbbdeba77d97e3ea9 [term 1 LEADER]: Becoming Leader. State: Replica: 9e62171503174f2bbbdeba77d97e3ea9, State: Running, Role: LEADER
I20260810 04:45:33.504182 28298 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9e62171503174f2bbbdeba77d97e3ea9 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 04:45:33.504231 28308 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9e62171503174f2bbbdeba77d97e3ea9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9e62171503174f2bbbdeba77d97e3ea9" member_type: VOTER last_known_addr { host: "127.22.167.126" port: 44153 } }
I20260810 04:45:33.504570 28308 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9e62171503174f2bbbdeba77d97e3ea9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9e62171503174f2bbbdeba77d97e3ea9. Latest consensus state: current_term: 1 leader_uuid: "9e62171503174f2bbbdeba77d97e3ea9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9e62171503174f2bbbdeba77d97e3ea9" member_type: VOTER last_known_addr { host: "127.22.167.126" port: 44153 } } }
I20260810 04:45:33.504624 28312 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9e62171503174f2bbbdeba77d97e3ea9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9e62171503174f2bbbdeba77d97e3ea9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9e62171503174f2bbbdeba77d97e3ea9" member_type: VOTER last_known_addr { host: "127.22.167.126" port: 44153 } } }
I20260810 04:45:33.504688 28308 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9e62171503174f2bbbdeba77d97e3ea9 [sys.catalog]: This master's current role is: LEADER
I20260810 04:45:33.504693 28312 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9e62171503174f2bbbdeba77d97e3ea9 [sys.catalog]: This master's current role is: LEADER
I20260810 04:45:33.504917 28323 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 04:45:33.505628 28323 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 04:45:33.507402 28323 catalog_manager.cc:1347] Generated new cluster ID: e307abd22e59410b8d4444cc3e61d148
I20260810 04:45:33.507457 28323 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 04:45:33.508422 28306 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:45:33.503302 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33085 (local address 127.22.167.126:44153)
0810 04:45:33.503489 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0810 04:45:33.503498 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 04:45:33.503543 (+ 45us) server_negotiation.cc:407] Connection header received
0810 04:45:33.503656 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:45:33.503664 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:45:33.503718 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:45:33.503871 (+ 153us) server_negotiation.cc:226] Negotiated authn=SASL
0810 04:45:33.504068 (+ 197us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:45:33.505607 (+ 1539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:45:33.506544 (+ 937us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:45:33.506991 (+ 447us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:45:33.507134 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:45:33.507773 (+ 639us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 04:45:33.507777 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 04:45:33.507777 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 04:45:33.507937 (+ 160us) server_negotiation.cc:922] calling sasl_server_start()
0810 04:45:33.507967 (+ 30us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 04:45:33.507978 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 04:45:33.508031 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0810 04:45:33.508116 (+ 85us) server_negotiation.cc:299] Negotiation successful
0810 04:45:33.508263 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":126,"thread_start_us":94,"threads_started":1}
I20260810 04:45:33.509279 23197 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 04:45:33.513047 28306 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:45:33.510558 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34071 (local address 127.22.167.126:44153)
0810 04:45:33.510593 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0810 04:45:33.510595 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 04:45:33.510704 (+ 109us) server_negotiation.cc:407] Connection header received
0810 04:45:33.510758 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:45:33.510759 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:45:33.510779 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:45:33.510823 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0810 04:45:33.510926 (+ 103us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:45:33.511391 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:45:33.512044 (+ 653us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:45:33.512251 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:45:33.512304 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:45:33.512846 (+ 542us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 04:45:33.512847 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 04:45:33.512848 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 04:45:33.512852 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0810 04:45:33.512864 (+ 12us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 04:45:33.512868 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 04:45:33.512909 (+ 41us) server_negotiation.cc:1035] Waiting for connection context
0810 04:45:33.512958 (+ 49us) server_negotiation.cc:299] Negotiation successful
0810 04:45:33.512982 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260810 04:45:33.519245 28323 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 04:45:33.519729 28323 catalog_manager.cc:1504] Loading token signing keys...
I20260810 04:45:33.524279 28323 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9e62171503174f2bbbdeba77d97e3ea9: Generated new TSK 0
I20260810 04:45:33.524430 28323 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 04:45:33.529325 23197 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskemv_AM/build/release/bin/kudu with pid 28171
2026-08-10T04:45:33Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/16 (1431 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/17
Loading random data
Initializing database '/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786336863388938-23197-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 04:45:33 dist-test-slave-bqcl krb5kdc[28368](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 04:45:33 dist-test-slave-bqcl krb5kdc[28368](info): set up 2 sockets
Aug 10 04:45:33 dist-test-slave-bqcl krb5kdc[28368](info): commencing operation
krb5kdc: starting...
W20260810 04:45:34.688818 23197 mini_kdc.cc:121] Time spent starting KDC: real 1.129s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786336863388938-23197-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786336863388938-23197-0/krb5kdc/test-user.keytab.
Aug 10 04:45:34 dist-test-slave-bqcl krb5kdc[28368](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786337134, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T04:45:34Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T04:45:34Z Disabled control of system clock
WARNING: no policy specified for kudu/127.22.167.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.167.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.167.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786336863388938-23197-0/krb5kdc/kudu_127.22.167.126.keytab.
Entry for principal kudu/127.22.167.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786336863388938-23197-0/krb5kdc/kudu_127.22.167.126.keytab.
I20260810 04:45:34.800505 23197 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskemv_AM/build/release/bin/kudu
/tmp/dist-test-taskemv_AM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786336863388938-23197-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786336863388938-23197-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786336863388938-23197-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786336863388938-23197-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.167.126:41651
--webserver_interface=127.22.167.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786336863388938-23197-0/krb5kdc/kudu_127.22.167.126.keytab
--principal=kudu/127.22.167.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.167.84:46131
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.167.126:41651
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786336863388938-23197-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786336863388938-23197-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786336863388938-23197-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 04:45:34.878895 28390 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 04:45:34.879074 28390 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 04:45:34.879094 28390 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 04:45:34.880144 28390 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 04:45:34.880177 28390 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 04:45:34.880189 28390 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 04:45:34.880200 28390 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 04:45:34.880213 28390 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 04:45:34.881585 28390 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.167.84:46131
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786336863388938-23197-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786336863388938-23197-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.22.167.126:41651
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786336863388938-23197-0/krb5kdc/kudu_127.22.167.126.keytab
--principal=kudu/127.22.167.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.167.126:41651
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786336863388938-23197-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.167.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28390
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786336863388938-23197-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 03:54:01 UTC on ccycloud.quasar-icijmt.root.comops.site
build id 4577009
I20260810 04:45:34.881762 28390 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 04:45:34.881920 28390 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 04:45:34.886310 28401 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 04:45:34.886396 28390 server_base.cc:1034] running on GCE node
W20260810 04:45:34.886384 28398 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 04:45:34.886363 28399 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 04:45:34.886718 28390 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 04:45:34.886976 28390 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 04:45:34.888111 28390 hybrid_clock.cc:648] HybridClock initialized: now 1786337134888098 us; error 30 us; skew 500 ppm
Aug 10 04:45:34 dist-test-slave-bqcl krb5kdc[28368](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786337134, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.167.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 04:45:34.890374 28390 init.cc:377] Logged in from keytab as kudu/127.22.167.126@KRBTEST.COM (short username kudu)
I20260810 04:45:34.890799 28390 webserver.cc:460] Webserver started at http://127.22.167.126:40659/ using document root <none> and password file <none>
I20260810 04:45:34.891004 28390 fs_manager.cc:359] Metadata directory not provided
I20260810 04:45:34.891052 28390 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 04:45:34.891176 28390 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 04:45:34.892118 28390 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786336863388938-23197-0/minicluster-data/master-0/data/instance:
uuid: "cee32472824149b98e955f270a54e790"
format_stamp: "Formatted at 2026-08-10 04:45:34 on dist-test-slave-bqcl"
server_key: "3096052a83e1d06dd25151ccac87f243"
server_key_iv: "b0817c8458742ad249cd958739c71133"
server_key_version: "encryptionkey@0"
I20260810 04:45:34.892431 28390 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786336863388938-23197-0/minicluster-data/master-0/wal/instance:
uuid: "cee32472824149b98e955f270a54e790"
format_stamp: "Formatted at 2026-08-10 04:45:34 on dist-test-slave-bqcl"
server_key: "3096052a83e1d06dd25151ccac87f243"
server_key_iv: "b0817c8458742ad249cd958739c71133"
server_key_version: "encryptionkey@0"
I20260810 04:45:34.893591 28390 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260810 04:45:34.894227 28413 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 04:45:34.894377 28390 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260810 04:45:34.894454 28390 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786336863388938-23197-0/minicluster-data/master-0/data,/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786336863388938-23197-0/minicluster-data/master-0/wal
uuid: "cee32472824149b98e955f270a54e790"
format_stamp: "Formatted at 2026-08-10 04:45:34 on dist-test-slave-bqcl"
server_key: "3096052a83e1d06dd25151ccac87f243"
server_key_iv: "b0817c8458742ad249cd958739c71133"
server_key_version: "encryptionkey@0"
I20260810 04:45:34.894505 28390 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786336863388938-23197-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786336863388938-23197-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786336863388938-23197-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 04:45:34.897529 28390 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 04:45:34.899154 28390 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 04:45:34.899256 28390 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 04:45:34.902977 28390 rpc_server.cc:307] RPC server started. Bound to: 127.22.167.126:41651
I20260810 04:45:34.903008 28504 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.167.126:41651 every 8 connection(s)
I20260810 04:45:34.903285 28390 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786336863388938-23197-0/minicluster-data/master-0/data/info.pb
I20260810 04:45:34.904198 28505 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 04:45:34.904405 23197 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskemv_AM/build/release/bin/kudu as pid 28390
I20260810 04:45:34.904516 23197 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786336863388938-23197-0/minicluster-data/master-0/wal/instance
I20260810 04:45:34.904701 23197 external_mini_cluster.cc:1428] Setting key 1abc2f00a9cbfa47f87b7be686add869
I20260810 04:45:34.906847 28509 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:45:34.905161 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44611 (local address 127.22.167.126:41651)
0810 04:45:34.905313 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0810 04:45:34.905320 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 04:45:34.905345 (+ 25us) server_negotiation.cc:407] Connection header received
0810 04:45:34.905735 (+ 390us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:45:34.905740 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:45:34.905765 (+ 25us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:45:34.905875 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0810 04:45:34.906465 (+ 590us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 04:45:34.906468 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 04:45:34.906469 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 04:45:34.906472 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0810 04:45:34.906511 (+ 39us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 04:45:34.906521 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 04:45:34.906576 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0810 04:45:34.906653 (+ 77us) server_negotiation.cc:299] Negotiation successful
0810 04:45:34.906682 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":105,"thread_start_us":70,"threads_started":1}
I20260810 04:45:34.906934 28505 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cee32472824149b98e955f270a54e790: Bootstrap starting.
I20260810 04:45:34.907577 28505 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P cee32472824149b98e955f270a54e790: Neither blocks nor log segments found. Creating new log.
I20260810 04:45:34.907707 23197 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 04:45:34.907871 28505 log.cc:824] T 00000000000000000000000000000000 P cee32472824149b98e955f270a54e790: Log is configured to *not* fsync() on all Append() calls
I20260810 04:45:34.908483 28505 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cee32472824149b98e955f270a54e790: No bootstrap required, opened a new log
I20260810 04:45:34.909900 28509 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:45:34.909020 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42753 (local address 127.22.167.126:41651)
0810 04:45:34.909056 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0810 04:45:34.909058 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 04:45:34.909141 (+ 83us) server_negotiation.cc:407] Connection header received
0810 04:45:34.909195 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:45:34.909196 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:45:34.909200 (+ 4us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:45:34.909244 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0810 04:45:34.909727 (+ 483us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 04:45:34.909728 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 04:45:34.909728 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 04:45:34.909732 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0810 04:45:34.909744 (+ 12us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 04:45:34.909748 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 04:45:34.909792 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0810 04:45:34.909832 (+ 40us) server_negotiation.cc:299] Negotiation successful
0810 04:45:34.909843 (+ 11us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260810 04:45:34.910543 28505 raft_consensus.cc:348] T 00000000000000000000000000000000 P cee32472824149b98e955f270a54e790 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cee32472824149b98e955f270a54e790" member_type: VOTER last_known_addr { host: "127.22.167.126" port: 41651 } }
I20260810 04:45:34.910671 28505 raft_consensus.cc:374] T 00000000000000000000000000000000 P cee32472824149b98e955f270a54e790 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 04:45:34.910692 28505 raft_consensus.cc:729] T 00000000000000000000000000000000 P cee32472824149b98e955f270a54e790 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cee32472824149b98e955f270a54e790, State: Initialized, Role: FOLLOWER
I20260810 04:45:34.910781 28505 consensus_queue.cc:260] T 00000000000000000000000000000000 P cee32472824149b98e955f270a54e790 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cee32472824149b98e955f270a54e790" member_type: VOTER last_known_addr { host: "127.22.167.126" port: 41651 } }
I20260810 04:45:34.910862 28505 raft_consensus.cc:388] T 00000000000000000000000000000000 P cee32472824149b98e955f270a54e790 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 04:45:34.910889 28505 raft_consensus.cc:482] T 00000000000000000000000000000000 P cee32472824149b98e955f270a54e790 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 04:45:34.910919 28505 raft_consensus.cc:3037] T 00000000000000000000000000000000 P cee32472824149b98e955f270a54e790 [term 0 FOLLOWER]: Advancing to term 1
I20260810 04:45:34.911371 28505 raft_consensus.cc:504] T 00000000000000000000000000000000 P cee32472824149b98e955f270a54e790 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cee32472824149b98e955f270a54e790" member_type: VOTER last_known_addr { host: "127.22.167.126" port: 41651 } }
I20260810 04:45:34.911458 28505 leader_election.cc:302] T 00000000000000000000000000000000 P cee32472824149b98e955f270a54e790 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: cee32472824149b98e955f270a54e790; no voters:
I20260810 04:45:34.911576 28505 leader_election.cc:288] T 00000000000000000000000000000000 P cee32472824149b98e955f270a54e790 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 04:45:34.911620 28523 raft_consensus.cc:2781] T 00000000000000000000000000000000 P cee32472824149b98e955f270a54e790 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 04:45:34.911775 28505 sys_catalog.cc:564] T 00000000000000000000000000000000 P cee32472824149b98e955f270a54e790 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 04:45:34.911801 28523 raft_consensus.cc:686] T 00000000000000000000000000000000 P cee32472824149b98e955f270a54e790 [term 1 LEADER]: Becoming Leader. State: Replica: cee32472824149b98e955f270a54e790, State: Running, Role: LEADER
I20260810 04:45:34.911881 28523 consensus_queue.cc:237] T 00000000000000000000000000000000 P cee32472824149b98e955f270a54e790 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cee32472824149b98e955f270a54e790" member_type: VOTER last_known_addr { host: "127.22.167.126" port: 41651 } }
I20260810 04:45:34.912218 28530 sys_catalog.cc:455] T 00000000000000000000000000000000 P cee32472824149b98e955f270a54e790 [sys.catalog]: SysCatalogTable state changed. Reason: New leader cee32472824149b98e955f270a54e790. Latest consensus state: current_term: 1 leader_uuid: "cee32472824149b98e955f270a54e790" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cee32472824149b98e955f270a54e790" member_type: VOTER last_known_addr { host: "127.22.167.126" port: 41651 } } }
I20260810 04:45:34.912320 28530 sys_catalog.cc:458] T 00000000000000000000000000000000 P cee32472824149b98e955f270a54e790 [sys.catalog]: This master's current role is: LEADER
I20260810 04:45:34.912215 28527 sys_catalog.cc:455] T 00000000000000000000000000000000 P cee32472824149b98e955f270a54e790 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "cee32472824149b98e955f270a54e790" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cee32472824149b98e955f270a54e790" member_type: VOTER last_known_addr { host: "127.22.167.126" port: 41651 } } }
I20260810 04:45:34.912370 28527 sys_catalog.cc:458] T 00000000000000000000000000000000 P cee32472824149b98e955f270a54e790 [sys.catalog]: This master's current role is: LEADER
I20260810 04:45:34.912824 28542 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 04:45:34.913257 28542 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 04:45:34.914757 28542 catalog_manager.cc:1347] Generated new cluster ID: aebf1f2f3c064c8a8d5ec0d377b2d70f
I20260810 04:45:34.914799 28542 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 04:45:34.931937 28542 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 04:45:34.932088 28542 catalog_manager.cc:1504] Loading token signing keys...
I20260810 04:45:34.946630 28542 catalog_manager.cc:5868] T 00000000000000000000000000000000 P cee32472824149b98e955f270a54e790: Generated new TSK 0
I20260810 04:45:34.946800 28542 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 04:45:34.957156 23197 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskemv_AM/build/release/bin/kudu with pid 28390
2026-08-10T04:45:34Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/17 (1425 ms)
[----------] 18 tests from AuthTokenIssuingTest (25791 ms total)
[----------] 2 tests from ConnectToFollowerMasterTest
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0
Loading random data
Initializing database '/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786336863388938-23197-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 04:45:35 dist-test-slave-bqcl krb5kdc[28563](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 04:45:35 dist-test-slave-bqcl krb5kdc[28563](info): set up 2 sockets
Aug 10 04:45:35 dist-test-slave-bqcl krb5kdc[28563](info): commencing operation
krb5kdc: starting...
W20260810 04:45:36.132771 23197 mini_kdc.cc:121] Time spent starting KDC: real 1.147s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786336863388938-23197-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786336863388938-23197-0/krb5kdc/test-user.keytab.
Aug 10 04:45:36 dist-test-slave-bqcl krb5kdc[28563](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786337136, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T04:45:36Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T04:45:36Z Disabled control of system clock
WARNING: no policy specified for kudu/127.22.167.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.167.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.167.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786336863388938-23197-0/krb5kdc/kudu_127.22.167.126.keytab.
Entry for principal kudu/127.22.167.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786336863388938-23197-0/krb5kdc/kudu_127.22.167.126.keytab.
I20260810 04:45:36.246668 23197 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskemv_AM/build/release/bin/kudu
/tmp/dist-test-taskemv_AM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786336863388938-23197-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786336863388938-23197-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786336863388938-23197-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786336863388938-23197-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.167.126:34487
--webserver_interface=127.22.167.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786336863388938-23197-0/krb5kdc/kudu_127.22.167.126.keytab
--principal=kudu/127.22.167.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.167.84:45549
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.167.126:34487,127.22.167.125:35119,127.22.167.124:33959
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786336863388938-23197-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786336863388938-23197-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786336863388938-23197-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 04:45:36.325075 28590 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260810 04:45:36.325285 28590 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 04:45:36.325311 28590 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 04:45:36.325335 28590 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 04:45:36.326455 28590 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 04:45:36.326493 28590 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 04:45:36.326511 28590 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 04:45:36.326527 28590 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 04:45:36.326541 28590 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 04:45:36.327948 28590 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.167.84:45549
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786336863388938-23197-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786336863388938-23197-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.22.167.126:34487,127.22.167.125:35119,127.22.167.124:33959
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786336863388938-23197-0/krb5kdc/kudu_127.22.167.126.keytab
--principal=kudu/127.22.167.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.167.126:34487
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786336863388938-23197-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.167.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28590
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786336863388938-23197-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 03:54:01 UTC on ccycloud.quasar-icijmt.root.comops.site
build id 4577009
I20260810 04:45:36.328178 28590 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 04:45:36.328368 28590 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 04:45:36.332810 28600 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 04:45:36.332796 28599 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 04:45:36.332824 28603 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 04:45:36.333070 28590 server_base.cc:1034] running on GCE node
I20260810 04:45:36.333307 28590 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 04:45:36.333576 28590 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 04:45:36.334720 28590 hybrid_clock.cc:648] HybridClock initialized: now 1786337136334707 us; error 32 us; skew 500 ppm
Aug 10 04:45:36 dist-test-slave-bqcl krb5kdc[28563](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786337136, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.167.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 04:45:36.337077 28590 init.cc:377] Logged in from keytab as kudu/127.22.167.126@KRBTEST.COM (short username kudu)
I20260810 04:45:36.337473 28590 webserver.cc:460] Webserver started at http://127.22.167.126:45865/ using document root <none> and password file <none>
I20260810 04:45:36.337670 28590 fs_manager.cc:359] Metadata directory not provided
I20260810 04:45:36.337713 28590 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 04:45:36.337832 28590 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 04:45:36.338852 28590 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786336863388938-23197-0/minicluster-data/master-0/data/instance:
uuid: "619bf37a41d74e0b959137961b2cf7cc"
format_stamp: "Formatted at 2026-08-10 04:45:36 on dist-test-slave-bqcl"
server_key: "3d36cad98a49d77575dfaddab3bba90a"
server_key_iv: "050d52345d984ba88dd408d861b0559a"
server_key_version: "encryptionkey@0"
I20260810 04:45:36.339160 28590 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786336863388938-23197-0/minicluster-data/master-0/wal/instance:
uuid: "619bf37a41d74e0b959137961b2cf7cc"
format_stamp: "Formatted at 2026-08-10 04:45:36 on dist-test-slave-bqcl"
server_key: "3d36cad98a49d77575dfaddab3bba90a"
server_key_iv: "050d52345d984ba88dd408d861b0559a"
server_key_version: "encryptionkey@0"
I20260810 04:45:36.340323 28590 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260810 04:45:36.341018 28614 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 04:45:36.341171 28590 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260810 04:45:36.341251 28590 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786336863388938-23197-0/minicluster-data/master-0/data,/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786336863388938-23197-0/minicluster-data/master-0/wal
uuid: "619bf37a41d74e0b959137961b2cf7cc"
format_stamp: "Formatted at 2026-08-10 04:45:36 on dist-test-slave-bqcl"
server_key: "3d36cad98a49d77575dfaddab3bba90a"
server_key_iv: "050d52345d984ba88dd408d861b0559a"
server_key_version: "encryptionkey@0"
I20260810 04:45:36.341315 28590 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786336863388938-23197-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786336863388938-23197-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786336863388938-23197-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 04:45:36.353137 28590 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 04:45:36.354763 28590 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 04:45:36.354880 28590 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 04:45:36.358666 28590 rpc_server.cc:307] RPC server started. Bound to: 127.22.167.126:34487
I20260810 04:45:36.358704 28708 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.167.126:34487 every 8 connection(s)
I20260810 04:45:36.358994 28590 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786336863388938-23197-0/minicluster-data/master-0/data/info.pb
I20260810 04:45:36.359798 28710 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 04:45:36.360844 23197 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskemv_AM/build/release/bin/kudu as pid 28590
I20260810 04:45:36.360936 23197 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786336863388938-23197-0/minicluster-data/master-0/wal/instance
I20260810 04:45:36.361122 23197 external_mini_cluster.cc:1428] Setting key 171ce0f3a063fd5f5ff587f099918320
I20260810 04:45:36.361373 28710 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.22.167.126" port: 34487 } has no permanent_uuid. Determining permanent_uuid...
Aug 10 04:45:36 dist-test-slave-bqcl krb5kdc[28563](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786337136, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.167.126@KRBTEST.COM for kudu/127.22.167.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.22.167.125@KRBTEST.COM; defaulting to no policy
I20260810 04:45:36.370550 28715 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:45:36.362060 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60706 (local address 127.22.167.126:34487)
0810 04:45:36.362241 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0810 04:45:36.362246 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 04:45:36.362337 (+ 91us) server_negotiation.cc:407] Connection header received
0810 04:45:36.362936 (+ 599us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:45:36.362939 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:45:36.362960 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:45:36.363046 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0810 04:45:36.363566 (+ 520us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:45:36.364360 (+ 794us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:45:36.365226 (+ 866us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:45:36.365553 (+ 327us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:45:36.365655 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:45:36.367660 (+ 2005us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 04:45:36.367667 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 04:45:36.367668 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 04:45:36.367675 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 04:45:36.369182 (+ 1507us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 04:45:36.369542 (+ 360us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 04:45:36.369545 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 04:45:36.369545 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 04:45:36.369628 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 04:45:36.369835 (+ 207us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 04:45:36.369836 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 04:45:36.369836 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 04:45:36.370013 (+ 177us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 04:45:36.370076 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0810 04:45:36.370266 (+ 190us) server_negotiation.cc:299] Negotiation successful
0810 04:45:36.370358 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":157,"thread_start_us":105,"threads_started":1}
I20260810 04:45:36.370560 28712 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:45:36.362008 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.126:34487 (local address 127.0.0.1:60706)
0810 04:45:36.362192 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0810 04:45:36.362235 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0810 04:45:36.362779 (+ 544us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 04:45:36.363056 (+ 277us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 04:45:36.363059 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 04:45:36.363314 (+ 255us) client_negotiation.cc:189] Negotiated authn=SASL
0810 04:45:36.363459 (+ 145us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:45:36.363463 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:45:36.364468 (+ 1005us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:45:36.364469 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:45:36.365137 (+ 668us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:45:36.365139 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:45:36.365634 (+ 495us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:45:36.365635 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:45:36.365704 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:45:36.366216 (+ 512us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 04:45:36.366224 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0810 04:45:36.367576 (+ 1352us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 04:45:36.369264 (+ 1688us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 04:45:36.369271 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 04:45:36.369273 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 04:45:36.369465 (+ 192us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 04:45:36.369693 (+ 228us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 04:45:36.369694 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 04:45:36.369694 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 04:45:36.369781 (+ 87us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 04:45:36.370063 (+ 282us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 04:45:36.370066 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 04:45:36.370118 (+ 52us) client_negotiation.cc:769] Sending connection context
0810 04:45:36.370234 (+ 116us) client_negotiation.cc:240] Negotiation successful
0810 04:45:36.370429 (+ 195us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":141,"mutex_wait_us":75,"thread_start_us":79,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.167.125@KRBTEST.COM" created.
I20260810 04:45:36.371389 28710 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.22.167.125" port: 35119 } has no permanent_uuid. Determining permanent_uuid...
I20260810 04:45:36.371701 28712 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:45:36.371600 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.125:35119 (local address 127.0.0.1:40990)
0810 04:45:36.371641 (+ 41us) negotiation.cc:107] Waiting for socket to connect
0810 04:45:36.371683 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.167.125:35119: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
W20260810 04:45:36.371829 28616 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.22.167.125:35119: connect: Connection refused (error 111)
I20260810 04:45:36.372205 28712 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:45:36.372141 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.125:35119 (local address 127.0.0.1:40998)
0810 04:45:36.372166 (+ 25us) negotiation.cc:107] Waiting for socket to connect
0810 04:45:36.372188 (+ 22us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.167.125:35119: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":14}
W20260810 04:45:36.372295 28710 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.22.167.125:35119: Network error: Client connection negotiation failed: client connection to 127.22.167.125:35119: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.167.125 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786336863388938-23197-0/krb5kdc/kudu_127.22.167.125.keytab.
Entry for principal kudu/127.22.167.125 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786336863388938-23197-0/krb5kdc/kudu_127.22.167.125.keytab.
I20260810 04:45:36.384308 23197 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskemv_AM/build/release/bin/kudu
/tmp/dist-test-taskemv_AM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786336863388938-23197-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786336863388938-23197-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786336863388938-23197-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786336863388938-23197-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.167.125:35119
--webserver_interface=127.22.167.125
--webserver_port=0
--keytab_file=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786336863388938-23197-0/krb5kdc/kudu_127.22.167.125.keytab
--principal=kudu/127.22.167.125
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.167.84:45549
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.167.126:34487,127.22.167.125:35119,127.22.167.124:33959
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786336863388938-23197-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786336863388938-23197-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786336863388938-23197-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 04:45:36.421523 28710 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.22.167.125" port: 35119 } attempt: 1
I20260810 04:45:36.421958 28712 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:45:36.421845 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.125:35119 (local address 127.0.0.1:41012)
0810 04:45:36.421899 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0810 04:45:36.421929 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.167.125:35119: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
I20260810 04:45:36.422299 28712 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:45:36.422215 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.125:35119 (local address 127.0.0.1:41028)
0810 04:45:36.422256 (+ 41us) negotiation.cc:107] Waiting for socket to connect
0810 04:45:36.422279 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.167.125:35119: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":25}
W20260810 04:45:36.422399 28710 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.22.167.125:35119: Network error: Client connection negotiation failed: client connection to 127.22.167.125:35119: connect: Connection refused (error 111)
W20260810 04:45:36.463549 28719 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260810 04:45:36.463762 28719 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 04:45:36.463788 28719 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 04:45:36.463814 28719 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 04:45:36.464927 28719 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 04:45:36.464967 28719 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 04:45:36.464986 28719 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 04:45:36.465001 28719 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 04:45:36.465019 28719 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 04:45:36.466661 28719 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.167.84:45549
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786336863388938-23197-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786336863388938-23197-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.22.167.126:34487,127.22.167.125:35119,127.22.167.124:33959
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786336863388938-23197-0/krb5kdc/kudu_127.22.167.125.keytab
--principal=kudu/127.22.167.125
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.167.125:35119
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786336863388938-23197-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.22.167.125
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28719
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786336863388938-23197-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 03:54:01 UTC on ccycloud.quasar-icijmt.root.comops.site
build id 4577009
I20260810 04:45:36.466903 28719 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 04:45:36.467123 28719 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 04:45:36.471949 28729 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 04:45:36.471952 28730 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 04:45:36.472105 28719 server_base.cc:1034] running on GCE node
W20260810 04:45:36.471946 28737 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 04:45:36.472432 28719 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 04:45:36.472700 28719 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 04:45:36.473840 28719 hybrid_clock.cc:648] HybridClock initialized: now 1786337136473830 us; error 31 us; skew 500 ppm
Aug 10 04:45:36 dist-test-slave-bqcl krb5kdc[28563](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786337136, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.167.125@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 04:45:36.476136 28719 init.cc:377] Logged in from keytab as kudu/127.22.167.125@KRBTEST.COM (short username kudu)
I20260810 04:45:36.476534 28719 webserver.cc:460] Webserver started at http://127.22.167.125:46143/ using document root <none> and password file <none>
I20260810 04:45:36.476744 28719 fs_manager.cc:359] Metadata directory not provided
I20260810 04:45:36.476791 28719 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 04:45:36.476917 28719 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 04:45:36.477896 28719 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786336863388938-23197-0/minicluster-data/master-1/data/instance:
uuid: "314e7978cdec43ee973630752d9a16fa"
format_stamp: "Formatted at 2026-08-10 04:45:36 on dist-test-slave-bqcl"
server_key: "7b56f6eedaa368345782fc394f1747d7"
server_key_iv: "9b4fc38de277d56538d786a30fb92e28"
server_key_version: "encryptionkey@0"
I20260810 04:45:36.478269 28719 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786336863388938-23197-0/minicluster-data/master-1/wal/instance:
uuid: "314e7978cdec43ee973630752d9a16fa"
format_stamp: "Formatted at 2026-08-10 04:45:36 on dist-test-slave-bqcl"
server_key: "7b56f6eedaa368345782fc394f1747d7"
server_key_iv: "9b4fc38de277d56538d786a30fb92e28"
server_key_version: "encryptionkey@0"
I20260810 04:45:36.479453 28719 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260810 04:45:36.480119 28746 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 04:45:36.480264 28719 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260810 04:45:36.480345 28719 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786336863388938-23197-0/minicluster-data/master-1/data,/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786336863388938-23197-0/minicluster-data/master-1/wal
uuid: "314e7978cdec43ee973630752d9a16fa"
format_stamp: "Formatted at 2026-08-10 04:45:36 on dist-test-slave-bqcl"
server_key: "7b56f6eedaa368345782fc394f1747d7"
server_key_iv: "9b4fc38de277d56538d786a30fb92e28"
server_key_version: "encryptionkey@0"
I20260810 04:45:36.480409 28719 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786336863388938-23197-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786336863388938-23197-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786336863388938-23197-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 04:45:36.490556 28710 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.22.167.125" port: 35119 } attempt: 2
I20260810 04:45:36.490970 28712 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:45:36.490831 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.125:35119 (local address 127.0.0.1:41038)
0810 04:45:36.490893 (+ 62us) negotiation.cc:107] Waiting for socket to connect
0810 04:45:36.490933 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.167.125:35119: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
I20260810 04:45:36.491317 28712 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:45:36.491241 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.125:35119 (local address 127.0.0.1:41046)
0810 04:45:36.491277 (+ 36us) negotiation.cc:107] Waiting for socket to connect
0810 04:45:36.491299 (+ 22us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.167.125:35119: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":25}
W20260810 04:45:36.491402 28710 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.22.167.125:35119: Network error: Client connection negotiation failed: client connection to 127.22.167.125:35119: connect: Connection refused (error 111)
I20260810 04:45:36.503142 28719 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 04:45:36.504849 28719 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 04:45:36.504963 28719 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 04:45:36.508809 28719 rpc_server.cc:307] RPC server started. Bound to: 127.22.167.125:35119
I20260810 04:45:36.508855 28842 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.167.125:35119 every 8 connection(s)
I20260810 04:45:36.509155 28719 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786336863388938-23197-0/minicluster-data/master-1/data/info.pb
I20260810 04:45:36.509968 28844 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 04:45:36.511539 28844 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.22.167.126" port: 34487 } has no permanent_uuid. Determining permanent_uuid...
Aug 10 04:45:36 dist-test-slave-bqcl krb5kdc[28563](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786337136, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.167.125@KRBTEST.COM for kudu/127.22.167.126@KRBTEST.COM
I20260810 04:45:36.519060 23197 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskemv_AM/build/release/bin/kudu as pid 28719
I20260810 04:45:36.519173 23197 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786336863388938-23197-0/minicluster-data/master-1/wal/instance
I20260810 04:45:36.519380 23197 external_mini_cluster.cc:1428] Setting key 517cdcc4f089421e7da8d613653d6dfd
I20260810 04:45:36.519516 28715 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:45:36.512174 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60710 (local address 127.22.167.126:34487)
0810 04:45:36.512225 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0810 04:45:36.512227 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 04:45:36.512423 (+ 196us) server_negotiation.cc:407] Connection header received
0810 04:45:36.512905 (+ 482us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:45:36.512906 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:45:36.512920 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:45:36.512970 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0810 04:45:36.513531 (+ 561us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:45:36.514004 (+ 473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:45:36.514965 (+ 961us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:45:36.515170 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:45:36.515225 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:45:36.517132 (+ 1907us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 04:45:36.517133 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 04:45:36.517133 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 04:45:36.517148 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0810 04:45:36.518278 (+ 1130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 04:45:36.518615 (+ 337us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 04:45:36.518616 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 04:45:36.518617 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 04:45:36.518673 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 04:45:36.518890 (+ 217us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 04:45:36.518891 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 04:45:36.518892 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 04:45:36.519002 (+ 110us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 04:45:36.519057 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0810 04:45:36.519343 (+ 286us) server_negotiation.cc:299] Negotiation successful
0810 04:45:36.519430 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260810 04:45:36.519666 28849 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:45:36.512148 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.126:34487 (local address 127.0.0.1:60710)
0810 04:45:36.512309 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0810 04:45:36.512343 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0810 04:45:36.512808 (+ 465us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 04:45:36.513035 (+ 227us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 04:45:36.513039 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 04:45:36.513260 (+ 221us) client_negotiation.cc:189] Negotiated authn=SASL
0810 04:45:36.513439 (+ 179us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:45:36.513443 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:45:36.514082 (+ 639us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:45:36.514083 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:45:36.514878 (+ 795us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:45:36.514880 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:45:36.515239 (+ 359us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:45:36.515240 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:45:36.515357 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:45:36.515751 (+ 394us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 04:45:36.515759 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0810 04:45:36.517044 (+ 1285us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 04:45:36.518341 (+ 1297us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 04:45:36.518348 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 04:45:36.518350 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 04:45:36.518552 (+ 202us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 04:45:36.518727 (+ 175us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 04:45:36.518727 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 04:45:36.518728 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 04:45:36.518835 (+ 107us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 04:45:36.519056 (+ 221us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 04:45:36.519063 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 04:45:36.519207 (+ 144us) client_negotiation.cc:769] Sending connection context
0810 04:45:36.519307 (+ 100us) client_negotiation.cc:240] Negotiation successful
0810 04:45:36.519449 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":84,"threads_started":1}
I20260810 04:45:36.520308 28844 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.22.167.125" port: 35119 } has no permanent_uuid. Determining permanent_uuid...
Aug 10 04:45:36 dist-test-slave-bqcl krb5kdc[28563](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786337136, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.167.125@KRBTEST.COM for kudu/127.22.167.125@KRBTEST.COM
I20260810 04:45:36.528004 28849 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:45:36.520587 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.125:35119 (local address 127.0.0.1:41060)
0810 04:45:36.520644 (+ 57us) negotiation.cc:107] Waiting for socket to connect
0810 04:45:36.520653 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0810 04:45:36.520690 (+ 37us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 04:45:36.520912 (+ 222us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 04:45:36.520913 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 04:45:36.521060 (+ 147us) client_negotiation.cc:189] Negotiated authn=SASL
0810 04:45:36.521128 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:45:36.521130 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:45:36.522276 (+ 1146us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:45:36.522277 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:45:36.522792 (+ 515us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:45:36.522795 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:45:36.523370 (+ 575us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:45:36.523371 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:45:36.523432 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:45:36.523900 (+ 468us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 04:45:36.523904 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0810 04:45:36.525200 (+ 1296us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 04:45:36.527037 (+ 1837us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 04:45:36.527037 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 04:45:36.527038 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 04:45:36.527204 (+ 166us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 04:45:36.527414 (+ 210us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 04:45:36.527415 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 04:45:36.527415 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 04:45:36.527470 (+ 55us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 04:45:36.527766 (+ 296us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 04:45:36.527767 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 04:45:36.527802 (+ 35us) client_negotiation.cc:769] Sending connection context
0810 04:45:36.527856 (+ 54us) client_negotiation.cc:240] Negotiation successful
0810 04:45:36.527933 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":43}
I20260810 04:45:36.528132 28852 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:45:36.520637 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41060 (local address 127.22.167.125:35119)
0810 04:45:36.520780 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0810 04:45:36.520784 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 04:45:36.520802 (+ 18us) server_negotiation.cc:407] Connection header received
0810 04:45:36.520823 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:45:36.520824 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:45:36.520834 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:45:36.520892 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0810 04:45:36.521213 (+ 321us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:45:36.522201 (+ 988us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:45:36.522875 (+ 674us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:45:36.523286 (+ 411us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:45:36.523355 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:45:36.525286 (+ 1931us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 04:45:36.525296 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 04:45:36.525296 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 04:45:36.525303 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 04:45:36.526941 (+ 1638us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 04:45:36.527278 (+ 337us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 04:45:36.527281 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 04:45:36.527281 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 04:45:36.527344 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 04:45:36.527528 (+ 184us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 04:45:36.527529 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 04:45:36.527530 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 04:45:36.527702 (+ 172us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 04:45:36.527792 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0810 04:45:36.527896 (+ 104us) server_negotiation.cc:299] Negotiation successful
0810 04:45:36.527995 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":22,"server-negotiator.queue_time_us":116,"thread_start_us":81,"threads_started":1}
WARNING: no policy specified for kudu/127.22.167.124@KRBTEST.COM; defaulting to no policy
I20260810 04:45:36.528650 28844 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.22.167.124" port: 33959 } has no permanent_uuid. Determining permanent_uuid...
I20260810 04:45:36.528946 28849 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:45:36.528843 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.124:33959 (local address 127.0.0.1:57610)
0810 04:45:36.528881 (+ 38us) negotiation.cc:107] Waiting for socket to connect
0810 04:45:36.528926 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.167.124:33959: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":23}
W20260810 04:45:36.529076 28753 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.22.167.124:33959: connect: Connection refused (error 111)
I20260810 04:45:36.529470 28849 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:45:36.529377 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.124:33959 (local address 127.0.0.1:57622)
0810 04:45:36.529415 (+ 38us) negotiation.cc:107] Waiting for socket to connect
0810 04:45:36.529446 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.167.124:33959: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":24}
W20260810 04:45:36.529575 28844 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.22.167.124:33959: Network error: Client connection negotiation failed: client connection to 127.22.167.124:33959: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.167.124@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.167.124 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786336863388938-23197-0/krb5kdc/kudu_127.22.167.124.keytab.
Entry for principal kudu/127.22.167.124 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786336863388938-23197-0/krb5kdc/kudu_127.22.167.124.keytab.
I20260810 04:45:36.544420 23197 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskemv_AM/build/release/bin/kudu
/tmp/dist-test-taskemv_AM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786336863388938-23197-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786336863388938-23197-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786336863388938-23197-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786336863388938-23197-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.167.124:33959
--webserver_interface=127.22.167.124
--webserver_port=0
--keytab_file=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786336863388938-23197-0/krb5kdc/kudu_127.22.167.124.keytab
--principal=kudu/127.22.167.124
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.167.84:45549
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.167.126:34487,127.22.167.125:35119,127.22.167.124:33959
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786336863388938-23197-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786336863388938-23197-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786336863388938-23197-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 04:45:36.578794 28844 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.22.167.124" port: 33959 } attempt: 1
I20260810 04:45:36.579283 28849 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:45:36.579121 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.124:33959 (local address 127.0.0.1:57624)
0810 04:45:36.579184 (+ 63us) negotiation.cc:107] Waiting for socket to connect
0810 04:45:36.579239 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.167.124:33959: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
I20260810 04:45:36.579665 28849 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:45:36.579565 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.124:33959 (local address 127.0.0.1:57632)
0810 04:45:36.579605 (+ 40us) negotiation.cc:107] Waiting for socket to connect
0810 04:45:36.579639 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.167.124:33959: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":25}
W20260810 04:45:36.579767 28844 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.22.167.124:33959: Network error: Client connection negotiation failed: client connection to 127.22.167.124:33959: connect: Connection refused (error 111)
I20260810 04:45:36.582546 28710 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.22.167.125" port: 35119 } attempt: 3
Aug 10 04:45:36 dist-test-slave-bqcl krb5kdc[28563](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786337136, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.167.126@KRBTEST.COM for kudu/127.22.167.125@KRBTEST.COM
I20260810 04:45:36.589035 28712 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:45:36.582826 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.125:35119 (local address 127.0.0.1:41070)
0810 04:45:36.582866 (+ 40us) negotiation.cc:107] Waiting for socket to connect
0810 04:45:36.582877 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0810 04:45:36.582926 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 04:45:36.583045 (+ 119us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 04:45:36.583046 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 04:45:36.583214 (+ 168us) client_negotiation.cc:189] Negotiated authn=SASL
0810 04:45:36.583280 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:45:36.583283 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:45:36.583958 (+ 675us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:45:36.583959 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:45:36.584488 (+ 529us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:45:36.584490 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:45:36.584862 (+ 372us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:45:36.584863 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:45:36.584932 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:45:36.585393 (+ 461us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 04:45:36.585398 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0810 04:45:36.586571 (+ 1173us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 04:45:36.588213 (+ 1642us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 04:45:36.588214 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 04:45:36.588214 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 04:45:36.588374 (+ 160us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 04:45:36.588561 (+ 187us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 04:45:36.588562 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 04:45:36.588562 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 04:45:36.588606 (+ 44us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 04:45:36.588822 (+ 216us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 04:45:36.588822 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 04:45:36.588858 (+ 36us) client_negotiation.cc:769] Sending connection context
0810 04:45:36.588901 (+ 43us) client_negotiation.cc:240] Negotiation successful
0810 04:45:36.588972 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":18}
I20260810 04:45:36.589102 28852 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:45:36.582858 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41070 (local address 127.22.167.125:35119)
0810 04:45:36.582899 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0810 04:45:36.582901 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 04:45:36.582912 (+ 11us) server_negotiation.cc:407] Connection header received
0810 04:45:36.582979 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:45:36.582980 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:45:36.582989 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:45:36.583034 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0810 04:45:36.583353 (+ 319us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:45:36.583879 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:45:36.584563 (+ 684us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:45:36.584795 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:45:36.584852 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:45:36.586652 (+ 1800us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 04:45:36.586653 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 04:45:36.586654 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 04:45:36.586661 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 04:45:36.588137 (+ 1476us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 04:45:36.588437 (+ 300us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 04:45:36.588438 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 04:45:36.588438 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 04:45:36.588502 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 04:45:36.588642 (+ 140us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 04:45:36.588642 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 04:45:36.588643 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 04:45:36.588773 (+ 130us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 04:45:36.588836 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0810 04:45:36.588927 (+ 91us) server_negotiation.cc:299] Negotiation successful
0810 04:45:36.589016 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260810 04:45:36.589540 28710 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.22.167.124" port: 33959 } has no permanent_uuid. Determining permanent_uuid...
I20260810 04:45:36.589846 28712 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:45:36.589757 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.124:33959 (local address 127.0.0.1:57638)
0810 04:45:36.589799 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0810 04:45:36.589823 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.167.124:33959: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":25}
I20260810 04:45:36.590183 28712 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:45:36.590090 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.124:33959 (local address 127.0.0.1:57650)
0810 04:45:36.590141 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0810 04:45:36.590164 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.167.124:33959: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":22}
W20260810 04:45:36.590258 28710 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.22.167.124:33959: Network error: Client connection negotiation failed: client connection to 127.22.167.124:33959: connect: Connection refused (error 111)
I20260810 04:45:36.621394 28710 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.22.167.124" port: 33959 } attempt: 1
I20260810 04:45:36.621870 28712 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:45:36.621732 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.124:33959 (local address 127.0.0.1:57654)
0810 04:45:36.621800 (+ 68us) negotiation.cc:107] Waiting for socket to connect
0810 04:45:36.621842 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.167.124:33959: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
I20260810 04:45:36.622241 28712 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:45:36.622158 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.124:33959 (local address 127.0.0.1:57662)
0810 04:45:36.622198 (+ 40us) negotiation.cc:107] Waiting for socket to connect
0810 04:45:36.622221 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.167.124:33959: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":26}
W20260810 04:45:36.622341 28710 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.22.167.124:33959: Network error: Client connection negotiation failed: client connection to 127.22.167.124:33959: connect: Connection refused (error 111)
W20260810 04:45:36.626780 28859 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260810 04:45:36.626974 28859 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 04:45:36.626997 28859 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 04:45:36.627019 28859 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 04:45:36.628113 28859 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 04:45:36.628146 28859 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 04:45:36.628158 28859 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 04:45:36.628170 28859 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 04:45:36.628185 28859 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 04:45:36.629567 28859 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.167.84:45549
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786336863388938-23197-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786336863388938-23197-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.22.167.126:34487,127.22.167.125:35119,127.22.167.124:33959
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786336863388938-23197-0/krb5kdc/kudu_127.22.167.124.keytab
--principal=kudu/127.22.167.124
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.167.124:33959
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786336863388938-23197-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.22.167.124
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28859
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786336863388938-23197-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 03:54:01 UTC on ccycloud.quasar-icijmt.root.comops.site
build id 4577009
I20260810 04:45:36.629760 28859 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 04:45:36.629935 28859 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 04:45:36.634596 28870 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 04:45:36.634599 28871 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 04:45:36.634624 28874 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 04:45:36.634877 28859 server_base.cc:1034] running on GCE node
I20260810 04:45:36.635130 28859 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 04:45:36.635418 28859 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 04:45:36.636576 28859 hybrid_clock.cc:648] HybridClock initialized: now 1786337136636560 us; error 30 us; skew 500 ppm
Aug 10 04:45:36 dist-test-slave-bqcl krb5kdc[28563](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786337136, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.167.124@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 04:45:36.639082 28859 init.cc:377] Logged in from keytab as kudu/127.22.167.124@KRBTEST.COM (short username kudu)
I20260810 04:45:36.639488 28859 webserver.cc:460] Webserver started at http://127.22.167.124:37525/ using document root <none> and password file <none>
I20260810 04:45:36.639678 28859 fs_manager.cc:359] Metadata directory not provided
I20260810 04:45:36.639721 28859 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 04:45:36.639819 28859 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 04:45:36.640820 28859 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786336863388938-23197-0/minicluster-data/master-2/data/instance:
uuid: "b493af56a8424b4cb0595223fe59d998"
format_stamp: "Formatted at 2026-08-10 04:45:36 on dist-test-slave-bqcl"
server_key: "d42a60167738ba6a4c43c65741a9de02"
server_key_iv: "3b2270b51d1253a3c128761d443055e4"
server_key_version: "encryptionkey@0"
I20260810 04:45:36.641129 28859 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786336863388938-23197-0/minicluster-data/master-2/wal/instance:
uuid: "b493af56a8424b4cb0595223fe59d998"
format_stamp: "Formatted at 2026-08-10 04:45:36 on dist-test-slave-bqcl"
server_key: "d42a60167738ba6a4c43c65741a9de02"
server_key_iv: "3b2270b51d1253a3c128761d443055e4"
server_key_version: "encryptionkey@0"
I20260810 04:45:36.642434 28859 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260810 04:45:36.643128 28889 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 04:45:36.643302 28859 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260810 04:45:36.643384 28859 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786336863388938-23197-0/minicluster-data/master-2/data,/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786336863388938-23197-0/minicluster-data/master-2/wal
uuid: "b493af56a8424b4cb0595223fe59d998"
format_stamp: "Formatted at 2026-08-10 04:45:36 on dist-test-slave-bqcl"
server_key: "d42a60167738ba6a4c43c65741a9de02"
server_key_iv: "3b2270b51d1253a3c128761d443055e4"
server_key_version: "encryptionkey@0"
I20260810 04:45:36.643443 28859 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786336863388938-23197-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786336863388938-23197-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786336863388938-23197-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 04:45:36.647910 28844 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.22.167.124" port: 33959 } attempt: 2
I20260810 04:45:36.648267 28849 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:45:36.648140 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.124:33959 (local address 127.0.0.1:57674)
0810 04:45:36.648200 (+ 60us) negotiation.cc:107] Waiting for socket to connect
0810 04:45:36.648237 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.167.124:33959: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
I20260810 04:45:36.648581 28849 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:45:36.648477 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.124:33959 (local address 127.0.0.1:57686)
0810 04:45:36.648523 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0810 04:45:36.648555 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.167.124:33959: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
W20260810 04:45:36.648686 28844 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.22.167.124:33959: Network error: Client connection negotiation failed: client connection to 127.22.167.124:33959: connect: Connection refused (error 111)
I20260810 04:45:36.655597 28859 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 04:45:36.657191 28859 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 04:45:36.657294 28859 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 04:45:36.660979 28859 rpc_server.cc:307] RPC server started. Bound to: 127.22.167.124:33959
I20260810 04:45:36.661013 28977 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.167.124:33959 every 8 connection(s)
I20260810 04:45:36.661310 28859 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786336863388938-23197-0/minicluster-data/master-2/data/info.pb
I20260810 04:45:36.662108 28978 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 04:45:36.663676 28978 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.22.167.126" port: 34487 } has no permanent_uuid. Determining permanent_uuid...
Aug 10 04:45:36 dist-test-slave-bqcl krb5kdc[28563](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786337136, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.167.124@KRBTEST.COM for kudu/127.22.167.126@KRBTEST.COM
I20260810 04:45:36.668838 23197 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskemv_AM/build/release/bin/kudu as pid 28859
I20260810 04:45:36.668936 23197 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786336863388938-23197-0/minicluster-data/master-2/wal/instance
I20260810 04:45:36.669137 23197 external_mini_cluster.cc:1428] Setting key fe004a3c5d1290406669ec7d6b83f428
I20260810 04:45:36.671967 28715 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:45:36.664297 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60716 (local address 127.22.167.126:34487)
0810 04:45:36.664357 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0810 04:45:36.664359 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 04:45:36.664528 (+ 169us) server_negotiation.cc:407] Connection header received
0810 04:45:36.665007 (+ 479us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:45:36.665009 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:45:36.665021 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:45:36.665073 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0810 04:45:36.665668 (+ 595us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:45:36.666171 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:45:36.667112 (+ 941us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:45:36.667302 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:45:36.667359 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:45:36.669353 (+ 1994us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 04:45:36.669355 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 04:45:36.669355 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 04:45:36.669363 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0810 04:45:36.670749 (+ 1386us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 04:45:36.671109 (+ 360us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 04:45:36.671111 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 04:45:36.671111 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 04:45:36.671167 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 04:45:36.671383 (+ 216us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 04:45:36.671384 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 04:45:36.671384 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 04:45:36.671496 (+ 112us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 04:45:36.671551 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0810 04:45:36.671802 (+ 251us) server_negotiation.cc:299] Negotiation successful
0810 04:45:36.671878 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260810 04:45:36.672119 28982 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:45:36.664258 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.126:34487 (local address 127.0.0.1:60716)
0810 04:45:36.664417 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0810 04:45:36.664452 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0810 04:45:36.664913 (+ 461us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 04:45:36.665137 (+ 224us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 04:45:36.665141 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 04:45:36.665383 (+ 242us) client_negotiation.cc:189] Negotiated authn=SASL
0810 04:45:36.665574 (+ 191us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:45:36.665578 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:45:36.666259 (+ 681us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:45:36.666260 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:45:36.667025 (+ 765us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:45:36.667027 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:45:36.667373 (+ 346us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:45:36.667374 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:45:36.667487 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:45:36.667889 (+ 402us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 04:45:36.667896 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0810 04:45:36.669258 (+ 1362us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 04:45:36.670825 (+ 1567us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 04:45:36.670833 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 04:45:36.670834 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 04:45:36.671043 (+ 209us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 04:45:36.671221 (+ 178us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 04:45:36.671222 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 04:45:36.671222 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 04:45:36.671328 (+ 106us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 04:45:36.671550 (+ 222us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 04:45:36.671557 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 04:45:36.671687 (+ 130us) client_negotiation.cc:769] Sending connection context
0810 04:45:36.671771 (+ 84us) client_negotiation.cc:240] Negotiation successful
0810 04:45:36.671920 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":78,"threads_started":1}
I20260810 04:45:36.672704 28978 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.22.167.125" port: 35119 } has no permanent_uuid. Determining permanent_uuid...
Aug 10 04:45:36 dist-test-slave-bqcl krb5kdc[28563](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786337136, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.167.124@KRBTEST.COM
Aug 10 04:45:36 dist-test-slave-bqcl krb5kdc[28563](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786337136, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.167.124@KRBTEST.COM for kudu/127.22.167.125@KRBTEST.COM
I20260810 04:45:36.677246 28983 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:45:36.669504 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57694 (local address 127.22.167.124:33959)
0810 04:45:36.669641 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0810 04:45:36.669646 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 04:45:36.669763 (+ 117us) server_negotiation.cc:407] Connection header received
0810 04:45:36.669806 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:45:36.669809 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:45:36.669822 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:45:36.669888 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0810 04:45:36.670501 (+ 613us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:45:36.671163 (+ 662us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:45:36.671856 (+ 693us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:45:36.672217 (+ 361us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:45:36.672277 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:45:36.674559 (+ 2282us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 04:45:36.674566 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 04:45:36.674567 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 04:45:36.674573 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0810 04:45:36.676124 (+ 1551us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 04:45:36.676474 (+ 350us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 04:45:36.676476 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 04:45:36.676477 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 04:45:36.676525 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 04:45:36.676707 (+ 182us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 04:45:36.676707 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 04:45:36.676707 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 04:45:36.676848 (+ 141us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 04:45:36.676994 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0810 04:45:36.677072 (+ 78us) server_negotiation.cc:299] Negotiation successful
0810 04:45:36.677155 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":107,"thread_start_us":64,"threads_started":1}
I20260810 04:45:36.679179 28982 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:45:36.672931 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.125:35119 (local address 127.0.0.1:41080)
0810 04:45:36.672969 (+ 38us) negotiation.cc:107] Waiting for socket to connect
0810 04:45:36.672978 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0810 04:45:36.673012 (+ 34us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 04:45:36.673164 (+ 152us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 04:45:36.673165 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 04:45:36.673288 (+ 123us) client_negotiation.cc:189] Negotiated authn=SASL
0810 04:45:36.673331 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:45:36.673332 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:45:36.674107 (+ 775us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:45:36.674108 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:45:36.674718 (+ 610us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:45:36.674720 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:45:36.675170 (+ 450us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:45:36.675171 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:45:36.675240 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:45:36.675535 (+ 295us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 04:45:36.675539 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0810 04:45:36.676770 (+ 1231us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 04:45:36.678306 (+ 1536us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 04:45:36.678307 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 04:45:36.678308 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 04:45:36.678464 (+ 156us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 04:45:36.678628 (+ 164us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 04:45:36.678628 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 04:45:36.678628 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 04:45:36.678683 (+ 55us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 04:45:36.678905 (+ 222us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 04:45:36.678905 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 04:45:36.678945 (+ 40us) client_negotiation.cc:769] Sending connection context
0810 04:45:36.679002 (+ 57us) client_negotiation.cc:240] Negotiation successful
0810 04:45:36.679083 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":24}
I20260810 04:45:36.679180 28852 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:45:36.673002 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41080 (local address 127.22.167.125:35119)
0810 04:45:36.673066 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0810 04:45:36.673068 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 04:45:36.673076 (+ 8us) server_negotiation.cc:407] Connection header received
0810 04:45:36.673098 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:45:36.673099 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:45:36.673108 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:45:36.673152 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0810 04:45:36.673395 (+ 243us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:45:36.674024 (+ 629us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:45:36.674806 (+ 782us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:45:36.675103 (+ 297us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:45:36.675158 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:45:36.676829 (+ 1671us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 04:45:36.676830 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 04:45:36.676830 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 04:45:36.676837 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 04:45:36.678242 (+ 1405us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 04:45:36.678522 (+ 280us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 04:45:36.678523 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 04:45:36.678523 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 04:45:36.678575 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 04:45:36.678732 (+ 157us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 04:45:36.678733 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 04:45:36.678733 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 04:45:36.678835 (+ 102us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 04:45:36.678887 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0810 04:45:36.679032 (+ 145us) server_negotiation.cc:299] Negotiation successful
0810 04:45:36.679096 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260810 04:45:36.679795 28978 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.22.167.124" port: 33959 } has no permanent_uuid. Determining permanent_uuid...
Aug 10 04:45:36 dist-test-slave-bqcl krb5kdc[28563](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786337136, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.167.126@KRBTEST.COM
Aug 10 04:45:36 dist-test-slave-bqcl krb5kdc[28563](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786337136, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.167.124@KRBTEST.COM for kudu/127.22.167.124@KRBTEST.COM
I20260810 04:45:36.684866 28715 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:45:36.678168 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60718 (local address 127.22.167.126:34487)
0810 04:45:36.678204 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0810 04:45:36.678206 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 04:45:36.678214 (+ 8us) server_negotiation.cc:407] Connection header received
0810 04:45:36.678269 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:45:36.678269 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:45:36.678278 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:45:36.678322 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0810 04:45:36.678753 (+ 431us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:45:36.679179 (+ 426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:45:36.679882 (+ 703us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:45:36.680135 (+ 253us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:45:36.680190 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:45:36.682568 (+ 2378us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 04:45:36.682569 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 04:45:36.682570 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 04:45:36.682576 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0810 04:45:36.683863 (+ 1287us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 04:45:36.684181 (+ 318us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 04:45:36.684182 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 04:45:36.684182 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 04:45:36.684243 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 04:45:36.684380 (+ 137us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 04:45:36.684381 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 04:45:36.684381 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 04:45:36.684488 (+ 107us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 04:45:36.684671 (+ 183us) server_negotiation.cc:1035] Waiting for connection context
0810 04:45:36.684714 (+ 43us) server_negotiation.cc:299] Negotiation successful
0810 04:45:36.684782 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260810 04:45:36.686530 28982 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:45:36.679989 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.124:33959 (local address 127.0.0.1:57710)
0810 04:45:36.680027 (+ 38us) negotiation.cc:107] Waiting for socket to connect
0810 04:45:36.680036 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0810 04:45:36.680070 (+ 34us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 04:45:36.680183 (+ 113us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 04:45:36.680184 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 04:45:36.680283 (+ 99us) client_negotiation.cc:189] Negotiated authn=SASL
0810 04:45:36.680307 (+ 24us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:45:36.680308 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:45:36.681110 (+ 802us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:45:36.681111 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:45:36.681668 (+ 557us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:45:36.681671 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:45:36.681999 (+ 328us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:45:36.681999 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:45:36.682046 (+ 47us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:45:36.682368 (+ 322us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 04:45:36.682372 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0810 04:45:36.684135 (+ 1763us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 04:45:36.685666 (+ 1531us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 04:45:36.685667 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 04:45:36.685667 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 04:45:36.685817 (+ 150us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 04:45:36.685971 (+ 154us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 04:45:36.685972 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 04:45:36.685972 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 04:45:36.686023 (+ 51us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 04:45:36.686254 (+ 231us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 04:45:36.686255 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 04:45:36.686290 (+ 35us) client_negotiation.cc:769] Sending connection context
0810 04:45:36.686350 (+ 60us) client_negotiation.cc:240] Negotiation successful
0810 04:45:36.686437 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":24}
I20260810 04:45:36.686583 28983 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:45:36.680015 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57710 (local address 127.22.167.124:33959)
0810 04:45:36.680047 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0810 04:45:36.680048 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0810 04:45:36.680058 (+ 10us) server_negotiation.cc:407] Connection header received
0810 04:45:36.680112 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:45:36.680113 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:45:36.680122 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:45:36.680181 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0810 04:45:36.680368 (+ 187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:45:36.681037 (+ 669us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:45:36.681755 (+ 718us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:45:36.681938 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:45:36.681992 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:45:36.684209 (+ 2217us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 04:45:36.684210 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 04:45:36.684210 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 04:45:36.684215 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0810 04:45:36.685593 (+ 1378us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 04:45:36.685871 (+ 278us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 04:45:36.685872 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 04:45:36.685873 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 04:45:36.685921 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 04:45:36.686071 (+ 150us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 04:45:36.686071 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 04:45:36.686072 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 04:45:36.686203 (+ 131us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 04:45:36.686256 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0810 04:45:36.686375 (+ 119us) server_negotiation.cc:299] Negotiation successful
0810 04:45:36.686484 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":48,"server-negotiator.queue_time_us":15}
I20260810 04:45:36.687952 28978 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b493af56a8424b4cb0595223fe59d998: Bootstrap starting.
I20260810 04:45:36.688967 28978 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b493af56a8424b4cb0595223fe59d998: Neither blocks nor log segments found. Creating new log.
I20260810 04:45:36.689292 28978 log.cc:824] T 00000000000000000000000000000000 P b493af56a8424b4cb0595223fe59d998: Log is configured to *not* fsync() on all Append() calls
Aug 10 04:45:36 dist-test-slave-bqcl krb5kdc[28563](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786337136, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.167.125@KRBTEST.COM
I20260810 04:45:36.690089 28978 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b493af56a8424b4cb0595223fe59d998: No bootstrap required, opened a new log
I20260810 04:45:36.691570 28978 raft_consensus.cc:348] T 00000000000000000000000000000000 P b493af56a8424b4cb0595223fe59d998 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "619bf37a41d74e0b959137961b2cf7cc" member_type: VOTER last_known_addr { host: "127.22.167.126" port: 34487 } } peers { permanent_uuid: "314e7978cdec43ee973630752d9a16fa" member_type: VOTER last_known_addr { host: "127.22.167.125" port: 35119 } } peers { permanent_uuid: "b493af56a8424b4cb0595223fe59d998" member_type: VOTER last_known_addr { host: "127.22.167.124" port: 33959 } }
I20260810 04:45:36.691720 28978 raft_consensus.cc:729] T 00000000000000000000000000000000 P b493af56a8424b4cb0595223fe59d998 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b493af56a8424b4cb0595223fe59d998, State: Initialized, Role: FOLLOWER
I20260810 04:45:36.691794 28978 consensus_queue.cc:260] T 00000000000000000000000000000000 P b493af56a8424b4cb0595223fe59d998 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "619bf37a41d74e0b959137961b2cf7cc" member_type: VOTER last_known_addr { host: "127.22.167.126" port: 34487 } } peers { permanent_uuid: "314e7978cdec43ee973630752d9a16fa" member_type: VOTER last_known_addr { host: "127.22.167.125" port: 35119 } } peers { permanent_uuid: "b493af56a8424b4cb0595223fe59d998" member_type: VOTER last_known_addr { host: "127.22.167.124" port: 33959 } }
I20260810 04:45:36.692092 28978 sys_catalog.cc:564] T 00000000000000000000000000000000 P b493af56a8424b4cb0595223fe59d998 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 04:45:36.692085 28987 sys_catalog.cc:455] T 00000000000000000000000000000000 P b493af56a8424b4cb0595223fe59d998 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "619bf37a41d74e0b959137961b2cf7cc" member_type: VOTER last_known_addr { host: "127.22.167.126" port: 34487 } } peers { permanent_uuid: "314e7978cdec43ee973630752d9a16fa" member_type: VOTER last_known_addr { host: "127.22.167.125" port: 35119 } } peers { permanent_uuid: "b493af56a8424b4cb0595223fe59d998" member_type: VOTER last_known_addr { host: "127.22.167.124" port: 33959 } } }
I20260810 04:45:36.692189 28987 sys_catalog.cc:458] T 00000000000000000000000000000000 P b493af56a8424b4cb0595223fe59d998 [sys.catalog]: This master's current role is: FOLLOWER
I20260810 04:45:36.692678 28852 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:45:36.685522 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41084 (local address 127.22.167.125:35119)
0810 04:45:36.685574 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0810 04:45:36.685576 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 04:45:36.685600 (+ 24us) server_negotiation.cc:407] Connection header received
0810 04:45:36.685670 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:45:36.685671 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:45:36.685680 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:45:36.685725 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0810 04:45:36.686155 (+ 430us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:45:36.686576 (+ 421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:45:36.687334 (+ 758us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:45:36.687605 (+ 271us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:45:36.687691 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:45:36.690030 (+ 2339us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 04:45:36.690031 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 04:45:36.690031 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 04:45:36.690037 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0810 04:45:36.691312 (+ 1275us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 04:45:36.691675 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 04:45:36.691675 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 04:45:36.691675 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 04:45:36.691724 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 04:45:36.692002 (+ 278us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 04:45:36.692003 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 04:45:36.692003 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 04:45:36.692147 (+ 144us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 04:45:36.692317 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0810 04:45:36.692497 (+ 180us) server_negotiation.cc:299] Negotiation successful
0810 04:45:36.692578 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
W20260810 04:45:36.693794 29008 catalog_manager.cc:1558] T 00000000000000000000000000000000 P b493af56a8424b4cb0595223fe59d998: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260810 04:45:36.693879 29008 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260810 04:45:36.694154 23197 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 04:45:36.694046 28859 master_runner.cc:186] Error getting master registration for 127.22.167.126:34487: OK, instance_id { permanent_uuid: "619bf37a41d74e0b959137961b2cf7cc" instance_seqno: 1786337136357559 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260810 04:45:36.695974 28859 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260810 04:45:36.695395 28660 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "619bf37a41d74e0b959137961b2cf7cc"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60718
I20260810 04:45:36.697479 28710 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.22.167.124" port: 33959 } attempt: 2
Aug 10 04:45:36 dist-test-slave-bqcl krb5kdc[28563](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786337136, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.167.126@KRBTEST.COM for kudu/127.22.167.124@KRBTEST.COM
I20260810 04:45:36.705487 28712 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:45:36.697773 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.124:33959 (local address 127.0.0.1:57712)
0810 04:45:36.697818 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0810 04:45:36.697827 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0810 04:45:36.697861 (+ 34us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 04:45:36.698012 (+ 151us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 04:45:36.698012 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 04:45:36.698210 (+ 198us) client_negotiation.cc:189] Negotiated authn=SASL
0810 04:45:36.698251 (+ 41us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:45:36.698254 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:45:36.699069 (+ 815us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:45:36.699070 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:45:36.700210 (+ 1140us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:45:36.700213 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:45:36.700529 (+ 316us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:45:36.700530 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:45:36.700584 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:45:36.700988 (+ 404us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 04:45:36.700993 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0810 04:45:36.702476 (+ 1483us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 04:45:36.704368 (+ 1892us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 04:45:36.704369 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 04:45:36.704370 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 04:45:36.704599 (+ 229us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 04:45:36.704807 (+ 208us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 04:45:36.704808 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 04:45:36.704808 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 04:45:36.704866 (+ 58us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 04:45:36.705179 (+ 313us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 04:45:36.705180 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 04:45:36.705224 (+ 44us) client_negotiation.cc:769] Sending connection context
0810 04:45:36.705304 (+ 80us) client_negotiation.cc:240] Negotiation successful
0810 04:45:36.705394 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":30}
I20260810 04:45:36.705760 28983 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:45:36.697845 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57712 (local address 127.22.167.124:33959)
0810 04:45:36.697883 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0810 04:45:36.697885 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 04:45:36.697907 (+ 22us) server_negotiation.cc:407] Connection header received
0810 04:45:36.697927 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:45:36.697928 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:45:36.697936 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:45:36.697989 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0810 04:45:36.698314 (+ 325us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:45:36.698999 (+ 685us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:45:36.700285 (+ 1286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:45:36.700473 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:45:36.700525 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:45:36.702564 (+ 2039us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 04:45:36.702565 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 04:45:36.702565 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 04:45:36.702571 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0810 04:45:36.704277 (+ 1706us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 04:45:36.704679 (+ 402us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 04:45:36.704680 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 04:45:36.704680 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 04:45:36.704739 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 04:45:36.704929 (+ 190us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 04:45:36.704930 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 04:45:36.704930 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 04:45:36.705078 (+ 148us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 04:45:36.705138 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0810 04:45:36.705606 (+ 468us) server_negotiation.cc:299] Negotiation successful
0810 04:45:36.705678 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260810 04:45:36.707262 28710 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 619bf37a41d74e0b959137961b2cf7cc: Bootstrap starting.
I20260810 04:45:36.707868 28710 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 619bf37a41d74e0b959137961b2cf7cc: Neither blocks nor log segments found. Creating new log.
I20260810 04:45:36.708164 28710 log.cc:824] T 00000000000000000000000000000000 P 619bf37a41d74e0b959137961b2cf7cc: Log is configured to *not* fsync() on all Append() calls
I20260810 04:45:36.708853 28710 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 619bf37a41d74e0b959137961b2cf7cc: No bootstrap required, opened a new log
I20260810 04:45:36.709810 28710 raft_consensus.cc:348] T 00000000000000000000000000000000 P 619bf37a41d74e0b959137961b2cf7cc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "619bf37a41d74e0b959137961b2cf7cc" member_type: VOTER last_known_addr { host: "127.22.167.126" port: 34487 } } peers { permanent_uuid: "314e7978cdec43ee973630752d9a16fa" member_type: VOTER last_known_addr { host: "127.22.167.125" port: 35119 } } peers { permanent_uuid: "b493af56a8424b4cb0595223fe59d998" member_type: VOTER last_known_addr { host: "127.22.167.124" port: 33959 } }
I20260810 04:45:36.709918 28710 raft_consensus.cc:729] T 00000000000000000000000000000000 P 619bf37a41d74e0b959137961b2cf7cc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 619bf37a41d74e0b959137961b2cf7cc, State: Initialized, Role: FOLLOWER
I20260810 04:45:36.709990 28710 consensus_queue.cc:260] T 00000000000000000000000000000000 P 619bf37a41d74e0b959137961b2cf7cc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "619bf37a41d74e0b959137961b2cf7cc" member_type: VOTER last_known_addr { host: "127.22.167.126" port: 34487 } } peers { permanent_uuid: "314e7978cdec43ee973630752d9a16fa" member_type: VOTER last_known_addr { host: "127.22.167.125" port: 35119 } } peers { permanent_uuid: "b493af56a8424b4cb0595223fe59d998" member_type: VOTER last_known_addr { host: "127.22.167.124" port: 33959 } }
I20260810 04:45:36.710283 29013 sys_catalog.cc:455] T 00000000000000000000000000000000 P 619bf37a41d74e0b959137961b2cf7cc [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "619bf37a41d74e0b959137961b2cf7cc" member_type: VOTER last_known_addr { host: "127.22.167.126" port: 34487 } } peers { permanent_uuid: "314e7978cdec43ee973630752d9a16fa" member_type: VOTER last_known_addr { host: "127.22.167.125" port: 35119 } } peers { permanent_uuid: "b493af56a8424b4cb0595223fe59d998" member_type: VOTER last_known_addr { host: "127.22.167.124" port: 33959 } } }
I20260810 04:45:36.710333 28710 sys_catalog.cc:564] T 00000000000000000000000000000000 P 619bf37a41d74e0b959137961b2cf7cc [sys.catalog]: configured and running, proceeding with master startup.
I20260810 04:45:36.710372 29013 sys_catalog.cc:458] T 00000000000000000000000000000000 P 619bf37a41d74e0b959137961b2cf7cc [sys.catalog]: This master's current role is: FOLLOWER
I20260810 04:45:36.710379 28660 raft_consensus.cc:482] T 00000000000000000000000000000000 P 619bf37a41d74e0b959137961b2cf7cc [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260810 04:45:36.710533 28660 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 619bf37a41d74e0b959137961b2cf7cc [term 0 FOLLOWER]: Advancing to term 1
I20260810 04:45:36.711263 28660 raft_consensus.cc:504] T 00000000000000000000000000000000 P 619bf37a41d74e0b959137961b2cf7cc [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "619bf37a41d74e0b959137961b2cf7cc" member_type: VOTER last_known_addr { host: "127.22.167.126" port: 34487 } } peers { permanent_uuid: "314e7978cdec43ee973630752d9a16fa" member_type: VOTER last_known_addr { host: "127.22.167.125" port: 35119 } } peers { permanent_uuid: "b493af56a8424b4cb0595223fe59d998" member_type: VOTER last_known_addr { host: "127.22.167.124" port: 33959 } }
W20260810 04:45:36.711985 29028 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 619bf37a41d74e0b959137961b2cf7cc: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260810 04:45:36.712029 29028 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260810 04:45:36.712261 28932 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "619bf37a41d74e0b959137961b2cf7cc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "b493af56a8424b4cb0595223fe59d998"
I20260810 04:45:36.712369 28932 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b493af56a8424b4cb0595223fe59d998 [term 0 FOLLOWER]: Advancing to term 1
I20260810 04:45:36.713186 28932 raft_consensus.cc:2454] T 00000000000000000000000000000000 P b493af56a8424b4cb0595223fe59d998 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 619bf37a41d74e0b959137961b2cf7cc in term 1.
I20260810 04:45:36.713397 28621 leader_election.cc:302] T 00000000000000000000000000000000 P 619bf37a41d74e0b959137961b2cf7cc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 619bf37a41d74e0b959137961b2cf7cc, b493af56a8424b4cb0595223fe59d998; no voters:
I20260810 04:45:36.713507 29013 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 619bf37a41d74e0b959137961b2cf7cc [term 1 FOLLOWER]: Leader election won for term 1
I20260810 04:45:36.713652 29013 raft_consensus.cc:686] T 00000000000000000000000000000000 P 619bf37a41d74e0b959137961b2cf7cc [term 1 LEADER]: Becoming Leader. State: Replica: 619bf37a41d74e0b959137961b2cf7cc, State: Running, Role: LEADER
I20260810 04:45:36.713732 29013 consensus_queue.cc:237] T 00000000000000000000000000000000 P 619bf37a41d74e0b959137961b2cf7cc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "619bf37a41d74e0b959137961b2cf7cc" member_type: VOTER last_known_addr { host: "127.22.167.126" port: 34487 } } peers { permanent_uuid: "314e7978cdec43ee973630752d9a16fa" member_type: VOTER last_known_addr { host: "127.22.167.125" port: 35119 } } peers { permanent_uuid: "b493af56a8424b4cb0595223fe59d998" member_type: VOTER last_known_addr { host: "127.22.167.124" port: 33959 } }
I20260810 04:45:36.714102 29013 sys_catalog.cc:455] T 00000000000000000000000000000000 P 619bf37a41d74e0b959137961b2cf7cc [sys.catalog]: SysCatalogTable state changed. Reason: New leader 619bf37a41d74e0b959137961b2cf7cc. Latest consensus state: current_term: 1 leader_uuid: "619bf37a41d74e0b959137961b2cf7cc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "619bf37a41d74e0b959137961b2cf7cc" member_type: VOTER last_known_addr { host: "127.22.167.126" port: 34487 } } peers { permanent_uuid: "314e7978cdec43ee973630752d9a16fa" member_type: VOTER last_known_addr { host: "127.22.167.125" port: 35119 } } peers { permanent_uuid: "b493af56a8424b4cb0595223fe59d998" member_type: VOTER last_known_addr { host: "127.22.167.124" port: 33959 } } }
I20260810 04:45:36.714231 29013 sys_catalog.cc:458] T 00000000000000000000000000000000 P 619bf37a41d74e0b959137961b2cf7cc [sys.catalog]: This master's current role is: LEADER
I20260810 04:45:36.714589 28660 leader_election.cc:288] T 00000000000000000000000000000000 P 619bf37a41d74e0b959137961b2cf7cc [CANDIDATE]: Term 1 election: Requested vote from peers 314e7978cdec43ee973630752d9a16fa (127.22.167.125:35119), b493af56a8424b4cb0595223fe59d998 (127.22.167.124:33959)
I20260810 04:45:36.712955 28590 master_runner.cc:186] Error getting master registration for 127.22.167.125:35119: OK, instance_id { permanent_uuid: "314e7978cdec43ee973630752d9a16fa" instance_seqno: 1786337136507764 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260810 04:45:36.715178 28590 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260810 04:45:36.715077 29035 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 04:45:36.715837 29035 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 04:45:36.717758 28932 raft_consensus.cc:1261] T 00000000000000000000000000000000 P b493af56a8424b4cb0595223fe59d998 [term 1 FOLLOWER]: Refusing update from remote peer 619bf37a41d74e0b959137961b2cf7cc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 04:45:36.718186 29031 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 619bf37a41d74e0b959137961b2cf7cc [LEADER]: Connected to new peer: Peer: permanent_uuid: "b493af56a8424b4cb0595223fe59d998" member_type: VOTER last_known_addr { host: "127.22.167.124" port: 33959 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 04:45:36.719511 28987 sys_catalog.cc:455] T 00000000000000000000000000000000 P b493af56a8424b4cb0595223fe59d998 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 619bf37a41d74e0b959137961b2cf7cc. Latest consensus state: current_term: 1 leader_uuid: "619bf37a41d74e0b959137961b2cf7cc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "619bf37a41d74e0b959137961b2cf7cc" member_type: VOTER last_known_addr { host: "127.22.167.126" port: 34487 } } peers { permanent_uuid: "314e7978cdec43ee973630752d9a16fa" member_type: VOTER last_known_addr { host: "127.22.167.125" port: 35119 } } peers { permanent_uuid: "b493af56a8424b4cb0595223fe59d998" member_type: VOTER last_known_addr { host: "127.22.167.124" port: 33959 } } }
I20260810 04:45:36.719602 28987 sys_catalog.cc:458] T 00000000000000000000000000000000 P b493af56a8424b4cb0595223fe59d998 [sys.catalog]: This master's current role is: FOLLOWER
I20260810 04:45:36.723256 29031 sys_catalog.cc:455] T 00000000000000000000000000000000 P 619bf37a41d74e0b959137961b2cf7cc [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "619bf37a41d74e0b959137961b2cf7cc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "619bf37a41d74e0b959137961b2cf7cc" member_type: VOTER last_known_addr { host: "127.22.167.126" port: 34487 } } peers { permanent_uuid: "314e7978cdec43ee973630752d9a16fa" member_type: VOTER last_known_addr { host: "127.22.167.125" port: 35119 } } peers { permanent_uuid: "b493af56a8424b4cb0595223fe59d998" member_type: VOTER last_known_addr { host: "127.22.167.124" port: 33959 } } }
I20260810 04:45:36.723331 29031 sys_catalog.cc:458] T 00000000000000000000000000000000 P 619bf37a41d74e0b959137961b2cf7cc [sys.catalog]: This master's current role is: LEADER
I20260810 04:45:36.723811 28987 sys_catalog.cc:455] T 00000000000000000000000000000000 P b493af56a8424b4cb0595223fe59d998 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "619bf37a41d74e0b959137961b2cf7cc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "619bf37a41d74e0b959137961b2cf7cc" member_type: VOTER last_known_addr { host: "127.22.167.126" port: 34487 } } peers { permanent_uuid: "314e7978cdec43ee973630752d9a16fa" member_type: VOTER last_known_addr { host: "127.22.167.125" port: 35119 } } peers { permanent_uuid: "b493af56a8424b4cb0595223fe59d998" member_type: VOTER last_known_addr { host: "127.22.167.124" port: 33959 } } }
I20260810 04:45:36.723883 28987 sys_catalog.cc:458] T 00000000000000000000000000000000 P b493af56a8424b4cb0595223fe59d998 [sys.catalog]: This master's current role is: FOLLOWER
I20260810 04:45:36.724385 29045 mvcc.cc:205] Tried to move back new op lower bound from 7316836911993036800 to 7316836911980072960. Current Snapshot: MvccSnapshot[applied={T|T < 7316836911993036800}]
I20260810 04:45:36.724594 29042 mvcc.cc:205] Tried to move back new op lower bound from 7316836911993036800 to 7316836911980072960. Current Snapshot: MvccSnapshot[applied={T|T < 7316836911993036800}]
I20260810 04:45:36.725592 28983 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:45:36.717051 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57714 (local address 127.22.167.124:33959)
0810 04:45:36.717084 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0810 04:45:36.717086 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 04:45:36.717340 (+ 254us) server_negotiation.cc:407] Connection header received
0810 04:45:36.717435 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:45:36.717436 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:45:36.717445 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:45:36.717472 (+ 27us) server_negotiation.cc:226] Negotiated authn=SASL
0810 04:45:36.718415 (+ 943us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:45:36.719118 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:45:36.720253 (+ 1135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:45:36.720540 (+ 287us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:45:36.720576 (+ 36us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:45:36.722093 (+ 1517us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 04:45:36.722095 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 04:45:36.722095 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 04:45:36.722108 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0810 04:45:36.723863 (+ 1755us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 04:45:36.724348 (+ 485us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 04:45:36.724349 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 04:45:36.724349 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 04:45:36.724429 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 04:45:36.724776 (+ 347us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 04:45:36.724776 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 04:45:36.724777 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 04:45:36.724901 (+ 124us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 04:45:36.724980 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0810 04:45:36.725428 (+ 448us) server_negotiation.cc:299] Negotiation successful
0810 04:45:36.725505 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260810 04:45:36.726763 29035 catalog_manager.cc:1347] Generated new cluster ID: 0f5a1bbe6abe45bebd4468bf2e96e645
I20260810 04:45:36.726807 29035 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 04:45:36.733639 28715 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:45:36.724865 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60734 (local address 127.22.167.126:34487)
0810 04:45:36.724900 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0810 04:45:36.724902 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 04:45:36.726057 (+ 1155us) server_negotiation.cc:407] Connection header received
0810 04:45:36.726243 (+ 186us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:45:36.726245 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:45:36.726256 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:45:36.726317 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0810 04:45:36.726937 (+ 620us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:45:36.727614 (+ 677us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:45:36.728569 (+ 955us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:45:36.728874 (+ 305us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:45:36.728946 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:45:36.730147 (+ 1201us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 04:45:36.730148 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 04:45:36.730149 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 04:45:36.730155 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0810 04:45:36.731880 (+ 1725us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 04:45:36.732524 (+ 644us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 04:45:36.732525 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 04:45:36.732526 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 04:45:36.732586 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 04:45:36.732914 (+ 328us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 04:45:36.732914 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 04:45:36.732915 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 04:45:36.733039 (+ 124us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 04:45:36.733118 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0810 04:45:36.733460 (+ 342us) server_negotiation.cc:299] Negotiation successful
0810 04:45:36.733552 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260810 04:45:36.739816 28844 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.22.167.124" port: 33959 } attempt: 3
Aug 10 04:45:36 dist-test-slave-bqcl krb5kdc[28563](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786337136, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.167.125@KRBTEST.COM for kudu/127.22.167.124@KRBTEST.COM
I20260810 04:45:36.744627 28852 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:45:36.725154 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41096 (local address 127.22.167.125:35119)
0810 04:45:36.725196 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0810 04:45:36.725198 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 04:45:36.734642 (+ 9444us) server_negotiation.cc:407] Connection header received
0810 04:45:36.734791 (+ 149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:45:36.734792 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:45:36.734815 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:45:36.734867 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0810 04:45:36.735954 (+ 1087us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:45:36.736700 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:45:36.738339 (+ 1639us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:45:36.738665 (+ 326us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:45:36.738707 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:45:36.740920 (+ 2213us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 04:45:36.740921 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 04:45:36.740922 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 04:45:36.740942 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0810 04:45:36.742685 (+ 1743us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 04:45:36.743372 (+ 687us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 04:45:36.743373 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 04:45:36.743374 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 04:45:36.743434 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 04:45:36.743783 (+ 349us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 04:45:36.743784 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 04:45:36.743785 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 04:45:36.743928 (+ 143us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 04:45:36.743993 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0810 04:45:36.744440 (+ 447us) server_negotiation.cc:299] Negotiation successful
0810 04:45:36.744522 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260810 04:45:36.747273 28849 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:45:36.740047 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.124:33959 (local address 127.0.0.1:57728)
0810 04:45:36.740093 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0810 04:45:36.740105 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0810 04:45:36.740153 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 04:45:36.740330 (+ 177us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 04:45:36.740331 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 04:45:36.740496 (+ 165us) client_negotiation.cc:189] Negotiated authn=SASL
0810 04:45:36.740533 (+ 37us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:45:36.740535 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:45:36.741387 (+ 852us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:45:36.741388 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:45:36.742185 (+ 797us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:45:36.742187 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:45:36.742659 (+ 472us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:45:36.742660 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:45:36.742716 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:45:36.743092 (+ 376us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 04:45:36.743096 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0810 04:45:36.744484 (+ 1388us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 04:45:36.746231 (+ 1747us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 04:45:36.746232 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 04:45:36.746232 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 04:45:36.746432 (+ 200us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 04:45:36.746694 (+ 262us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 04:45:36.746695 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 04:45:36.746696 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 04:45:36.746753 (+ 57us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 04:45:36.746999 (+ 246us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 04:45:36.746999 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 04:45:36.747039 (+ 40us) client_negotiation.cc:769] Sending connection context
0810 04:45:36.747097 (+ 58us) client_negotiation.cc:240] Negotiation successful
0810 04:45:36.747184 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":23}
I20260810 04:45:36.747607 28983 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:45:36.740150 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57728 (local address 127.22.167.124:33959)
0810 04:45:36.740201 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0810 04:45:36.740202 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0810 04:45:36.740210 (+ 8us) server_negotiation.cc:407] Connection header received
0810 04:45:36.740229 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:45:36.740230 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:45:36.740237 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:45:36.740288 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0810 04:45:36.740607 (+ 319us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:45:36.741303 (+ 696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:45:36.742275 (+ 972us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:45:36.742545 (+ 270us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:45:36.742598 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:45:36.744572 (+ 1974us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 04:45:36.744573 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 04:45:36.744573 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 04:45:36.744579 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0810 04:45:36.746118 (+ 1539us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 04:45:36.746558 (+ 440us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 04:45:36.746559 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 04:45:36.746559 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 04:45:36.746614 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 04:45:36.746814 (+ 200us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 04:45:36.746814 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 04:45:36.746814 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 04:45:36.746925 (+ 111us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 04:45:36.746976 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0810 04:45:36.747452 (+ 476us) server_negotiation.cc:299] Negotiation successful
0810 04:45:36.747521 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260810 04:45:36.749087 28844 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 314e7978cdec43ee973630752d9a16fa: Bootstrap starting.
I20260810 04:45:36.749845 28844 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 314e7978cdec43ee973630752d9a16fa: Neither blocks nor log segments found. Creating new log.
I20260810 04:45:36.750166 28844 log.cc:824] T 00000000000000000000000000000000 P 314e7978cdec43ee973630752d9a16fa: Log is configured to *not* fsync() on all Append() calls
I20260810 04:45:36.754189 29035 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 04:45:36.754770 28844 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 314e7978cdec43ee973630752d9a16fa: No bootstrap required, opened a new log
I20260810 04:45:36.755095 29035 catalog_manager.cc:1504] Loading token signing keys...
I20260810 04:45:36.756664 28844 raft_consensus.cc:348] T 00000000000000000000000000000000 P 314e7978cdec43ee973630752d9a16fa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "619bf37a41d74e0b959137961b2cf7cc" member_type: VOTER last_known_addr { host: "127.22.167.126" port: 34487 } } peers { permanent_uuid: "314e7978cdec43ee973630752d9a16fa" member_type: VOTER last_known_addr { host: "127.22.167.125" port: 35119 } } peers { permanent_uuid: "b493af56a8424b4cb0595223fe59d998" member_type: VOTER last_known_addr { host: "127.22.167.124" port: 33959 } }
I20260810 04:45:36.756798 28844 raft_consensus.cc:729] T 00000000000000000000000000000000 P 314e7978cdec43ee973630752d9a16fa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 314e7978cdec43ee973630752d9a16fa, State: Initialized, Role: FOLLOWER
I20260810 04:45:36.756870 28844 consensus_queue.cc:260] T 00000000000000000000000000000000 P 314e7978cdec43ee973630752d9a16fa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "619bf37a41d74e0b959137961b2cf7cc" member_type: VOTER last_known_addr { host: "127.22.167.126" port: 34487 } } peers { permanent_uuid: "314e7978cdec43ee973630752d9a16fa" member_type: VOTER last_known_addr { host: "127.22.167.125" port: 35119 } } peers { permanent_uuid: "b493af56a8424b4cb0595223fe59d998" member_type: VOTER last_known_addr { host: "127.22.167.124" port: 33959 } }
I20260810 04:45:36.757155 28844 sys_catalog.cc:564] T 00000000000000000000000000000000 P 314e7978cdec43ee973630752d9a16fa [sys.catalog]: configured and running, proceeding with master startup.
I20260810 04:45:36.757211 28799 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "619bf37a41d74e0b959137961b2cf7cc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "314e7978cdec43ee973630752d9a16fa"
I20260810 04:45:36.757320 28799 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 314e7978cdec43ee973630752d9a16fa [term 0 FOLLOWER]: Advancing to term 1
I20260810 04:45:36.758153 28799 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 314e7978cdec43ee973630752d9a16fa [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 619bf37a41d74e0b959137961b2cf7cc in term 1.
I20260810 04:45:36.758615 29055 sys_catalog.cc:455] T 00000000000000000000000000000000 P 314e7978cdec43ee973630752d9a16fa [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "619bf37a41d74e0b959137961b2cf7cc" member_type: VOTER last_known_addr { host: "127.22.167.126" port: 34487 } } peers { permanent_uuid: "314e7978cdec43ee973630752d9a16fa" member_type: VOTER last_known_addr { host: "127.22.167.125" port: 35119 } } peers { permanent_uuid: "b493af56a8424b4cb0595223fe59d998" member_type: VOTER last_known_addr { host: "127.22.167.124" port: 33959 } } }
I20260810 04:45:36.758772 29055 sys_catalog.cc:458] T 00000000000000000000000000000000 P 314e7978cdec43ee973630752d9a16fa [sys.catalog]: This master's current role is: FOLLOWER
I20260810 04:45:36.758368 28797 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 314e7978cdec43ee973630752d9a16fa [term 1 FOLLOWER]: Refusing update from remote peer 619bf37a41d74e0b959137961b2cf7cc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 04:45:36.759135 29031 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 619bf37a41d74e0b959137961b2cf7cc [LEADER]: Connected to new peer: Peer: permanent_uuid: "314e7978cdec43ee973630752d9a16fa" member_type: VOTER last_known_addr { host: "127.22.167.125" port: 35119 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260810 04:45:36.762956 29079 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 314e7978cdec43ee973630752d9a16fa: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
I20260810 04:45:36.762985 29055 sys_catalog.cc:455] T 00000000000000000000000000000000 P 314e7978cdec43ee973630752d9a16fa [sys.catalog]: SysCatalogTable state changed. Reason: New leader 619bf37a41d74e0b959137961b2cf7cc. Latest consensus state: current_term: 1 leader_uuid: "619bf37a41d74e0b959137961b2cf7cc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "619bf37a41d74e0b959137961b2cf7cc" member_type: VOTER last_known_addr { host: "127.22.167.126" port: 34487 } } peers { permanent_uuid: "314e7978cdec43ee973630752d9a16fa" member_type: VOTER last_known_addr { host: "127.22.167.125" port: 35119 } } peers { permanent_uuid: "b493af56a8424b4cb0595223fe59d998" member_type: VOTER last_known_addr { host: "127.22.167.124" port: 33959 } } }
I20260810 04:45:36.763207 29055 sys_catalog.cc:458] T 00000000000000000000000000000000 P 314e7978cdec43ee973630752d9a16fa [sys.catalog]: This master's current role is: FOLLOWER
I20260810 04:45:36.763286 29055 sys_catalog.cc:455] T 00000000000000000000000000000000 P 314e7978cdec43ee973630752d9a16fa [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "619bf37a41d74e0b959137961b2cf7cc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "619bf37a41d74e0b959137961b2cf7cc" member_type: VOTER last_known_addr { host: "127.22.167.126" port: 34487 } } peers { permanent_uuid: "314e7978cdec43ee973630752d9a16fa" member_type: VOTER last_known_addr { host: "127.22.167.125" port: 35119 } } peers { permanent_uuid: "b493af56a8424b4cb0595223fe59d998" member_type: VOTER last_known_addr { host: "127.22.167.124" port: 33959 } } }
I20260810 04:45:36.763391 29055 sys_catalog.cc:458] T 00000000000000000000000000000000 P 314e7978cdec43ee973630752d9a16fa [sys.catalog]: This master's current role is: FOLLOWER
W20260810 04:45:36.763008 29079 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260810 04:45:36.764619 29031 sys_catalog.cc:455] T 00000000000000000000000000000000 P 619bf37a41d74e0b959137961b2cf7cc [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "619bf37a41d74e0b959137961b2cf7cc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "619bf37a41d74e0b959137961b2cf7cc" member_type: VOTER last_known_addr { host: "127.22.167.126" port: 34487 } } peers { permanent_uuid: "314e7978cdec43ee973630752d9a16fa" member_type: VOTER last_known_addr { host: "127.22.167.125" port: 35119 } } peers { permanent_uuid: "b493af56a8424b4cb0595223fe59d998" member_type: VOTER last_known_addr { host: "127.22.167.124" port: 33959 } } }
I20260810 04:45:36.764712 29031 sys_catalog.cc:458] T 00000000000000000000000000000000 P 619bf37a41d74e0b959137961b2cf7cc [sys.catalog]: This master's current role is: LEADER
I20260810 04:45:36.765332 29035 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 619bf37a41d74e0b959137961b2cf7cc: Generated new TSK 0
I20260810 04:45:36.765522 29035 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 04:45:36.772244 29080 mvcc.cc:205] Tried to move back new op lower bound from 7316836912141266944 to 7316836911980072960. Current Snapshot: MvccSnapshot[applied={T|T < 7316836911993036800}]
W20260810 04:45:36.787473 28983 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 04:45:36.786386 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57738 (local address 127.22.167.124:33959)
0810 04:45:36.786430 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0810 04:45:36.786431 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0810 04:45:36.786819 (+ 388us) server_negotiation.cc:407] Connection header received
0810 04:45:36.786927 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:45:36.786928 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:45:36.786938 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:45:36.786968 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0810 04:45:36.787410 (+ 442us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:57738: BlockingRecv error: recv got EOF from 127.0.0.1:57738 (error 108)
Metrics: {"server-negotiator.queue_time_us":22}
W20260810 04:45:36.788812 28715 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 04:45:36.787911 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60742 (local address 127.22.167.126:34487)
0810 04:45:36.787950 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0810 04:45:36.787952 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 04:45:36.788272 (+ 320us) server_negotiation.cc:407] Connection header received
0810 04:45:36.788371 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:45:36.788372 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:45:36.788383 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:45:36.788411 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0810 04:45:36.788747 (+ 336us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60742: BlockingRecv error: recv got EOF from 127.0.0.1:60742 (error 108)
Metrics: {"server-negotiator.queue_time_us":20}
W20260810 04:45:36.789647 28852 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 04:45:36.788152 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41112 (local address 127.22.167.125:35119)
0810 04:45:36.788191 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0810 04:45:36.788192 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0810 04:45:36.789122 (+ 930us) server_negotiation.cc:407] Connection header received
0810 04:45:36.789219 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:45:36.789220 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:45:36.789230 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:45:36.789259 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0810 04:45:36.789589 (+ 330us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:41112: BlockingRecv error: recv got EOF from 127.0.0.1:41112 (error 108)
Metrics: {"server-negotiator.queue_time_us":20}
W20260810 04:45:36.795131 28852 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 04:45:36.794364 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41128 (local address 127.22.167.125:35119)
0810 04:45:36.794396 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0810 04:45:36.794398 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 04:45:36.794637 (+ 239us) server_negotiation.cc:407] Connection header received
0810 04:45:36.794739 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:45:36.794740 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:45:36.794756 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:45:36.794784 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0810 04:45:36.795091 (+ 307us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:41128: BlockingRecv error: recv got EOF from 127.0.0.1:41128 (error 108)
Metrics: {"server-negotiator.queue_time_us":16}
W20260810 04:45:36.798066 28852 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 04:45:36.797340 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41134 (local address 127.22.167.125:35119)
0810 04:45:36.797370 (+ 30us) server_negotiation.cc:206] Beginning negotiation
0810 04:45:36.797371 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0810 04:45:36.797590 (+ 219us) server_negotiation.cc:407] Connection header received
0810 04:45:36.797683 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:45:36.797684 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:45:36.797693 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:45:36.797722 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0810 04:45:36.798027 (+ 305us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:41134: BlockingRecv error: recv got EOF from 127.0.0.1:41134 (error 108)
Metrics: {"server-negotiator.queue_time_us":16}
W20260810 04:45:36.801944 28852 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 04:45:36.801226 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41142 (local address 127.22.167.125:35119)
0810 04:45:36.801258 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0810 04:45:36.801259 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0810 04:45:36.801467 (+ 208us) server_negotiation.cc:407] Connection header received
0810 04:45:36.801572 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:45:36.801573 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:45:36.801582 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:45:36.801608 (+ 26us) server_negotiation.cc:226] Negotiated authn=SASL
0810 04:45:36.801906 (+ 298us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:41142: BlockingRecv error: recv got EOF from 127.0.0.1:41142 (error 108)
Metrics: {"server-negotiator.queue_time_us":17}
W20260810 04:45:36.807868 28852 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 04:45:36.807151 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41156 (local address 127.22.167.125:35119)
0810 04:45:36.807183 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0810 04:45:36.807185 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 04:45:36.807387 (+ 202us) server_negotiation.cc:407] Connection header received
0810 04:45:36.807478 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:45:36.807479 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:45:36.807488 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:45:36.807528 (+ 40us) server_negotiation.cc:226] Negotiated authn=SASL
0810 04:45:36.807830 (+ 302us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:41156: BlockingRecv error: recv got EOF from 127.0.0.1:41156 (error 108)
Metrics: {"server-negotiator.queue_time_us":17}
W20260810 04:45:36.817750 28852 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 04:45:36.817021 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41164 (local address 127.22.167.125:35119)
0810 04:45:36.817052 (+ 31us) server_negotiation.cc:206] Beginning negotiation
0810 04:45:36.817054 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 04:45:36.817262 (+ 208us) server_negotiation.cc:407] Connection header received
0810 04:45:36.817352 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:45:36.817353 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:45:36.817362 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:45:36.817389 (+ 27us) server_negotiation.cc:226] Negotiated authn=SASL
0810 04:45:36.817711 (+ 322us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:41164: BlockingRecv error: recv got EOF from 127.0.0.1:41164 (error 108)
Metrics: {"server-negotiator.queue_time_us":17}
W20260810 04:45:36.835978 28852 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 04:45:36.835122 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41172 (local address 127.22.167.125:35119)
0810 04:45:36.835163 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0810 04:45:36.835164 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0810 04:45:36.835405 (+ 241us) server_negotiation.cc:407] Connection header received
0810 04:45:36.835503 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:45:36.835504 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:45:36.835528 (+ 24us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:45:36.835559 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0810 04:45:36.835922 (+ 363us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:41172: BlockingRecv error: recv got EOF from 127.0.0.1:41172 (error 108)
Metrics: {"server-negotiator.queue_time_us":20}
W20260810 04:45:36.869928 28852 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 04:45:36.869267 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41186 (local address 127.22.167.125:35119)
0810 04:45:36.869308 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0810 04:45:36.869310 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 04:45:36.869436 (+ 126us) server_negotiation.cc:407] Connection header received
0810 04:45:36.869498 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:45:36.869499 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:45:36.869510 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:45:36.869559 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0810 04:45:36.869866 (+ 307us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:41186: BlockingRecv error: recv got EOF from 127.0.0.1:41186 (error 108)
Metrics: {"server-negotiator.queue_time_us":20}
W20260810 04:45:36.935668 28852 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 04:45:36.935013 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41202 (local address 127.22.167.125:35119)
0810 04:45:36.935078 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0810 04:45:36.935080 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 04:45:36.935162 (+ 82us) server_negotiation.cc:407] Connection header received
0810 04:45:36.935212 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:45:36.935213 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:45:36.935224 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:45:36.935273 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0810 04:45:36.935606 (+ 333us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:41202: BlockingRecv error: recv got EOF from 127.0.0.1:41202 (error 108)
Metrics: {"server-negotiator.queue_time_us":40}
W20260810 04:45:37.065455 28852 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 04:45:37.064747 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41204 (local address 127.22.167.125:35119)
0810 04:45:37.064819 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0810 04:45:37.064821 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 04:45:37.064907 (+ 86us) server_negotiation.cc:407] Connection header received
0810 04:45:37.064983 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:45:37.064984 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:45:37.064996 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:45:37.065041 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0810 04:45:37.065391 (+ 350us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:41204: BlockingRecv error: recv got EOF from 127.0.0.1:41204 (error 108)
Metrics: {"server-negotiator.queue_time_us":41}
W20260810 04:45:37.323189 28852 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 04:45:37.322538 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41210 (local address 127.22.167.125:35119)
0810 04:45:37.322605 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0810 04:45:37.322607 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 04:45:37.322665 (+ 58us) server_negotiation.cc:407] Connection header received
0810 04:45:37.322734 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:45:37.322735 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:45:37.322747 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:45:37.322797 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0810 04:45:37.323120 (+ 323us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:41210: BlockingRecv error: recv got EOF from 127.0.0.1:41210 (error 108)
Metrics: {"server-negotiator.queue_time_us":42}
I20260810 04:45:37.694437 29008 catalog_manager.cc:1259] Loaded cluster ID: 0f5a1bbe6abe45bebd4468bf2e96e645
I20260810 04:45:37.694512 29008 catalog_manager.cc:1552] T 00000000000000000000000000000000 P b493af56a8424b4cb0595223fe59d998: loading cluster ID for follower catalog manager: success
I20260810 04:45:37.695314 29008 catalog_manager.cc:1574] T 00000000000000000000000000000000 P b493af56a8424b4cb0595223fe59d998: acquiring CA information for follower catalog manager: success
I20260810 04:45:37.695570 29008 catalog_manager.cc:1602] T 00000000000000000000000000000000 P b493af56a8424b4cb0595223fe59d998: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260810 04:45:37.763878 29079 catalog_manager.cc:1259] Loaded cluster ID: 0f5a1bbe6abe45bebd4468bf2e96e645
I20260810 04:45:37.763944 29079 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 314e7978cdec43ee973630752d9a16fa: loading cluster ID for follower catalog manager: success
I20260810 04:45:37.764739 29079 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 314e7978cdec43ee973630752d9a16fa: acquiring CA information for follower catalog manager: success
I20260810 04:45:37.765045 29079 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 314e7978cdec43ee973630752d9a16fa: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260810 04:45:37.838815 29212 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:45:37.836253 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41216 (local address 127.22.167.125:35119)
0810 04:45:37.836371 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0810 04:45:37.836373 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 04:45:37.836416 (+ 43us) server_negotiation.cc:407] Connection header received
0810 04:45:37.836487 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:45:37.836488 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:45:37.836548 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:45:37.836619 (+ 71us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 04:45:37.836761 (+ 142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:45:37.837269 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:45:37.838012 (+ 743us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:45:37.838259 (+ 247us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:45:37.838311 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:45:37.838471 (+ 160us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 04:45:37.838590 (+ 119us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 04:45:37.838659 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0810 04:45:37.838702 (+ 43us) server_negotiation.cc:299] Negotiation successful
0810 04:45:37.838724 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":96,"thread_start_us":56,"threads_started":1}
I20260810 04:45:37.842509 29219 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:45:37.840391 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57744 (local address 127.22.167.124:33959)
0810 04:45:37.840507 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0810 04:45:37.840509 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 04:45:37.840518 (+ 9us) server_negotiation.cc:407] Connection header received
0810 04:45:37.840560 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:45:37.840560 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:45:37.840595 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:45:37.840660 (+ 65us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 04:45:37.840728 (+ 68us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:45:37.841198 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:45:37.841827 (+ 629us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:45:37.841999 (+ 172us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:45:37.842044 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:45:37.842212 (+ 168us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 04:45:37.842313 (+ 101us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 04:45:37.842386 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0810 04:45:37.842401 (+ 15us) server_negotiation.cc:299] Negotiation successful
0810 04:45:37.842422 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":94,"thread_start_us":56,"threads_started":1}
I20260810 04:45:37.846999 29212 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:45:37.844160 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41224 (local address 127.22.167.125:35119)
0810 04:45:37.844202 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0810 04:45:37.844204 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 04:45:37.844232 (+ 28us) server_negotiation.cc:407] Connection header received
0810 04:45:37.844298 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:45:37.844298 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:45:37.844307 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:45:37.844346 (+ 39us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 04:45:37.844510 (+ 164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:45:37.845114 (+ 604us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:45:37.846237 (+ 1123us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:45:37.846517 (+ 280us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:45:37.846576 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:45:37.846712 (+ 136us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 04:45:37.846748 (+ 36us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 04:45:37.846813 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0810 04:45:37.846890 (+ 77us) server_negotiation.cc:299] Negotiation successful
0810 04:45:37.846915 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260810 04:45:37.847113 29219 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:45:37.844018 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57760 (local address 127.22.167.124:33959)
0810 04:45:37.844161 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0810 04:45:37.844163 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 04:45:37.844686 (+ 523us) server_negotiation.cc:407] Connection header received
0810 04:45:37.844712 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:45:37.844713 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:45:37.844724 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:45:37.844789 (+ 65us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 04:45:37.844893 (+ 104us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:45:37.845493 (+ 600us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:45:37.846277 (+ 784us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:45:37.846549 (+ 272us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:45:37.846598 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:45:37.846768 (+ 170us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 04:45:37.846797 (+ 29us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 04:45:37.846850 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0810 04:45:37.847028 (+ 178us) server_negotiation.cc:299] Negotiation successful
0810 04:45:37.847054 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":109}
I20260810 04:45:37.848333 29227 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:45:37.843969 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60746 (local address 127.22.167.126:34487)
0810 04:45:37.844344 (+ 375us) server_negotiation.cc:206] Beginning negotiation
0810 04:45:37.844346 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 04:45:37.844354 (+ 8us) server_negotiation.cc:407] Connection header received
0810 04:45:37.844400 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:45:37.844400 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:45:37.844455 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:45:37.844539 (+ 84us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 04:45:37.845356 (+ 817us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:45:37.846158 (+ 802us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:45:37.847719 (+ 1561us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:45:37.847916 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:45:37.847970 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:45:37.848088 (+ 118us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 04:45:37.848159 (+ 71us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 04:45:37.848204 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0810 04:45:37.848254 (+ 50us) server_negotiation.cc:299] Negotiation successful
0810 04:45:37.848274 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":353,"thread_start_us":66,"threads_started":1}
I20260810 04:45:37.849812 23197 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskemv_AM/build/release/bin/kudu with pid 28590
I20260810 04:45:37.869547 23197 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskemv_AM/build/release/bin/kudu with pid 28719
I20260810 04:45:37.881613 23197 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskemv_AM/build/release/bin/kudu with pid 28859
2026-08-10T04:45:37Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0 (2924 ms)
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1
Loading random data
Initializing database '/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786336863388938-23197-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 04:45:37 dist-test-slave-bqcl krb5kdc[29233](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 04:45:37 dist-test-slave-bqcl krb5kdc[29233](info): set up 2 sockets
Aug 10 04:45:37 dist-test-slave-bqcl krb5kdc[29233](info): commencing operation
krb5kdc: starting...
W20260810 04:45:39.080050 23197 mini_kdc.cc:121] Time spent starting KDC: real 1.170s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786336863388938-23197-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786336863388938-23197-0/krb5kdc/test-user.keytab.
Aug 10 04:45:39 dist-test-slave-bqcl krb5kdc[29233](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786337139, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T04:45:39Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T04:45:39Z Disabled control of system clock
WARNING: no policy specified for kudu/127.22.167.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.167.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.167.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786336863388938-23197-0/krb5kdc/kudu_127.22.167.126.keytab.
Entry for principal kudu/127.22.167.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786336863388938-23197-0/krb5kdc/kudu_127.22.167.126.keytab.
I20260810 04:45:39.198506 23197 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskemv_AM/build/release/bin/kudu
/tmp/dist-test-taskemv_AM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786336863388938-23197-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786336863388938-23197-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786336863388938-23197-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786336863388938-23197-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.167.126:44793
--webserver_interface=127.22.167.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786336863388938-23197-0/krb5kdc/kudu_127.22.167.126.keytab
--principal=kudu/127.22.167.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.167.84:38451
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.167.126:44793,127.22.167.125:37681,127.22.167.124:38931
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786336863388938-23197-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786336863388938-23197-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786336863388938-23197-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 04:45:39.281684 29255 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260810 04:45:39.281896 29255 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 04:45:39.281914 29255 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 04:45:39.281934 29255 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 04:45:39.283129 29255 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 04:45:39.283179 29255 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 04:45:39.283195 29255 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 04:45:39.283208 29255 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 04:45:39.283228 29255 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 04:45:39.284642 29255 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.167.84:38451
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786336863388938-23197-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786336863388938-23197-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.22.167.126:44793,127.22.167.125:37681,127.22.167.124:38931
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786336863388938-23197-0/krb5kdc/kudu_127.22.167.126.keytab
--principal=kudu/127.22.167.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.167.126:44793
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786336863388938-23197-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.167.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29255
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786336863388938-23197-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 03:54:01 UTC on ccycloud.quasar-icijmt.root.comops.site
build id 4577009
I20260810 04:45:39.284840 29255 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 04:45:39.285020 29255 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260810 04:45:39.289672 29255 server_base.cc:1034] running on GCE node
W20260810 04:45:39.289630 29262 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 04:45:39.289644 29265 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 04:45:39.289638 29263 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 04:45:39.290206 29255 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 04:45:39.290468 29255 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 04:45:39.291613 29255 hybrid_clock.cc:648] HybridClock initialized: now 1786337139291600 us; error 26 us; skew 500 ppm
Aug 10 04:45:39 dist-test-slave-bqcl krb5kdc[29233](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786337139, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.167.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 04:45:39.297894 29255 init.cc:377] Logged in from keytab as kudu/127.22.167.126@KRBTEST.COM (short username kudu)
I20260810 04:45:39.298386 29255 webserver.cc:460] Webserver started at http://127.22.167.126:37441/ using document root <none> and password file <none>
I20260810 04:45:39.298591 29255 fs_manager.cc:359] Metadata directory not provided
I20260810 04:45:39.298645 29255 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 04:45:39.298779 29255 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 04:45:39.299849 29255 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786336863388938-23197-0/minicluster-data/master-0/data/instance:
uuid: "46ef2267d42c436eb2dbfc72ec315e86"
format_stamp: "Formatted at 2026-08-10 04:45:39 on dist-test-slave-bqcl"
server_key: "c7741e6fbd4f3ba8ce2e1c41e7271f54"
server_key_iv: "ae7cb37428742083f3958fc4cdaa9c5f"
server_key_version: "encryptionkey@0"
I20260810 04:45:39.300210 29255 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786336863388938-23197-0/minicluster-data/master-0/wal/instance:
uuid: "46ef2267d42c436eb2dbfc72ec315e86"
format_stamp: "Formatted at 2026-08-10 04:45:39 on dist-test-slave-bqcl"
server_key: "c7741e6fbd4f3ba8ce2e1c41e7271f54"
server_key_iv: "ae7cb37428742083f3958fc4cdaa9c5f"
server_key_version: "encryptionkey@0"
I20260810 04:45:39.301524 29255 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260810 04:45:39.302217 29280 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 04:45:39.302377 29255 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260810 04:45:39.302470 29255 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786336863388938-23197-0/minicluster-data/master-0/data,/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786336863388938-23197-0/minicluster-data/master-0/wal
uuid: "46ef2267d42c436eb2dbfc72ec315e86"
format_stamp: "Formatted at 2026-08-10 04:45:39 on dist-test-slave-bqcl"
server_key: "c7741e6fbd4f3ba8ce2e1c41e7271f54"
server_key_iv: "ae7cb37428742083f3958fc4cdaa9c5f"
server_key_version: "encryptionkey@0"
I20260810 04:45:39.302546 29255 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786336863388938-23197-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786336863388938-23197-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786336863388938-23197-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 04:45:39.312482 29255 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 04:45:39.314203 29255 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 04:45:39.314347 29255 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 04:45:39.318374 29255 rpc_server.cc:307] RPC server started. Bound to: 127.22.167.126:44793
I20260810 04:45:39.318405 29383 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.167.126:44793 every 8 connection(s)
I20260810 04:45:39.318728 29255 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786336863388938-23197-0/minicluster-data/master-0/data/info.pb
I20260810 04:45:39.319561 29385 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 04:45:39.320994 29385 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.22.167.126" port: 44793 } has no permanent_uuid. Determining permanent_uuid...
I20260810 04:45:39.323184 23197 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskemv_AM/build/release/bin/kudu as pid 29255
I20260810 04:45:39.323320 23197 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786336863388938-23197-0/minicluster-data/master-0/wal/instance
I20260810 04:45:39.323580 23197 external_mini_cluster.cc:1428] Setting key ed5e344597651182e404366bcd0d357e
Aug 10 04:45:39 dist-test-slave-bqcl krb5kdc[29233](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786337139, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.167.126@KRBTEST.COM for kudu/127.22.167.126@KRBTEST.COM
I20260810 04:45:39.330484 29387 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:45:39.321579 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.126:44793 (local address 127.0.0.1:46790)
0810 04:45:39.321753 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0810 04:45:39.321790 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0810 04:45:39.322240 (+ 450us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 04:45:39.322485 (+ 245us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 04:45:39.322487 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 04:45:39.322713 (+ 226us) client_negotiation.cc:189] Negotiated authn=SASL
0810 04:45:39.322861 (+ 148us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:45:39.322865 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:45:39.324258 (+ 1393us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:45:39.324259 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:45:39.324917 (+ 658us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:45:39.324919 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:45:39.325522 (+ 603us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:45:39.325523 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:45:39.325598 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:45:39.326081 (+ 483us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 04:45:39.326090 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0810 04:45:39.327520 (+ 1430us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 04:45:39.329173 (+ 1653us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 04:45:39.329179 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 04:45:39.329181 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 04:45:39.329361 (+ 180us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 04:45:39.329563 (+ 202us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 04:45:39.329563 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 04:45:39.329564 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 04:45:39.329643 (+ 79us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 04:45:39.329949 (+ 306us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 04:45:39.329954 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 04:45:39.330013 (+ 59us) client_negotiation.cc:769] Sending connection context
0810 04:45:39.330092 (+ 79us) client_negotiation.cc:240] Negotiation successful
0810 04:45:39.330335 (+ 243us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":135,"mutex_wait_us":78,"thread_start_us":70,"threads_started":1}
I20260810 04:45:39.330499 29389 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:45:39.321635 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46790 (local address 127.22.167.126:44793)
0810 04:45:39.321779 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0810 04:45:39.321787 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 04:45:39.321852 (+ 65us) server_negotiation.cc:407] Connection header received
0810 04:45:39.322370 (+ 518us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:45:39.322372 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:45:39.322391 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:45:39.322475 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0810 04:45:39.322956 (+ 481us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:45:39.324157 (+ 1201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:45:39.325004 (+ 847us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:45:39.325434 (+ 430us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:45:39.325525 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:45:39.327608 (+ 2083us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 04:45:39.327618 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 04:45:39.327619 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 04:45:39.327627 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0810 04:45:39.329096 (+ 1469us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 04:45:39.329430 (+ 334us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 04:45:39.329433 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 04:45:39.329434 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 04:45:39.329508 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 04:45:39.329696 (+ 188us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 04:45:39.329697 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 04:45:39.329697 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 04:45:39.329889 (+ 192us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 04:45:39.329958 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0810 04:45:39.330153 (+ 195us) server_negotiation.cc:299] Negotiation successful
0810 04:45:39.330263 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":109,"thread_start_us":71,"threads_started":1}
I20260810 04:45:39.331108 29385 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.22.167.125" port: 37681 } has no permanent_uuid. Determining permanent_uuid...
I20260810 04:45:39.331431 29387 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:45:39.331329 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.125:37681 (local address 127.0.0.1:39536)
0810 04:45:39.331369 (+ 40us) negotiation.cc:107] Waiting for socket to connect
0810 04:45:39.331411 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.167.125:37681: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":26}
W20260810 04:45:39.331544 29284 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.22.167.125:37681: connect: Connection refused (error 111)
WARNING: no policy specified for kudu/127.22.167.125@KRBTEST.COM; defaulting to no policy
I20260810 04:45:39.331908 29387 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:45:39.331824 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.125:37681 (local address 127.0.0.1:39544)
0810 04:45:39.331863 (+ 39us) negotiation.cc:107] Waiting for socket to connect
0810 04:45:39.331886 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.167.125:37681: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
W20260810 04:45:39.332000 29385 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.22.167.125:37681: Network error: Client connection negotiation failed: client connection to 127.22.167.125:37681: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.167.125@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.167.125 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786336863388938-23197-0/krb5kdc/kudu_127.22.167.125.keytab.
Entry for principal kudu/127.22.167.125 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786336863388938-23197-0/krb5kdc/kudu_127.22.167.125.keytab.
I20260810 04:45:39.348419 23197 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskemv_AM/build/release/bin/kudu
/tmp/dist-test-taskemv_AM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786336863388938-23197-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786336863388938-23197-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786336863388938-23197-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786336863388938-23197-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.167.125:37681
--webserver_interface=127.22.167.125
--webserver_port=0
--keytab_file=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786336863388938-23197-0/krb5kdc/kudu_127.22.167.125.keytab
--principal=kudu/127.22.167.125
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.167.84:38451
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.167.126:44793,127.22.167.125:37681,127.22.167.124:38931
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786336863388938-23197-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786336863388938-23197-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786336863388938-23197-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 04:45:39.381229 29385 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.22.167.125" port: 37681 } attempt: 1
I20260810 04:45:39.381764 29387 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:45:39.381630 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.125:37681 (local address 127.0.0.1:39552)
0810 04:45:39.381691 (+ 61us) negotiation.cc:107] Waiting for socket to connect
0810 04:45:39.381732 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.167.125:37681: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
I20260810 04:45:39.382193 29387 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:45:39.382081 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.125:37681 (local address 127.0.0.1:39558)
0810 04:45:39.382143 (+ 62us) negotiation.cc:107] Waiting for socket to connect
0810 04:45:39.382171 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.167.125:37681: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
W20260810 04:45:39.382285 29385 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.22.167.125:37681: Network error: Client connection negotiation failed: client connection to 127.22.167.125:37681: connect: Connection refused (error 111)
W20260810 04:45:39.431612 29396 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260810 04:45:39.431825 29396 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 04:45:39.431852 29396 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 04:45:39.431880 29396 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 04:45:39.432991 29396 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 04:45:39.433037 29396 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 04:45:39.433060 29396 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 04:45:39.433079 29396 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 04:45:39.433099 29396 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 04:45:39.434597 29396 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.167.84:38451
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786336863388938-23197-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786336863388938-23197-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.22.167.126:44793,127.22.167.125:37681,127.22.167.124:38931
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786336863388938-23197-0/krb5kdc/kudu_127.22.167.125.keytab
--principal=kudu/127.22.167.125
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.167.125:37681
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786336863388938-23197-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.22.167.125
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29396
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786336863388938-23197-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 03:54:01 UTC on ccycloud.quasar-icijmt.root.comops.site
build id 4577009
I20260810 04:45:39.434839 29396 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 04:45:39.435050 29396 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 04:45:39.439738 29404 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 04:45:39.439719 29403 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 04:45:39.439709 29407 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 04:45:39.440021 29396 server_base.cc:1034] running on GCE node
I20260810 04:45:39.440274 29396 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 04:45:39.440572 29396 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 04:45:39.441722 29396 hybrid_clock.cc:648] HybridClock initialized: now 1786337139441705 us; error 33 us; skew 500 ppm
Aug 10 04:45:39 dist-test-slave-bqcl krb5kdc[29233](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786337139, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.167.125@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 04:45:39.444097 29396 init.cc:377] Logged in from keytab as kudu/127.22.167.125@KRBTEST.COM (short username kudu)
I20260810 04:45:39.444520 29396 webserver.cc:460] Webserver started at http://127.22.167.125:39167/ using document root <none> and password file <none>
I20260810 04:45:39.444738 29396 fs_manager.cc:359] Metadata directory not provided
I20260810 04:45:39.444785 29396 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 04:45:39.444911 29396 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 04:45:39.445945 29396 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786336863388938-23197-0/minicluster-data/master-1/data/instance:
uuid: "354894974ce2462cbeea05e8f6e93798"
format_stamp: "Formatted at 2026-08-10 04:45:39 on dist-test-slave-bqcl"
server_key: "a60ce8d02c6ea0ac2c8f4948663cccea"
server_key_iv: "270e9aec409d0bff54cd1e3ba5a2b309"
server_key_version: "encryptionkey@0"
I20260810 04:45:39.446301 29396 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786336863388938-23197-0/minicluster-data/master-1/wal/instance:
uuid: "354894974ce2462cbeea05e8f6e93798"
format_stamp: "Formatted at 2026-08-10 04:45:39 on dist-test-slave-bqcl"
server_key: "a60ce8d02c6ea0ac2c8f4948663cccea"
server_key_iv: "270e9aec409d0bff54cd1e3ba5a2b309"
server_key_version: "encryptionkey@0"
I20260810 04:45:39.447505 29396 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260810 04:45:39.448173 29421 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 04:45:39.448326 29396 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260810 04:45:39.448400 29396 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786336863388938-23197-0/minicluster-data/master-1/data,/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786336863388938-23197-0/minicluster-data/master-1/wal
uuid: "354894974ce2462cbeea05e8f6e93798"
format_stamp: "Formatted at 2026-08-10 04:45:39 on dist-test-slave-bqcl"
server_key: "a60ce8d02c6ea0ac2c8f4948663cccea"
server_key_iv: "270e9aec409d0bff54cd1e3ba5a2b309"
server_key_version: "encryptionkey@0"
I20260810 04:45:39.448462 29396 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786336863388938-23197-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786336863388938-23197-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786336863388938-23197-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 04:45:39.450409 29385 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.22.167.125" port: 37681 } attempt: 2
I20260810 04:45:39.450802 29387 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:45:39.450662 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.125:37681 (local address 127.0.0.1:39560)
0810 04:45:39.450721 (+ 59us) negotiation.cc:107] Waiting for socket to connect
0810 04:45:39.450764 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.167.125:37681: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
I20260810 04:45:39.451129 29387 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:45:39.451033 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.125:37681 (local address 127.0.0.1:39564)
0810 04:45:39.451076 (+ 43us) negotiation.cc:107] Waiting for socket to connect
0810 04:45:39.451106 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.167.125:37681: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
W20260810 04:45:39.451216 29385 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.22.167.125:37681: Network error: Client connection negotiation failed: client connection to 127.22.167.125:37681: connect: Connection refused (error 111)
I20260810 04:45:39.471827 29396 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 04:45:39.473934 29396 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 04:45:39.474057 29396 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 04:45:39.477950 29396 rpc_server.cc:307] RPC server started. Bound to: 127.22.167.125:37681
I20260810 04:45:39.477985 29513 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.167.125:37681 every 8 connection(s)
I20260810 04:45:39.478356 29396 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786336863388938-23197-0/minicluster-data/master-1/data/info.pb
I20260810 04:45:39.479221 29514 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 04:45:39.480738 29514 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.22.167.126" port: 44793 } has no permanent_uuid. Determining permanent_uuid...
I20260810 04:45:39.483214 23197 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskemv_AM/build/release/bin/kudu as pid 29396
I20260810 04:45:39.483314 23197 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786336863388938-23197-0/minicluster-data/master-1/wal/instance
I20260810 04:45:39.483479 23197 external_mini_cluster.cc:1428] Setting key 8c26c2fa06448a8606a563624c16e6c0
Aug 10 04:45:39 dist-test-slave-bqcl krb5kdc[29233](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786337139, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.167.125@KRBTEST.COM for kudu/127.22.167.126@KRBTEST.COM
I20260810 04:45:39.489661 29389 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:45:39.481401 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46792 (local address 127.22.167.126:44793)
0810 04:45:39.481503 (+ 102us) server_negotiation.cc:206] Beginning negotiation
0810 04:45:39.481506 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 04:45:39.481681 (+ 175us) server_negotiation.cc:407] Connection header received
0810 04:45:39.482372 (+ 691us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:45:39.482374 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:45:39.482391 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:45:39.482446 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0810 04:45:39.483105 (+ 659us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:45:39.483690 (+ 585us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:45:39.484682 (+ 992us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:45:39.484897 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:45:39.484956 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:45:39.487064 (+ 2108us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 04:45:39.487065 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 04:45:39.487066 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 04:45:39.487074 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0810 04:45:39.488385 (+ 1311us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 04:45:39.488763 (+ 378us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 04:45:39.488764 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 04:45:39.488764 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 04:45:39.488819 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 04:45:39.489040 (+ 221us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 04:45:39.489041 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 04:45:39.489042 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 04:45:39.489163 (+ 121us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 04:45:39.489220 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0810 04:45:39.489482 (+ 262us) server_negotiation.cc:299] Negotiation successful
0810 04:45:39.489567 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260810 04:45:39.489847 29516 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:45:39.481343 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.126:44793 (local address 127.0.0.1:46792)
0810 04:45:39.481556 (+ 213us) negotiation.cc:107] Waiting for socket to connect
0810 04:45:39.481601 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0810 04:45:39.482253 (+ 652us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 04:45:39.482527 (+ 274us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 04:45:39.482531 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 04:45:39.482796 (+ 265us) client_negotiation.cc:189] Negotiated authn=SASL
0810 04:45:39.482990 (+ 194us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:45:39.482995 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:45:39.483782 (+ 787us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:45:39.483783 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:45:39.484597 (+ 814us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:45:39.484599 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:45:39.484967 (+ 368us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:45:39.484968 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:45:39.485103 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:45:39.485593 (+ 490us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 04:45:39.485602 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0810 04:45:39.486970 (+ 1368us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 04:45:39.488460 (+ 1490us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 04:45:39.488467 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 04:45:39.488470 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 04:45:39.488696 (+ 226us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 04:45:39.488875 (+ 179us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 04:45:39.488875 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 04:45:39.488876 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 04:45:39.488986 (+ 110us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 04:45:39.489220 (+ 234us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 04:45:39.489227 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 04:45:39.489366 (+ 139us) client_negotiation.cc:769] Sending connection context
0810 04:45:39.489454 (+ 88us) client_negotiation.cc:240] Negotiation successful
0810 04:45:39.489610 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":166,"thread_start_us":84,"threads_started":1}
I20260810 04:45:39.490396 29514 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.22.167.125" port: 37681 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.22.167.124@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.167.124@KRBTEST.COM" created.
Aug 10 04:45:39 dist-test-slave-bqcl krb5kdc[29233](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786337139, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.167.125@KRBTEST.COM for kudu/127.22.167.125@KRBTEST.COM
I20260810 04:45:39.497928 29516 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:45:39.490629 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.125:37681 (local address 127.0.0.1:39572)
0810 04:45:39.490692 (+ 63us) negotiation.cc:107] Waiting for socket to connect
0810 04:45:39.490702 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0810 04:45:39.490738 (+ 36us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 04:45:39.490990 (+ 252us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 04:45:39.490991 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 04:45:39.491119 (+ 128us) client_negotiation.cc:189] Negotiated authn=SASL
0810 04:45:39.491173 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:45:39.491175 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:45:39.491980 (+ 805us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:45:39.491981 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:45:39.492474 (+ 493us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:45:39.492475 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:45:39.492992 (+ 517us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:45:39.492993 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:45:39.493060 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:45:39.493406 (+ 346us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 04:45:39.493410 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0810 04:45:39.495234 (+ 1824us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 04:45:39.496890 (+ 1656us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 04:45:39.496891 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 04:45:39.496891 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 04:45:39.497042 (+ 151us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 04:45:39.497260 (+ 218us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 04:45:39.497261 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 04:45:39.497261 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 04:45:39.497326 (+ 65us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 04:45:39.497626 (+ 300us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 04:45:39.497626 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 04:45:39.497666 (+ 40us) client_negotiation.cc:769] Sending connection context
0810 04:45:39.497727 (+ 61us) client_negotiation.cc:240] Negotiation successful
0810 04:45:39.497824 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":47}
I20260810 04:45:39.498013 29523 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:45:39.490717 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39572 (local address 127.22.167.125:37681)
0810 04:45:39.490877 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0810 04:45:39.490881 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 04:45:39.490895 (+ 14us) server_negotiation.cc:407] Connection header received
0810 04:45:39.490909 (+ 14us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:45:39.490910 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:45:39.490920 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:45:39.490977 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0810 04:45:39.491234 (+ 257us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:45:39.491906 (+ 672us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:45:39.492540 (+ 634us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:45:39.492918 (+ 378us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:45:39.492988 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:45:39.495311 (+ 2323us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 04:45:39.495319 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 04:45:39.495319 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 04:45:39.495325 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0810 04:45:39.496801 (+ 1476us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 04:45:39.497108 (+ 307us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 04:45:39.497111 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 04:45:39.497111 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 04:45:39.497177 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 04:45:39.497390 (+ 213us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 04:45:39.497391 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 04:45:39.497391 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 04:45:39.497562 (+ 171us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 04:45:39.497639 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0810 04:45:39.497756 (+ 117us) server_negotiation.cc:299] Negotiation successful
0810 04:45:39.497878 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":56,"server-negotiator.queue_time_us":139,"thread_start_us":101,"threads_started":1}
I20260810 04:45:39.498557 29514 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.22.167.124" port: 38931 } has no permanent_uuid. Determining permanent_uuid...
I20260810 04:45:39.498885 29516 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:45:39.498779 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.124:38931 (local address 127.0.0.1:38370)
0810 04:45:39.498819 (+ 40us) negotiation.cc:107] Waiting for socket to connect
0810 04:45:39.498863 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.167.124:38931: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":24}
W20260810 04:45:39.499020 29427 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.22.167.124:38931: connect: Connection refused (error 111)
I20260810 04:45:39.499408 29516 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:45:39.499346 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.124:38931 (local address 127.0.0.1:38382)
0810 04:45:39.499369 (+ 23us) negotiation.cc:107] Waiting for socket to connect
0810 04:45:39.499392 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.167.124:38931: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":14}
W20260810 04:45:39.499509 29514 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.22.167.124:38931: Network error: Client connection negotiation failed: client connection to 127.22.167.124:38931: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.167.124 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786336863388938-23197-0/krb5kdc/kudu_127.22.167.124.keytab.
Entry for principal kudu/127.22.167.124 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786336863388938-23197-0/krb5kdc/kudu_127.22.167.124.keytab.
I20260810 04:45:39.508895 23197 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskemv_AM/build/release/bin/kudu
/tmp/dist-test-taskemv_AM/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786336863388938-23197-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786336863388938-23197-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786336863388938-23197-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786336863388938-23197-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.167.124:38931
--webserver_interface=127.22.167.124
--webserver_port=0
--keytab_file=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786336863388938-23197-0/krb5kdc/kudu_127.22.167.124.keytab
--principal=kudu/127.22.167.124
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.167.84:38451
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.167.126:44793,127.22.167.125:37681,127.22.167.124:38931
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786336863388938-23197-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786336863388938-23197-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786336863388938-23197-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 04:45:39.542348 29385 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.22.167.125" port: 37681 } attempt: 3
Aug 10 04:45:39 dist-test-slave-bqcl krb5kdc[29233](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786337139, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.167.126@KRBTEST.COM for kudu/127.22.167.125@KRBTEST.COM
I20260810 04:45:39.548745 29514 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.22.167.124" port: 38931 } attempt: 1
I20260810 04:45:39.549155 29516 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:45:39.549026 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.124:38931 (local address 127.0.0.1:38384)
0810 04:45:39.549076 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0810 04:45:39.549103 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.167.124:38931: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
I20260810 04:45:39.549504 29516 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:45:39.549402 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.124:38931 (local address 127.0.0.1:38400)
0810 04:45:39.549450 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0810 04:45:39.549478 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.167.124:38931: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260810 04:45:39.549600 29514 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.22.167.124:38931: Network error: Client connection negotiation failed: client connection to 127.22.167.124:38931: connect: Connection refused (error 111)
I20260810 04:45:39.549769 29387 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:45:39.542742 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.125:37681 (local address 127.0.0.1:39582)
0810 04:45:39.542806 (+ 64us) negotiation.cc:107] Waiting for socket to connect
0810 04:45:39.542819 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0810 04:45:39.542876 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 04:45:39.543007 (+ 131us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 04:45:39.543008 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 04:45:39.543163 (+ 155us) client_negotiation.cc:189] Negotiated authn=SASL
0810 04:45:39.543237 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:45:39.543239 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:45:39.544167 (+ 928us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:45:39.544167 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:45:39.544692 (+ 525us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:45:39.544693 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:45:39.545188 (+ 495us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:45:39.545189 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:45:39.545247 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:45:39.545731 (+ 484us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 04:45:39.545735 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0810 04:45:39.547111 (+ 1376us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 04:45:39.548732 (+ 1621us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 04:45:39.548733 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 04:45:39.548733 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 04:45:39.548932 (+ 199us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 04:45:39.549137 (+ 205us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 04:45:39.549137 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 04:45:39.549138 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 04:45:39.549207 (+ 69us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 04:45:39.549492 (+ 285us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 04:45:39.549493 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 04:45:39.549542 (+ 49us) client_negotiation.cc:769] Sending connection context
0810 04:45:39.549606 (+ 64us) client_negotiation.cc:240] Negotiation successful
0810 04:45:39.549707 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":38}
I20260810 04:45:39.550005 29523 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:45:39.542802 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39582 (local address 127.22.167.125:37681)
0810 04:45:39.542861 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0810 04:45:39.542863 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 04:45:39.542876 (+ 13us) server_negotiation.cc:407] Connection header received
0810 04:45:39.542928 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:45:39.542929 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:45:39.542943 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:45:39.543002 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0810 04:45:39.543311 (+ 309us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:45:39.544091 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:45:39.544780 (+ 689us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:45:39.545119 (+ 339us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:45:39.545196 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:45:39.547207 (+ 2011us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 04:45:39.547209 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 04:45:39.547209 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 04:45:39.547217 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0810 04:45:39.548632 (+ 1415us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 04:45:39.548995 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 04:45:39.548996 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 04:45:39.548997 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 04:45:39.549069 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 04:45:39.549286 (+ 217us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 04:45:39.549287 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 04:45:39.549287 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 04:45:39.549406 (+ 119us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 04:45:39.549463 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0810 04:45:39.549874 (+ 411us) server_negotiation.cc:299] Negotiation successful
0810 04:45:39.549953 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260810 04:45:39.550323 29385 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.22.167.124" port: 38931 } has no permanent_uuid. Determining permanent_uuid...
I20260810 04:45:39.550625 29387 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:45:39.550540 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.124:38931 (local address 127.0.0.1:38404)
0810 04:45:39.550581 (+ 41us) negotiation.cc:107] Waiting for socket to connect
0810 04:45:39.550606 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.167.124:38931: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
I20260810 04:45:39.550935 29387 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:45:39.550861 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.124:38931 (local address 127.0.0.1:38408)
0810 04:45:39.550893 (+ 32us) negotiation.cc:107] Waiting for socket to connect
0810 04:45:39.550915 (+ 22us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.167.124:38931: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":22}
W20260810 04:45:39.551011 29385 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.22.167.124:38931: Network error: Client connection negotiation failed: client connection to 127.22.167.124:38931: connect: Connection refused (error 111)
I20260810 04:45:39.582167 29385 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.22.167.124" port: 38931 } attempt: 1
I20260810 04:45:39.582572 29387 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:45:39.582460 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.124:38931 (local address 127.0.0.1:38412)
0810 04:45:39.582512 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0810 04:45:39.582542 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.167.124:38931: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
I20260810 04:45:39.582935 29387 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:45:39.582849 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.124:38931 (local address 127.0.0.1:38426)
0810 04:45:39.582887 (+ 38us) negotiation.cc:107] Waiting for socket to connect
0810 04:45:39.582912 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.167.124:38931: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":24}
W20260810 04:45:39.583017 29385 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.22.167.124:38931: Network error: Client connection negotiation failed: client connection to 127.22.167.124:38931: connect: Connection refused (error 111)
W20260810 04:45:39.591398 29527 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260810 04:45:39.591598 29527 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 04:45:39.591622 29527 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 04:45:39.591655 29527 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 04:45:39.592756 29527 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 04:45:39.592793 29527 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 04:45:39.592813 29527 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 04:45:39.592828 29527 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 04:45:39.592844 29527 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 04:45:39.594269 29527 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.167.84:38451
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786336863388938-23197-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786336863388938-23197-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.22.167.126:44793,127.22.167.125:37681,127.22.167.124:38931
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786336863388938-23197-0/krb5kdc/kudu_127.22.167.124.keytab
--principal=kudu/127.22.167.124
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.167.124:38931
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786336863388938-23197-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.22.167.124
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29527
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786336863388938-23197-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 03:54:01 UTC on ccycloud.quasar-icijmt.root.comops.site
build id 4577009
I20260810 04:45:39.594506 29527 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 04:45:39.594708 29527 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 04:45:39.599144 29536 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 04:45:39.599174 29542 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 04:45:39.599242 29527 server_base.cc:1034] running on GCE node
W20260810 04:45:39.599205 29538 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 04:45:39.599565 29527 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 04:45:39.599833 29527 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 04:45:39.600962 29527 hybrid_clock.cc:648] HybridClock initialized: now 1786337139600955 us; error 26 us; skew 500 ppm
Aug 10 04:45:39 dist-test-slave-bqcl krb5kdc[29233](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786337139, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.167.124@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 04:45:39.603452 29527 init.cc:377] Logged in from keytab as kudu/127.22.167.124@KRBTEST.COM (short username kudu)
I20260810 04:45:39.603845 29527 webserver.cc:460] Webserver started at http://127.22.167.124:39861/ using document root <none> and password file <none>
I20260810 04:45:39.604036 29527 fs_manager.cc:359] Metadata directory not provided
I20260810 04:45:39.604084 29527 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 04:45:39.604206 29527 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 04:45:39.605222 29527 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786336863388938-23197-0/minicluster-data/master-2/data/instance:
uuid: "01dd5541dbad45308cb81844e1567391"
format_stamp: "Formatted at 2026-08-10 04:45:39 on dist-test-slave-bqcl"
server_key: "cdf2b9e19f6ab16b4e8b49a46fa1ee55"
server_key_iv: "9383d4c879666ad8d00b1beef115b09c"
server_key_version: "encryptionkey@0"
I20260810 04:45:39.605549 29527 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786336863388938-23197-0/minicluster-data/master-2/wal/instance:
uuid: "01dd5541dbad45308cb81844e1567391"
format_stamp: "Formatted at 2026-08-10 04:45:39 on dist-test-slave-bqcl"
server_key: "cdf2b9e19f6ab16b4e8b49a46fa1ee55"
server_key_iv: "9383d4c879666ad8d00b1beef115b09c"
server_key_version: "encryptionkey@0"
I20260810 04:45:39.606894 29527 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260810 04:45:39.607649 29554 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 04:45:39.607805 29527 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260810 04:45:39.607895 29527 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786336863388938-23197-0/minicluster-data/master-2/data,/tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786336863388938-23197-0/minicluster-data/master-2/wal
uuid: "01dd5541dbad45308cb81844e1567391"
format_stamp: "Formatted at 2026-08-10 04:45:39 on dist-test-slave-bqcl"
server_key: "cdf2b9e19f6ab16b4e8b49a46fa1ee55"
server_key_iv: "9383d4c879666ad8d00b1beef115b09c"
server_key_version: "encryptionkey@0"
I20260810 04:45:39.607959 29527 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786336863388938-23197-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786336863388938-23197-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786336863388938-23197-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 04:45:39.617726 29514 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.22.167.124" port: 38931 } attempt: 2
I20260810 04:45:39.618114 29516 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:45:39.617994 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.124:38931 (local address 127.0.0.1:38434)
0810 04:45:39.618048 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0810 04:45:39.618086 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.167.124:38931: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
I20260810 04:45:39.618425 29516 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:45:39.618354 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.124:38931 (local address 127.0.0.1:38446)
0810 04:45:39.618386 (+ 32us) negotiation.cc:107] Waiting for socket to connect
0810 04:45:39.618409 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.167.124:38931: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":22}
W20260810 04:45:39.618505 29514 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.22.167.124:38931: Network error: Client connection negotiation failed: client connection to 127.22.167.124:38931: connect: Connection refused (error 111)
I20260810 04:45:39.633581 29527 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 04:45:39.635763 29527 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 04:45:39.635878 29527 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 04:45:39.640014 29527 rpc_server.cc:307] RPC server started. Bound to: 127.22.167.124:38931
I20260810 04:45:39.640040 29642 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.167.124:38931 every 8 connection(s)
I20260810 04:45:39.640370 29527 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786336863388938-23197-0/minicluster-data/master-2/data/info.pb
I20260810 04:45:39.641227 29643 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 04:45:39.642730 29643 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.22.167.126" port: 44793 } has no permanent_uuid. Determining permanent_uuid...
I20260810 04:45:39.643622 23197 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskemv_AM/build/release/bin/kudu as pid 29527
I20260810 04:45:39.643719 23197 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskemv_AM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786336863388938-23197-0/minicluster-data/master-2/wal/instance
I20260810 04:45:39.643890 23197 external_mini_cluster.cc:1428] Setting key e7d893cbb5409b4164a1638e458bc47f
Aug 10 04:45:39 dist-test-slave-bqcl krb5kdc[29233](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786337139, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.167.124@KRBTEST.COM for kudu/127.22.167.126@KRBTEST.COM
Aug 10 04:45:39 dist-test-slave-bqcl krb5kdc[29233](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786337139, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.167.124@KRBTEST.COM
I20260810 04:45:39.651577 29389 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:45:39.643332 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46796 (local address 127.22.167.126:44793)
0810 04:45:39.643372 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0810 04:45:39.643374 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 04:45:39.643560 (+ 186us) server_negotiation.cc:407] Connection header received
0810 04:45:39.644120 (+ 560us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:45:39.644122 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:45:39.644135 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:45:39.644202 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0810 04:45:39.644875 (+ 673us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:45:39.645416 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:45:39.646335 (+ 919us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:45:39.646521 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:45:39.646577 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:45:39.648741 (+ 2164us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 04:45:39.648743 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 04:45:39.648743 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 04:45:39.648751 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0810 04:45:39.650207 (+ 1456us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 04:45:39.650651 (+ 444us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 04:45:39.650652 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 04:45:39.650653 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 04:45:39.650716 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 04:45:39.650927 (+ 211us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 04:45:39.650928 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 04:45:39.650929 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 04:45:39.651065 (+ 136us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 04:45:39.651139 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0810 04:45:39.651389 (+ 250us) server_negotiation.cc:299] Negotiation successful
0810 04:45:39.651463 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260810 04:45:39.651738 29645 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:45:39.643301 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.126:44793 (local address 127.0.0.1:46796)
0810 04:45:39.643449 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0810 04:45:39.643483 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0810 04:45:39.644007 (+ 524us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 04:45:39.644277 (+ 270us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 04:45:39.644281 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 04:45:39.644533 (+ 252us) client_negotiation.cc:189] Negotiated authn=SASL
0810 04:45:39.644786 (+ 253us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:45:39.644793 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:45:39.645494 (+ 701us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:45:39.645495 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:45:39.646254 (+ 759us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:45:39.646256 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:45:39.646589 (+ 333us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:45:39.646589 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:45:39.646725 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:45:39.647184 (+ 459us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 04:45:39.647191 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0810 04:45:39.648655 (+ 1464us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 04:45:39.650279 (+ 1624us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 04:45:39.650286 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 04:45:39.650287 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 04:45:39.650583 (+ 296us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 04:45:39.650783 (+ 200us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 04:45:39.650783 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 04:45:39.650783 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 04:45:39.650876 (+ 93us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 04:45:39.651124 (+ 248us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 04:45:39.651127 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 04:45:39.651239 (+ 112us) client_negotiation.cc:769] Sending connection context
0810 04:45:39.651323 (+ 84us) client_negotiation.cc:240] Negotiation successful
0810 04:45:39.651470 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":63,"threads_started":1}
I20260810 04:45:39.652263 29643 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.22.167.125" port: 37681 } has no permanent_uuid. Determining permanent_uuid...
I20260810 04:45:39.652410 29649 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:45:39.644235 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38454 (local address 127.22.167.124:38931)
0810 04:45:39.644496 (+ 261us) server_negotiation.cc:206] Beginning negotiation
0810 04:45:39.644499 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 04:45:39.644514 (+ 15us) server_negotiation.cc:407] Connection header received
0810 04:45:39.644531 (+ 17us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:45:39.644532 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:45:39.644556 (+ 24us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:45:39.644619 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0810 04:45:39.645270 (+ 651us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:45:39.646114 (+ 844us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:45:39.646977 (+ 863us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:45:39.647351 (+ 374us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:45:39.647400 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:45:39.649620 (+ 2220us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 04:45:39.649629 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 04:45:39.649629 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 04:45:39.649636 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 04:45:39.651235 (+ 1599us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 04:45:39.651568 (+ 333us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 04:45:39.651571 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 04:45:39.651571 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 04:45:39.651635 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 04:45:39.651899 (+ 264us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 04:45:39.651901 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 04:45:39.651901 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 04:45:39.652049 (+ 148us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 04:45:39.652181 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0810 04:45:39.652256 (+ 75us) server_negotiation.cc:299] Negotiation successful
0810 04:45:39.652319 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":237,"thread_start_us":72,"threads_started":1}
Aug 10 04:45:39 dist-test-slave-bqcl krb5kdc[29233](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786337139, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.167.124@KRBTEST.COM for kudu/127.22.167.125@KRBTEST.COM
Aug 10 04:45:39 dist-test-slave-bqcl krb5kdc[29233](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786337139, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.167.126@KRBTEST.COM
I20260810 04:45:39.658149 29385 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.22.167.124" port: 38931 } attempt: 2
I20260810 04:45:39.658542 29523 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:45:39.652728 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39598 (local address 127.22.167.125:37681)
0810 04:45:39.652776 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0810 04:45:39.652778 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 04:45:39.652786 (+ 8us) server_negotiation.cc:407] Connection header received
0810 04:45:39.652809 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:45:39.652810 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:45:39.652818 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:45:39.652859 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0810 04:45:39.653135 (+ 276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:45:39.653589 (+ 454us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:45:39.654224 (+ 635us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:45:39.654456 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:45:39.654510 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:45:39.656158 (+ 1648us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 04:45:39.656159 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 04:45:39.656160 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 04:45:39.656166 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0810 04:45:39.657475 (+ 1309us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 04:45:39.657810 (+ 335us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 04:45:39.657811 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 04:45:39.657812 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 04:45:39.657864 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 04:45:39.658037 (+ 173us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 04:45:39.658038 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 04:45:39.658039 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 04:45:39.658194 (+ 155us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 04:45:39.658256 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0810 04:45:39.658395 (+ 139us) server_negotiation.cc:299] Negotiation successful
0810 04:45:39.658458 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260810 04:45:39.658540 29645 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:45:39.652484 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.125:37681 (local address 127.0.0.1:39598)
0810 04:45:39.652514 (+ 30us) negotiation.cc:107] Waiting for socket to connect
0810 04:45:39.652525 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0810 04:45:39.652563 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 04:45:39.652885 (+ 322us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 04:45:39.652886 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 04:45:39.653022 (+ 136us) client_negotiation.cc:189] Negotiated authn=SASL
0810 04:45:39.653075 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:45:39.653077 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:45:39.653660 (+ 583us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:45:39.653661 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:45:39.654116 (+ 455us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:45:39.654117 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:45:39.654514 (+ 397us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:45:39.654516 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:45:39.654580 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:45:39.654911 (+ 331us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 04:45:39.654916 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0810 04:45:39.656078 (+ 1162us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 04:45:39.657547 (+ 1469us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 04:45:39.657548 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 04:45:39.657548 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 04:45:39.657745 (+ 197us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 04:45:39.657925 (+ 180us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 04:45:39.657926 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 04:45:39.657926 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 04:45:39.657978 (+ 52us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 04:45:39.658259 (+ 281us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 04:45:39.658260 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 04:45:39.658304 (+ 44us) client_negotiation.cc:769] Sending connection context
0810 04:45:39.658368 (+ 64us) client_negotiation.cc:240] Negotiation successful
0810 04:45:39.658451 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":17}
I20260810 04:45:39.658885 29643 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.22.167.124" port: 38931 } has no permanent_uuid. Determining permanent_uuid...
I20260810 04:45:39.659811 29389 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:45:39.653104 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46800 (local address 127.22.167.126:44793)
0810 04:45:39.653137 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0810 04:45:39.653139 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 04:45:39.653185 (+ 46us) server_negotiation.cc:407] Connection header received
0810 04:45:39.653250 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:45:39.653251 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:45:39.653259 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:45:39.653303 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0810 04:45:39.653703 (+ 400us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:45:39.654155 (+ 452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:45:39.654786 (+ 631us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:45:39.655049 (+ 263us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:45:39.655114 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:45:39.657208 (+ 2094us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 04:45:39.657209 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 04:45:39.657209 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 04:45:39.657215 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0810 04:45:39.658722 (+ 1507us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 04:45:39.659055 (+ 333us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 04:45:39.659056 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 04:45:39.659057 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 04:45:39.659121 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 04:45:39.659317 (+ 196us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 04:45:39.659318 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 04:45:39.659319 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 04:45:39.659440 (+ 121us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 04:45:39.659617 (+ 177us) server_negotiation.cc:1035] Waiting for connection context
0810 04:45:39.659656 (+ 39us) server_negotiation.cc:299] Negotiation successful
0810 04:45:39.659726 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15}
Aug 10 04:45:39 dist-test-slave-bqcl krb5kdc[29233](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786337139, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.167.126@KRBTEST.COM for kudu/127.22.167.124@KRBTEST.COM
Aug 10 04:45:39 dist-test-slave-bqcl krb5kdc[29233](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786337139, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.167.124@KRBTEST.COM for kudu/127.22.167.124@KRBTEST.COM
Aug 10 04:45:39 dist-test-slave-bqcl krb5kdc[29233](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786337139, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.167.125@KRBTEST.COM
I20260810 04:45:39.665398 29387 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:45:39.658387 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.124:38931 (local address 127.0.0.1:38466)
0810 04:45:39.658464 (+ 77us) negotiation.cc:107] Waiting for socket to connect
0810 04:45:39.658474 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0810 04:45:39.658509 (+ 35us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 04:45:39.658969 (+ 460us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 04:45:39.658970 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 04:45:39.659102 (+ 132us) client_negotiation.cc:189] Negotiated authn=SASL
0810 04:45:39.659141 (+ 39us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:45:39.659143 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:45:39.659991 (+ 848us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:45:39.659992 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:45:39.660696 (+ 704us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:45:39.660699 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:45:39.661165 (+ 466us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:45:39.661166 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:45:39.661221 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:45:39.661544 (+ 323us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 04:45:39.661548 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0810 04:45:39.662686 (+ 1138us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 04:45:39.664439 (+ 1753us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 04:45:39.664439 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 04:45:39.664440 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 04:45:39.664634 (+ 194us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 04:45:39.664842 (+ 208us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 04:45:39.664842 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 04:45:39.664843 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 04:45:39.664899 (+ 56us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 04:45:39.665171 (+ 272us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 04:45:39.665172 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 04:45:39.665213 (+ 41us) client_negotiation.cc:769] Sending connection context
0810 04:45:39.665269 (+ 56us) client_negotiation.cc:240] Negotiation successful
0810 04:45:39.665342 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":64}
I20260810 04:45:39.665679 29649 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:45:39.658650 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38466 (local address 127.22.167.124:38931)
0810 04:45:39.658762 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0810 04:45:39.658764 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 04:45:39.658771 (+ 7us) server_negotiation.cc:407] Connection header received
0810 04:45:39.658788 (+ 17us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:45:39.658788 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:45:39.658862 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:45:39.658940 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0810 04:45:39.659263 (+ 323us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:45:39.659908 (+ 645us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:45:39.660789 (+ 881us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:45:39.661096 (+ 307us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:45:39.661155 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:45:39.662782 (+ 1627us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 04:45:39.662783 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 04:45:39.662784 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 04:45:39.662790 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0810 04:45:39.664381 (+ 1591us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 04:45:39.664714 (+ 333us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 04:45:39.664716 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 04:45:39.664716 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 04:45:39.664767 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 04:45:39.664968 (+ 201us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 04:45:39.664969 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 04:45:39.664969 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 04:45:39.665096 (+ 127us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 04:45:39.665493 (+ 397us) server_negotiation.cc:1035] Waiting for connection context
0810 04:45:39.665532 (+ 39us) server_negotiation.cc:299] Negotiation successful
0810 04:45:39.665599 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":96}
I20260810 04:45:39.666446 29645 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:45:39.659080 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.124:38931 (local address 127.0.0.1:38478)
0810 04:45:39.659116 (+ 36us) negotiation.cc:107] Waiting for socket to connect
0810 04:45:39.659127 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0810 04:45:39.659174 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 04:45:39.659369 (+ 195us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 04:45:39.659370 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 04:45:39.659501 (+ 131us) client_negotiation.cc:189] Negotiated authn=SASL
0810 04:45:39.659531 (+ 30us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:45:39.659533 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:45:39.660351 (+ 818us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:45:39.660352 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:45:39.660862 (+ 510us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:45:39.660864 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:45:39.661212 (+ 348us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:45:39.661213 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:45:39.661259 (+ 46us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:45:39.661714 (+ 455us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 04:45:39.661719 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0810 04:45:39.663178 (+ 1459us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 04:45:39.665240 (+ 2062us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 04:45:39.665241 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 04:45:39.665242 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 04:45:39.665407 (+ 165us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 04:45:39.665611 (+ 204us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 04:45:39.665612 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 04:45:39.665612 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 04:45:39.665668 (+ 56us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 04:45:39.666172 (+ 504us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 04:45:39.666173 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 04:45:39.666210 (+ 37us) client_negotiation.cc:769] Sending connection context
0810 04:45:39.666263 (+ 53us) client_negotiation.cc:240] Negotiation successful
0810 04:45:39.666352 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":20}
I20260810 04:45:39.666541 29651 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:45:39.659145 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38478 (local address 127.22.167.124:38931)
0810 04:45:39.659275 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0810 04:45:39.659276 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0810 04:45:39.659282 (+ 6us) server_negotiation.cc:407] Connection header received
0810 04:45:39.659297 (+ 15us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:45:39.659297 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:45:39.659305 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:45:39.659362 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0810 04:45:39.659614 (+ 252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:45:39.660278 (+ 664us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:45:39.660927 (+ 649us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:45:39.661132 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:45:39.661209 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:45:39.663257 (+ 2048us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 04:45:39.663258 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 04:45:39.663258 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 04:45:39.663264 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0810 04:45:39.665172 (+ 1908us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 04:45:39.665476 (+ 304us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 04:45:39.665478 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 04:45:39.665478 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 04:45:39.665533 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 04:45:39.665724 (+ 191us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 04:45:39.665725 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 04:45:39.665725 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 04:45:39.665977 (+ 252us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 04:45:39.666037 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0810 04:45:39.666306 (+ 269us) server_negotiation.cc:299] Negotiation successful
0810 04:45:39.666400 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":30,"server-negotiator.queue_time_us":113,"thread_start_us":83,"threads_started":1}
I20260810 04:45:39.667366 29385 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 46ef2267d42c436eb2dbfc72ec315e86: Bootstrap starting.
I20260810 04:45:39.667531 29643 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 01dd5541dbad45308cb81844e1567391: Bootstrap starting.
I20260810 04:45:39.668069 29385 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 46ef2267d42c436eb2dbfc72ec315e86: Neither blocks nor log segments found. Creating new log.
I20260810 04:45:39.668136 29643 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 01dd5541dbad45308cb81844e1567391: Neither blocks nor log segments found. Creating new log.
I20260810 04:45:39.668330 29385 log.cc:824] T 00000000000000000000000000000000 P 46ef2267d42c436eb2dbfc72ec315e86: Log is configured to *not* fsync() on all Append() calls
I20260810 04:45:39.668378 29643 log.cc:824] T 00000000000000000000000000000000 P 01dd5541dbad45308cb81844e1567391: Log is configured to *not* fsync() on all Append() calls
I20260810 04:45:39.669018 29643 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 01dd5541dbad45308cb81844e1567391: No bootstrap required, opened a new log
I20260810 04:45:39.669517 29385 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 46ef2267d42c436eb2dbfc72ec315e86: No bootstrap required, opened a new log
I20260810 04:45:39.670285 29523 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:45:39.660400 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39608 (local address 127.22.167.125:37681)
0810 04:45:39.660435 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0810 04:45:39.660437 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 04:45:39.660444 (+ 7us) server_negotiation.cc:407] Connection header received
0810 04:45:39.660476 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:45:39.660477 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:45:39.660485 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:45:39.660528 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0810 04:45:39.661038 (+ 510us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:45:39.661586 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:45:39.662409 (+ 823us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:45:39.662640 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:45:39.662701 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:45:39.664883 (+ 2182us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 04:45:39.664884 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 04:45:39.664885 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 04:45:39.664891 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0810 04:45:39.669113 (+ 4222us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 04:45:39.669452 (+ 339us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 04:45:39.669454 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 04:45:39.669454 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 04:45:39.669515 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 04:45:39.669768 (+ 253us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 04:45:39.669769 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 04:45:39.669769 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 04:45:39.669892 (+ 123us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 04:45:39.670070 (+ 178us) server_negotiation.cc:1035] Waiting for connection context
0810 04:45:39.670109 (+ 39us) server_negotiation.cc:299] Negotiation successful
0810 04:45:39.670204 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260810 04:45:39.670919 29643 raft_consensus.cc:348] T 00000000000000000000000000000000 P 01dd5541dbad45308cb81844e1567391 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46ef2267d42c436eb2dbfc72ec315e86" member_type: VOTER last_known_addr { host: "127.22.167.126" port: 44793 } } peers { permanent_uuid: "354894974ce2462cbeea05e8f6e93798" member_type: VOTER last_known_addr { host: "127.22.167.125" port: 37681 } } peers { permanent_uuid: "01dd5541dbad45308cb81844e1567391" member_type: VOTER last_known_addr { host: "127.22.167.124" port: 38931 } }
I20260810 04:45:39.671074 29643 raft_consensus.cc:729] T 00000000000000000000000000000000 P 01dd5541dbad45308cb81844e1567391 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 01dd5541dbad45308cb81844e1567391, State: Initialized, Role: FOLLOWER
I20260810 04:45:39.671165 29643 consensus_queue.cc:260] T 00000000000000000000000000000000 P 01dd5541dbad45308cb81844e1567391 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46ef2267d42c436eb2dbfc72ec315e86" member_type: VOTER last_known_addr { host: "127.22.167.126" port: 44793 } } peers { permanent_uuid: "354894974ce2462cbeea05e8f6e93798" member_type: VOTER last_known_addr { host: "127.22.167.125" port: 37681 } } peers { permanent_uuid: "01dd5541dbad45308cb81844e1567391" member_type: VOTER last_known_addr { host: "127.22.167.124" port: 38931 } }
I20260810 04:45:39.671386 29385 raft_consensus.cc:348] T 00000000000000000000000000000000 P 46ef2267d42c436eb2dbfc72ec315e86 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46ef2267d42c436eb2dbfc72ec315e86" member_type: VOTER last_known_addr { host: "127.22.167.126" port: 44793 } } peers { permanent_uuid: "354894974ce2462cbeea05e8f6e93798" member_type: VOTER last_known_addr { host: "127.22.167.125" port: 37681 } } peers { permanent_uuid: "01dd5541dbad45308cb81844e1567391" member_type: VOTER last_known_addr { host: "127.22.167.124" port: 38931 } }
I20260810 04:45:39.671514 29385 raft_consensus.cc:729] T 00000000000000000000000000000000 P 46ef2267d42c436eb2dbfc72ec315e86 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 46ef2267d42c436eb2dbfc72ec315e86, State: Initialized, Role: FOLLOWER
I20260810 04:45:39.671594 29643 sys_catalog.cc:564] T 00000000000000000000000000000000 P 01dd5541dbad45308cb81844e1567391 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 04:45:39.671581 29385 consensus_queue.cc:260] T 00000000000000000000000000000000 P 46ef2267d42c436eb2dbfc72ec315e86 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46ef2267d42c436eb2dbfc72ec315e86" member_type: VOTER last_known_addr { host: "127.22.167.126" port: 44793 } } peers { permanent_uuid: "354894974ce2462cbeea05e8f6e93798" member_type: VOTER last_known_addr { host: "127.22.167.125" port: 37681 } } peers { permanent_uuid: "01dd5541dbad45308cb81844e1567391" member_type: VOTER last_known_addr { host: "127.22.167.124" port: 38931 } }
I20260810 04:45:39.671679 23197 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 04:45:39.671831 29655 sys_catalog.cc:455] T 00000000000000000000000000000000 P 46ef2267d42c436eb2dbfc72ec315e86 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46ef2267d42c436eb2dbfc72ec315e86" member_type: VOTER last_known_addr { host: "127.22.167.126" port: 44793 } } peers { permanent_uuid: "354894974ce2462cbeea05e8f6e93798" member_type: VOTER last_known_addr { host: "127.22.167.125" port: 37681 } } peers { permanent_uuid: "01dd5541dbad45308cb81844e1567391" member_type: VOTER last_known_addr { host: "127.22.167.124" port: 38931 } } }
I20260810 04:45:39.671900 29385 sys_catalog.cc:564] T 00000000000000000000000000000000 P 46ef2267d42c436eb2dbfc72ec315e86 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 04:45:39.672005 29654 sys_catalog.cc:455] T 00000000000000000000000000000000 P 01dd5541dbad45308cb81844e1567391 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46ef2267d42c436eb2dbfc72ec315e86" member_type: VOTER last_known_addr { host: "127.22.167.126" port: 44793 } } peers { permanent_uuid: "354894974ce2462cbeea05e8f6e93798" member_type: VOTER last_known_addr { host: "127.22.167.125" port: 37681 } } peers { permanent_uuid: "01dd5541dbad45308cb81844e1567391" member_type: VOTER last_known_addr { host: "127.22.167.124" port: 38931 } } }
I20260810 04:45:39.672082 29654 sys_catalog.cc:458] T 00000000000000000000000000000000 P 01dd5541dbad45308cb81844e1567391 [sys.catalog]: This master's current role is: FOLLOWER
I20260810 04:45:39.672605 29341 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "46ef2267d42c436eb2dbfc72ec315e86"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46800
I20260810 04:45:39.672739 29341 raft_consensus.cc:482] T 00000000000000000000000000000000 P 46ef2267d42c436eb2dbfc72ec315e86 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260810 04:45:39.672787 29341 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 46ef2267d42c436eb2dbfc72ec315e86 [term 0 FOLLOWER]: Advancing to term 1
W20260810 04:45:39.673419 29690 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 01dd5541dbad45308cb81844e1567391: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260810 04:45:39.673468 29690 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260810 04:45:39.673573 29341 raft_consensus.cc:504] T 00000000000000000000000000000000 P 46ef2267d42c436eb2dbfc72ec315e86 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46ef2267d42c436eb2dbfc72ec315e86" member_type: VOTER last_known_addr { host: "127.22.167.126" port: 44793 } } peers { permanent_uuid: "354894974ce2462cbeea05e8f6e93798" member_type: VOTER last_known_addr { host: "127.22.167.125" port: 37681 } } peers { permanent_uuid: "01dd5541dbad45308cb81844e1567391" member_type: VOTER last_known_addr { host: "127.22.167.124" port: 38931 } }
I20260810 04:45:39.673629 29655 sys_catalog.cc:458] T 00000000000000000000000000000000 P 46ef2267d42c436eb2dbfc72ec315e86 [sys.catalog]: This master's current role is: FOLLOWER
I20260810 04:45:39.673804 29341 leader_election.cc:288] T 00000000000000000000000000000000 P 46ef2267d42c436eb2dbfc72ec315e86 [CANDIDATE]: Term 1 election: Requested vote from peers 354894974ce2462cbeea05e8f6e93798 (127.22.167.125:37681), 01dd5541dbad45308cb81844e1567391 (127.22.167.124:38931)
I20260810 04:45:39.674309 29601 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "46ef2267d42c436eb2dbfc72ec315e86" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "01dd5541dbad45308cb81844e1567391"
I20260810 04:45:39.674407 29601 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 01dd5541dbad45308cb81844e1567391 [term 0 FOLLOWER]: Advancing to term 1
I20260810 04:45:39.675190 29601 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 01dd5541dbad45308cb81844e1567391 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 46ef2267d42c436eb2dbfc72ec315e86 in term 1.
I20260810 04:45:39.675442 29291 leader_election.cc:302] T 00000000000000000000000000000000 P 46ef2267d42c436eb2dbfc72ec315e86 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01dd5541dbad45308cb81844e1567391, 46ef2267d42c436eb2dbfc72ec315e86; no voters:
I20260810 04:45:39.675575 29655 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 46ef2267d42c436eb2dbfc72ec315e86 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 04:45:39.675729 29655 raft_consensus.cc:686] T 00000000000000000000000000000000 P 46ef2267d42c436eb2dbfc72ec315e86 [term 1 LEADER]: Becoming Leader. State: Replica: 46ef2267d42c436eb2dbfc72ec315e86, State: Running, Role: LEADER
I20260810 04:45:39.675819 29655 consensus_queue.cc:237] T 00000000000000000000000000000000 P 46ef2267d42c436eb2dbfc72ec315e86 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46ef2267d42c436eb2dbfc72ec315e86" member_type: VOTER last_known_addr { host: "127.22.167.126" port: 44793 } } peers { permanent_uuid: "354894974ce2462cbeea05e8f6e93798" member_type: VOTER last_known_addr { host: "127.22.167.125" port: 37681 } } peers { permanent_uuid: "01dd5541dbad45308cb81844e1567391" member_type: VOTER last_known_addr { host: "127.22.167.124" port: 38931 } }
I20260810 04:45:39.673837 29527 master_runner.cc:186] Error getting master registration for 127.22.167.126:44793: OK, instance_id { permanent_uuid: "46ef2267d42c436eb2dbfc72ec315e86" instance_seqno: 1786337139317209 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260810 04:45:39.676296 29713 sys_catalog.cc:455] T 00000000000000000000000000000000 P 46ef2267d42c436eb2dbfc72ec315e86 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 46ef2267d42c436eb2dbfc72ec315e86. Latest consensus state: current_term: 1 leader_uuid: "46ef2267d42c436eb2dbfc72ec315e86" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46ef2267d42c436eb2dbfc72ec315e86" member_type: VOTER last_known_addr { host: "127.22.167.126" port: 44793 } } peers { permanent_uuid: "354894974ce2462cbeea05e8f6e93798" member_type: VOTER last_known_addr { host: "127.22.167.125" port: 37681 } } peers { permanent_uuid: "01dd5541dbad45308cb81844e1567391" member_type: VOTER last_known_addr { host: "127.22.167.124" port: 38931 } } }
I20260810 04:45:39.676419 29527 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260810 04:45:39.676431 29713 sys_catalog.cc:458] T 00000000000000000000000000000000 P 46ef2267d42c436eb2dbfc72ec315e86 [sys.catalog]: This master's current role is: LEADER
W20260810 04:45:39.677270 29704 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 46ef2267d42c436eb2dbfc72ec315e86: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260810 04:45:39.677348 29704 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260810 04:45:39.675028 29255 master_runner.cc:186] Error getting master registration for 127.22.167.125:37681: OK, instance_id { permanent_uuid: "354894974ce2462cbeea05e8f6e93798" instance_seqno: 1786337139476882 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260810 04:45:39.677673 29255 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260810 04:45:39.686234 29715 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 04:45:39.686554 29715 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 04:45:39.688081 29601 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 01dd5541dbad45308cb81844e1567391 [term 1 FOLLOWER]: Refusing update from remote peer 46ef2267d42c436eb2dbfc72ec315e86: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 04:45:39.688369 29713 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 46ef2267d42c436eb2dbfc72ec315e86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01dd5541dbad45308cb81844e1567391" member_type: VOTER last_known_addr { host: "127.22.167.124" port: 38931 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 04:45:39.691246 29654 sys_catalog.cc:455] T 00000000000000000000000000000000 P 01dd5541dbad45308cb81844e1567391 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 46ef2267d42c436eb2dbfc72ec315e86. Latest consensus state: current_term: 1 leader_uuid: "46ef2267d42c436eb2dbfc72ec315e86" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46ef2267d42c436eb2dbfc72ec315e86" member_type: VOTER last_known_addr { host: "127.22.167.126" port: 44793 } } peers { permanent_uuid: "354894974ce2462cbeea05e8f6e93798" member_type: VOTER last_known_addr { host: "127.22.167.125" port: 37681 } } peers { permanent_uuid: "01dd5541dbad45308cb81844e1567391" member_type: VOTER last_known_addr { host: "127.22.167.124" port: 38931 } } }
I20260810 04:45:39.691365 29654 sys_catalog.cc:458] T 00000000000000000000000000000000 P 01dd5541dbad45308cb81844e1567391 [sys.catalog]: This master's current role is: FOLLOWER
I20260810 04:45:39.691256 29651 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:45:39.677920 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38492 (local address 127.22.167.124:38931)
0810 04:45:39.677967 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0810 04:45:39.677969 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 04:45:39.680284 (+ 2315us) server_negotiation.cc:407] Connection header received
0810 04:45:39.680347 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:45:39.680348 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:45:39.680382 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:45:39.680439 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0810 04:45:39.681523 (+ 1084us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:45:39.682244 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:45:39.683731 (+ 1487us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:45:39.684044 (+ 313us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:45:39.684086 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:45:39.686117 (+ 2031us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 04:45:39.686119 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 04:45:39.686119 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 04:45:39.686157 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0810 04:45:39.688040 (+ 1883us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 04:45:39.689497 (+ 1457us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 04:45:39.689498 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 04:45:39.689498 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 04:45:39.689574 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 04:45:39.689927 (+ 353us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 04:45:39.689928 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 04:45:39.689929 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 04:45:39.690073 (+ 144us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 04:45:39.690186 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0810 04:45:39.691087 (+ 901us) server_negotiation.cc:299] Negotiation successful
0810 04:45:39.691169 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260810 04:45:39.691830 29389 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:45:39.674944 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46814 (local address 127.22.167.126:44793)
0810 04:45:39.674978 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0810 04:45:39.674979 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0810 04:45:39.678369 (+ 3390us) server_negotiation.cc:407] Connection header received
0810 04:45:39.678462 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:45:39.678463 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:45:39.678472 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:45:39.678499 (+ 27us) server_negotiation.cc:226] Negotiated authn=SASL
0810 04:45:39.679437 (+ 938us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:45:39.680132 (+ 695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:45:39.683634 (+ 3502us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:45:39.683926 (+ 292us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:45:39.683967 (+ 41us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:45:39.686768 (+ 2801us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 04:45:39.686769 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 04:45:39.686770 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 04:45:39.686776 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0810 04:45:39.688246 (+ 1470us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 04:45:39.689108 (+ 862us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 04:45:39.689109 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 04:45:39.689110 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 04:45:39.689183 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 04:45:39.690557 (+ 1374us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 04:45:39.690558 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 04:45:39.690558 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 04:45:39.690706 (+ 148us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 04:45:39.690779 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0810 04:45:39.691671 (+ 892us) server_negotiation.cc:299] Negotiation successful
0810 04:45:39.691745 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260810 04:45:39.693794 29713 sys_catalog.cc:455] T 00000000000000000000000000000000 P 46ef2267d42c436eb2dbfc72ec315e86 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "46ef2267d42c436eb2dbfc72ec315e86" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46ef2267d42c436eb2dbfc72ec315e86" member_type: VOTER last_known_addr { host: "127.22.167.126" port: 44793 } } peers { permanent_uuid: "354894974ce2462cbeea05e8f6e93798" member_type: VOTER last_known_addr { host: "127.22.167.125" port: 37681 } } peers { permanent_uuid: "01dd5541dbad45308cb81844e1567391" member_type: VOTER last_known_addr { host: "127.22.167.124" port: 38931 } } }
I20260810 04:45:39.693872 29713 sys_catalog.cc:458] T 00000000000000000000000000000000 P 46ef2267d42c436eb2dbfc72ec315e86 [sys.catalog]: This master's current role is: LEADER
I20260810 04:45:39.694368 29654 sys_catalog.cc:455] T 00000000000000000000000000000000 P 01dd5541dbad45308cb81844e1567391 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "46ef2267d42c436eb2dbfc72ec315e86" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46ef2267d42c436eb2dbfc72ec315e86" member_type: VOTER last_known_addr { host: "127.22.167.126" port: 44793 } } peers { permanent_uuid: "354894974ce2462cbeea05e8f6e93798" member_type: VOTER last_known_addr { host: "127.22.167.125" port: 37681 } } peers { permanent_uuid: "01dd5541dbad45308cb81844e1567391" member_type: VOTER last_known_addr { host: "127.22.167.124" port: 38931 } } }
I20260810 04:45:39.694445 29654 sys_catalog.cc:458] T 00000000000000000000000000000000 P 01dd5541dbad45308cb81844e1567391 [sys.catalog]: This master's current role is: FOLLOWER
I20260810 04:45:39.694458 29523 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:45:39.677997 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39624 (local address 127.22.167.125:37681)
0810 04:45:39.678044 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0810 04:45:39.678046 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 04:45:39.679277 (+ 1231us) server_negotiation.cc:407] Connection header received
0810 04:45:39.679338 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:45:39.679339 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:45:39.679349 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:45:39.679377 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0810 04:45:39.681952 (+ 2575us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:45:39.682661 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:45:39.684294 (+ 1633us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:45:39.685045 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:45:39.685089 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:45:39.690793 (+ 5704us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 04:45:39.690794 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 04:45:39.690795 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 04:45:39.690802 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 04:45:39.692704 (+ 1902us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 04:45:39.693292 (+ 588us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 04:45:39.693293 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 04:45:39.693294 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 04:45:39.693365 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 04:45:39.693716 (+ 351us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 04:45:39.693717 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 04:45:39.693717 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 04:45:39.693851 (+ 134us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 04:45:39.693917 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0810 04:45:39.694293 (+ 376us) server_negotiation.cc:299] Negotiation successful
0810 04:45:39.694370 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260810 04:45:39.694679 29721 mvcc.cc:205] Tried to move back new op lower bound from 7316836924159852544 to 7316836924112764928. Current Snapshot: MvccSnapshot[applied={T|T < 7316836924159852544}]
I20260810 04:45:39.695571 29726 mvcc.cc:205] Tried to move back new op lower bound from 7316836924159852544 to 7316836924112764928. Current Snapshot: MvccSnapshot[applied={T|T < 7316836924159852544}]
I20260810 04:45:39.696027 29715 catalog_manager.cc:1347] Generated new cluster ID: f6103e3f8d7245aa84e9e875c09be360
I20260810 04:45:39.696079 29715 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 04:45:39.707906 29715 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 04:45:39.708616 29715 catalog_manager.cc:1504] Loading token signing keys...
I20260810 04:45:39.709638 29514 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.22.167.124" port: 38931 } attempt: 3
Aug 10 04:45:39 dist-test-slave-bqcl krb5kdc[29233](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786337139, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.167.125@KRBTEST.COM for kudu/127.22.167.124@KRBTEST.COM
I20260810 04:45:39.718814 29516 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:45:39.709887 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.167.124:38931 (local address 127.0.0.1:38498)
0810 04:45:39.709967 (+ 80us) negotiation.cc:107] Waiting for socket to connect
0810 04:45:39.709977 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0810 04:45:39.710016 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 04:45:39.710273 (+ 257us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 04:45:39.710274 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 04:45:39.710446 (+ 172us) client_negotiation.cc:189] Negotiated authn=SASL
0810 04:45:39.710489 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:45:39.710491 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:45:39.711414 (+ 923us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:45:39.711415 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:45:39.712228 (+ 813us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 04:45:39.712231 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 04:45:39.712772 (+ 541us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 04:45:39.712773 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 04:45:39.712844 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:45:39.713677 (+ 833us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 04:45:39.713682 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0810 04:45:39.715310 (+ 1628us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 04:45:39.717645 (+ 2335us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 04:45:39.717647 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 04:45:39.717647 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 04:45:39.717866 (+ 219us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 04:45:39.718101 (+ 235us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 04:45:39.718102 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 04:45:39.718102 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 04:45:39.718181 (+ 79us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 04:45:39.718495 (+ 314us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 04:45:39.718496 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 04:45:39.718553 (+ 57us) client_negotiation.cc:769] Sending connection context
0810 04:45:39.718622 (+ 69us) client_negotiation.cc:240] Negotiation successful
0810 04:45:39.718714 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":60}
I20260810 04:45:39.719446 29651 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:45:39.710093 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38498 (local address 127.22.167.124:38931)
0810 04:45:39.710148 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0810 04:45:39.710150 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 04:45:39.710156 (+ 6us) server_negotiation.cc:407] Connection header received
0810 04:45:39.710175 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:45:39.710176 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:45:39.710185 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:45:39.710228 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0810 04:45:39.710573 (+ 345us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:45:39.711300 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:45:39.712328 (+ 1028us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:45:39.712639 (+ 311us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:45:39.712695 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:45:39.715415 (+ 2720us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 04:45:39.715416 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 04:45:39.715416 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 04:45:39.715422 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0810 04:45:39.717527 (+ 2105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 04:45:39.717955 (+ 428us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 04:45:39.717956 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 04:45:39.717956 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 04:45:39.718025 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 04:45:39.718259 (+ 234us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 04:45:39.718260 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 04:45:39.718260 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 04:45:39.718399 (+ 139us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 04:45:39.718455 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0810 04:45:39.719279 (+ 824us) server_negotiation.cc:299] Negotiation successful
0810 04:45:39.719355 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260810 04:45:39.720983 29514 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 354894974ce2462cbeea05e8f6e93798: Bootstrap starting.
I20260810 04:45:39.721735 29514 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 354894974ce2462cbeea05e8f6e93798: Neither blocks nor log segments found. Creating new log.
I20260810 04:45:39.722050 29514 log.cc:824] T 00000000000000000000000000000000 P 354894974ce2462cbeea05e8f6e93798: Log is configured to *not* fsync() on all Append() calls
I20260810 04:45:39.723294 29514 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 354894974ce2462cbeea05e8f6e93798: No bootstrap required, opened a new log
I20260810 04:45:39.727579 29514 raft_consensus.cc:348] T 00000000000000000000000000000000 P 354894974ce2462cbeea05e8f6e93798 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46ef2267d42c436eb2dbfc72ec315e86" member_type: VOTER last_known_addr { host: "127.22.167.126" port: 44793 } } peers { permanent_uuid: "354894974ce2462cbeea05e8f6e93798" member_type: VOTER last_known_addr { host: "127.22.167.125" port: 37681 } } peers { permanent_uuid: "01dd5541dbad45308cb81844e1567391" member_type: VOTER last_known_addr { host: "127.22.167.124" port: 38931 } }
I20260810 04:45:39.727731 29514 raft_consensus.cc:729] T 00000000000000000000000000000000 P 354894974ce2462cbeea05e8f6e93798 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 354894974ce2462cbeea05e8f6e93798, State: Initialized, Role: FOLLOWER
I20260810 04:45:39.727813 29514 consensus_queue.cc:260] T 00000000000000000000000000000000 P 354894974ce2462cbeea05e8f6e93798 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46ef2267d42c436eb2dbfc72ec315e86" member_type: VOTER last_known_addr { host: "127.22.167.126" port: 44793 } } peers { permanent_uuid: "354894974ce2462cbeea05e8f6e93798" member_type: VOTER last_known_addr { host: "127.22.167.125" port: 37681 } } peers { permanent_uuid: "01dd5541dbad45308cb81844e1567391" member_type: VOTER last_known_addr { host: "127.22.167.124" port: 38931 } }
I20260810 04:45:39.728122 29514 sys_catalog.cc:564] T 00000000000000000000000000000000 P 354894974ce2462cbeea05e8f6e93798 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 04:45:39.728215 29464 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 354894974ce2462cbeea05e8f6e93798 [term 0 FOLLOWER]: Advancing to term 1
I20260810 04:45:39.729046 29464 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 354894974ce2462cbeea05e8f6e93798 [term 1 FOLLOWER]: Refusing update from remote peer 46ef2267d42c436eb2dbfc72ec315e86: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 04:45:39.729225 29466 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "46ef2267d42c436eb2dbfc72ec315e86" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "354894974ce2462cbeea05e8f6e93798"
I20260810 04:45:39.729329 29713 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 46ef2267d42c436eb2dbfc72ec315e86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "354894974ce2462cbeea05e8f6e93798" member_type: VOTER last_known_addr { host: "127.22.167.125" port: 37681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 04:45:39.730307 29466 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 354894974ce2462cbeea05e8f6e93798 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 46ef2267d42c436eb2dbfc72ec315e86 in term 1.
I20260810 04:45:39.728121 29736 sys_catalog.cc:455] T 00000000000000000000000000000000 P 354894974ce2462cbeea05e8f6e93798 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46ef2267d42c436eb2dbfc72ec315e86" member_type: VOTER last_known_addr { host: "127.22.167.126" port: 44793 } } peers { permanent_uuid: "354894974ce2462cbeea05e8f6e93798" member_type: VOTER last_known_addr { host: "127.22.167.125" port: 37681 } } peers { permanent_uuid: "01dd5541dbad45308cb81844e1567391" member_type: VOTER last_known_addr { host: "127.22.167.124" port: 38931 } } }
I20260810 04:45:39.730638 29736 sys_catalog.cc:458] T 00000000000000000000000000000000 P 354894974ce2462cbeea05e8f6e93798 [sys.catalog]: This master's current role is: FOLLOWER
I20260810 04:45:39.735909 29736 sys_catalog.cc:455] T 00000000000000000000000000000000 P 354894974ce2462cbeea05e8f6e93798 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 46ef2267d42c436eb2dbfc72ec315e86. Latest consensus state: current_term: 1 leader_uuid: "46ef2267d42c436eb2dbfc72ec315e86" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46ef2267d42c436eb2dbfc72ec315e86" member_type: VOTER last_known_addr { host: "127.22.167.126" port: 44793 } } peers { permanent_uuid: "354894974ce2462cbeea05e8f6e93798" member_type: VOTER last_known_addr { host: "127.22.167.125" port: 37681 } } peers { permanent_uuid: "01dd5541dbad45308cb81844e1567391" member_type: VOTER last_known_addr { host: "127.22.167.124" port: 38931 } } }
I20260810 04:45:39.736044 29736 sys_catalog.cc:458] T 00000000000000000000000000000000 P 354894974ce2462cbeea05e8f6e93798 [sys.catalog]: This master's current role is: FOLLOWER
I20260810 04:45:39.736120 29736 sys_catalog.cc:455] T 00000000000000000000000000000000 P 354894974ce2462cbeea05e8f6e93798 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "46ef2267d42c436eb2dbfc72ec315e86" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46ef2267d42c436eb2dbfc72ec315e86" member_type: VOTER last_known_addr { host: "127.22.167.126" port: 44793 } } peers { permanent_uuid: "354894974ce2462cbeea05e8f6e93798" member_type: VOTER last_known_addr { host: "127.22.167.125" port: 37681 } } peers { permanent_uuid: "01dd5541dbad45308cb81844e1567391" member_type: VOTER last_known_addr { host: "127.22.167.124" port: 38931 } } }
I20260810 04:45:39.736210 29736 sys_catalog.cc:458] T 00000000000000000000000000000000 P 354894974ce2462cbeea05e8f6e93798 [sys.catalog]: This master's current role is: FOLLOWER
I20260810 04:45:39.736292 29757 mvcc.cc:205] Tried to move back new op lower bound from 7316836924236697600 to 7316836924112764928. Current Snapshot: MvccSnapshot[applied={T|T < 7316836924159852544}]
W20260810 04:45:39.738402 29766 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 354894974ce2462cbeea05e8f6e93798: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260810 04:45:39.738457 29766 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260810 04:45:39.740751 29713 sys_catalog.cc:455] T 00000000000000000000000000000000 P 46ef2267d42c436eb2dbfc72ec315e86 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "46ef2267d42c436eb2dbfc72ec315e86" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46ef2267d42c436eb2dbfc72ec315e86" member_type: VOTER last_known_addr { host: "127.22.167.126" port: 44793 } } peers { permanent_uuid: "354894974ce2462cbeea05e8f6e93798" member_type: VOTER last_known_addr { host: "127.22.167.125" port: 37681 } } peers { permanent_uuid: "01dd5541dbad45308cb81844e1567391" member_type: VOTER last_known_addr { host: "127.22.167.124" port: 38931 } } }
I20260810 04:45:39.740846 29713 sys_catalog.cc:458] T 00000000000000000000000000000000 P 46ef2267d42c436eb2dbfc72ec315e86 [sys.catalog]: This master's current role is: LEADER
I20260810 04:45:39.742286 29715 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 46ef2267d42c436eb2dbfc72ec315e86: Generated new TSK 0
I20260810 04:45:39.742506 29715 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260810 04:45:39.764909 29523 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 04:45:39.763214 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39640 (local address 127.22.167.125:37681)
0810 04:45:39.763255 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0810 04:45:39.763257 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 04:45:39.763955 (+ 698us) server_negotiation.cc:407] Connection header received
0810 04:45:39.764180 (+ 225us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:45:39.764181 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:45:39.764192 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:45:39.764246 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0810 04:45:39.764841 (+ 595us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:39640: BlockingRecv error: recv got EOF from 127.0.0.1:39640 (error 108)
Metrics: {"server-negotiator.queue_time_us":20}
W20260810 04:45:39.765625 29389 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 04:45:39.763674 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46816 (local address 127.22.167.126:44793)
0810 04:45:39.763712 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0810 04:45:39.763714 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 04:45:39.765125 (+ 1411us) server_negotiation.cc:407] Connection header received
0810 04:45:39.765217 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:45:39.765218 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:45:39.765227 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:45:39.765273 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0810 04:45:39.765564 (+ 291us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:46816: BlockingRecv error: recv got EOF from 127.0.0.1:46816 (error 108)
Metrics: {"server-negotiator.queue_time_us":20}
W20260810 04:45:39.773377 29651 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 04:45:39.766336 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38510 (local address 127.22.167.124:38931)
0810 04:45:39.772683 (+ 6347us) server_negotiation.cc:206] Beginning negotiation
0810 04:45:39.772685 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 04:45:39.772696 (+ 11us) server_negotiation.cc:407] Connection header received
0810 04:45:39.772719 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:45:39.772720 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:45:39.772731 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:45:39.772794 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0810 04:45:39.773320 (+ 526us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38510: BlockingRecv error: recv got EOF from 127.0.0.1:38510 (error 108)
Metrics: {"server-negotiator.queue_time_us":6322}
W20260810 04:45:39.779130 29523 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 04:45:39.778361 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39652 (local address 127.22.167.125:37681)
0810 04:45:39.778397 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0810 04:45:39.778399 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 04:45:39.778650 (+ 251us) server_negotiation.cc:407] Connection header received
0810 04:45:39.778725 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:45:39.778726 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:45:39.778743 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:45:39.778804 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0810 04:45:39.779080 (+ 276us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:39652: BlockingRecv error: recv got EOF from 127.0.0.1:39652 (error 108)
Metrics: {"server-negotiator.queue_time_us":17}
W20260810 04:45:39.782164 29523 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 04:45:39.781349 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39660 (local address 127.22.167.125:37681)
0810 04:45:39.781382 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0810 04:45:39.781384 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 04:45:39.781610 (+ 226us) server_negotiation.cc:407] Connection header received
0810 04:45:39.781709 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:45:39.781710 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:45:39.781721 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:45:39.781748 (+ 27us) server_negotiation.cc:226] Negotiated authn=SASL
0810 04:45:39.782079 (+ 331us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:39660: BlockingRecv error: recv got EOF from 127.0.0.1:39660 (error 108)
Metrics: {"server-negotiator.queue_time_us":18}
W20260810 04:45:39.791148 29523 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 04:45:39.790372 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39662 (local address 127.22.167.125:37681)
0810 04:45:39.790407 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0810 04:45:39.790408 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0810 04:45:39.790662 (+ 254us) server_negotiation.cc:407] Connection header received
0810 04:45:39.790757 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:45:39.790758 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:45:39.790768 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:45:39.790795 (+ 27us) server_negotiation.cc:226] Negotiated authn=SASL
0810 04:45:39.791104 (+ 309us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:39662: BlockingRecv error: recv got EOF from 127.0.0.1:39662 (error 108)
Metrics: {"server-negotiator.queue_time_us":17}
W20260810 04:45:39.797077 29523 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 04:45:39.796291 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39664 (local address 127.22.167.125:37681)
0810 04:45:39.796323 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0810 04:45:39.796325 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 04:45:39.796589 (+ 264us) server_negotiation.cc:407] Connection header received
0810 04:45:39.796684 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:45:39.796685 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:45:39.796695 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:45:39.796723 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0810 04:45:39.797036 (+ 313us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:39664: BlockingRecv error: recv got EOF from 127.0.0.1:39664 (error 108)
Metrics: {"server-negotiator.queue_time_us":17}
W20260810 04:45:39.806989 29523 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 04:45:39.806223 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39676 (local address 127.22.167.125:37681)
0810 04:45:39.806254 (+ 31us) server_negotiation.cc:206] Beginning negotiation
0810 04:45:39.806255 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0810 04:45:39.806487 (+ 232us) server_negotiation.cc:407] Connection header received
0810 04:45:39.806588 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:45:39.806589 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:45:39.806600 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:45:39.806628 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0810 04:45:39.806941 (+ 313us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:39676: BlockingRecv error: recv got EOF from 127.0.0.1:39676 (error 108)
Metrics: {"server-negotiator.queue_time_us":15}
W20260810 04:45:39.824954 29523 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 04:45:39.824172 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39692 (local address 127.22.167.125:37681)
0810 04:45:39.824206 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0810 04:45:39.824207 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0810 04:45:39.824442 (+ 235us) server_negotiation.cc:407] Connection header received
0810 04:45:39.824530 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:45:39.824531 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:45:39.824541 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:45:39.824580 (+ 39us) server_negotiation.cc:226] Negotiated authn=SASL
0810 04:45:39.824905 (+ 325us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:39692: BlockingRecv error: recv got EOF from 127.0.0.1:39692 (error 108)
Metrics: {"server-negotiator.queue_time_us":16}
W20260810 04:45:39.858937 29523 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 04:45:39.858225 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39694 (local address 127.22.167.125:37681)
0810 04:45:39.858268 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0810 04:45:39.858271 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 04:45:39.858426 (+ 155us) server_negotiation.cc:407] Connection header received
0810 04:45:39.858490 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:45:39.858491 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:45:39.858502 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:45:39.858553 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0810 04:45:39.858877 (+ 324us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:39694: BlockingRecv error: recv got EOF from 127.0.0.1:39694 (error 108)
Metrics: {"server-negotiator.queue_time_us":20}
W20260810 04:45:39.924675 29523 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 04:45:39.923987 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39698 (local address 127.22.167.125:37681)
0810 04:45:39.924057 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0810 04:45:39.924059 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 04:45:39.924153 (+ 94us) server_negotiation.cc:407] Connection header received
0810 04:45:39.924213 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:45:39.924214 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:45:39.924226 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:45:39.924271 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0810 04:45:39.924616 (+ 345us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:39698: BlockingRecv error: recv got EOF from 127.0.0.1:39698 (error 108)
Metrics: {"server-negotiator.queue_time_us":42}
W20260810 04:45:40.054508 29523 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 04:45:40.053783 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39702 (local address 127.22.167.125:37681)
0810 04:45:40.053870 (+ 87us) server_negotiation.cc:206] Beginning negotiation
0810 04:45:40.053872 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 04:45:40.053953 (+ 81us) server_negotiation.cc:407] Connection header received
0810 04:45:40.054026 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:45:40.054027 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:45:40.054039 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:45:40.054086 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0810 04:45:40.054447 (+ 361us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:39702: BlockingRecv error: recv got EOF from 127.0.0.1:39702 (error 108)
Metrics: {"server-negotiator.queue_time_us":60}
W20260810 04:45:40.312300 29523 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 04:45:40.311696 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39704 (local address 127.22.167.125:37681)
0810 04:45:40.311751 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0810 04:45:40.311753 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 04:45:40.311788 (+ 35us) server_negotiation.cc:407] Connection header received
0810 04:45:40.311854 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:45:40.311855 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:45:40.311866 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:45:40.311921 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0810 04:45:40.312231 (+ 310us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:39704: BlockingRecv error: recv got EOF from 127.0.0.1:39704 (error 108)
Metrics: {"server-negotiator.queue_time_us":29}
I20260810 04:45:40.673918 29690 catalog_manager.cc:1259] Loaded cluster ID: f6103e3f8d7245aa84e9e875c09be360
I20260810 04:45:40.673990 29690 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 01dd5541dbad45308cb81844e1567391: loading cluster ID for follower catalog manager: success
I20260810 04:45:40.674852 29690 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 01dd5541dbad45308cb81844e1567391: acquiring CA information for follower catalog manager: success
I20260810 04:45:40.675168 29690 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 01dd5541dbad45308cb81844e1567391: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260810 04:45:40.738857 29766 catalog_manager.cc:1259] Loaded cluster ID: f6103e3f8d7245aa84e9e875c09be360
I20260810 04:45:40.738926 29766 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 354894974ce2462cbeea05e8f6e93798: loading cluster ID for follower catalog manager: success
I20260810 04:45:40.739708 29766 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 354894974ce2462cbeea05e8f6e93798: acquiring CA information for follower catalog manager: success
I20260810 04:45:40.740000 29766 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 354894974ce2462cbeea05e8f6e93798: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260810 04:45:40.827765 29899 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:45:40.825388 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39716 (local address 127.22.167.125:37681)
0810 04:45:40.825491 (+ 103us) server_negotiation.cc:206] Beginning negotiation
0810 04:45:40.825493 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 04:45:40.825502 (+ 9us) server_negotiation.cc:407] Connection header received
0810 04:45:40.825565 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:45:40.825566 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:45:40.825614 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:45:40.825683 (+ 69us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 04:45:40.825799 (+ 116us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:45:40.826315 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:45:40.827080 (+ 765us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:45:40.827283 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:45:40.827327 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:45:40.827462 (+ 135us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 04:45:40.827586 (+ 124us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 04:45:40.827647 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0810 04:45:40.827662 (+ 15us) server_negotiation.cc:299] Negotiation successful
0810 04:45:40.827678 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":85,"thread_start_us":56,"threads_started":1}
I20260810 04:45:40.831483 29908 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:45:40.829484 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38524 (local address 127.22.167.124:38931)
0810 04:45:40.829577 (+ 93us) server_negotiation.cc:206] Beginning negotiation
0810 04:45:40.829578 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0810 04:45:40.829587 (+ 9us) server_negotiation.cc:407] Connection header received
0810 04:45:40.829623 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:45:40.829624 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:45:40.829658 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:45:40.829719 (+ 61us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 04:45:40.829804 (+ 85us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:45:40.830330 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:45:40.830955 (+ 625us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:45:40.831134 (+ 179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:45:40.831174 (+ 40us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:45:40.831269 (+ 95us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 04:45:40.831353 (+ 84us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 04:45:40.831398 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0810 04:45:40.831409 (+ 11us) server_negotiation.cc:299] Negotiation successful
0810 04:45:40.831423 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72,"thread_start_us":49,"threads_started":1}
I20260810 04:45:40.835505 29908 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:45:40.833001 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38540 (local address 127.22.167.124:38931)
0810 04:45:40.833034 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0810 04:45:40.833036 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 04:45:40.833224 (+ 188us) server_negotiation.cc:407] Connection header received
0810 04:45:40.833284 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:45:40.833285 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:45:40.833292 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:45:40.833336 (+ 44us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 04:45:40.833430 (+ 94us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:45:40.833907 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:45:40.834853 (+ 946us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:45:40.835131 (+ 278us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:45:40.835185 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:45:40.835305 (+ 120us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 04:45:40.835334 (+ 29us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 04:45:40.835372 (+ 38us) server_negotiation.cc:1035] Waiting for connection context
0810 04:45:40.835422 (+ 50us) server_negotiation.cc:299] Negotiation successful
0810 04:45:40.835441 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260810 04:45:40.835882 29899 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:45:40.832993 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39720 (local address 127.22.167.125:37681)
0810 04:45:40.833029 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0810 04:45:40.833030 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0810 04:45:40.833038 (+ 8us) server_negotiation.cc:407] Connection header received
0810 04:45:40.833060 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:45:40.833061 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:45:40.833070 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:45:40.833120 (+ 50us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 04:45:40.833273 (+ 153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:45:40.833945 (+ 672us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:45:40.834788 (+ 843us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:45:40.835072 (+ 284us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:45:40.835132 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:45:40.835652 (+ 520us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 04:45:40.835682 (+ 30us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 04:45:40.835730 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0810 04:45:40.835795 (+ 65us) server_negotiation.cc:299] Negotiation successful
0810 04:45:40.835818 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260810 04:45:40.835999 29918 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 04:45:40.832902 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46826 (local address 127.22.167.126:44793)
0810 04:45:40.833195 (+ 293us) server_negotiation.cc:206] Beginning negotiation
0810 04:45:40.833197 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 04:45:40.833204 (+ 7us) server_negotiation.cc:407] Connection header received
0810 04:45:40.833247 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 04:45:40.833247 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 04:45:40.833298 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 04:45:40.833368 (+ 70us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 04:45:40.833541 (+ 173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:45:40.834292 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:45:40.835140 (+ 848us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 04:45:40.835352 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 04:45:40.835405 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 04:45:40.835590 (+ 185us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 04:45:40.835699 (+ 109us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 04:45:40.835752 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0810 04:45:40.835907 (+ 155us) server_negotiation.cc:299] Negotiation successful
0810 04:45:40.835933 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":272,"thread_start_us":43,"threads_started":1}
I20260810 04:45:40.837924 23197 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskemv_AM/build/release/bin/kudu with pid 29255
I20260810 04:45:40.851843 23197 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskemv_AM/build/release/bin/kudu with pid 29396
I20260810 04:45:40.861478 23197 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskemv_AM/build/release/bin/kudu with pid 29527
2026-08-10T04:45:40Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1 (2979 ms)
[----------] 2 tests from ConnectToFollowerMasterTest (5904 ms total)
[----------] Global test environment tear-down
[==========] 56 tests from 4 test suites ran. (277489 ms total)
[ PASSED ] 55 tests.
[ FAILED ] 1 test, listed below:
[ FAILED ] SecurityITest.TestJwtMiniCluster
1 FAILED TEST